Apple’s new Transformer-powered predictive text model (jackcook.com)

Article summary

Apple has introduced a new predictive text feature powered by a Transformer language model, which provides suggestions as users type. The model is based on a GPT-2 architecture and has around 34 million parameters. The feature is designed to run quickly and efficiently on Apple devices, making it suitable for tasks like text messaging. The model's performance and limitations are explored in the article, including its ability to generate text and its reliance on a unique vocabulary set.

Main themes

• predictive text models
• autocorrect
• machine learning
• language support
• voice-to-text
• keyboard input
• algorithm optimization
• user experience

What commenters say

• The new predictive text feature is seen as an improvement, but some users are frustrated with the current state of autocorrect on their devices.
• Some users believe that the predictive text model should be able to learn from their typing history and improve over time.
• Others are skeptical about the effectiveness of machine learning-based autocorrect models and prefer simpler heuristic-based approaches.
• There are mixed opinions on the performance of voice-to-text features, with some users finding them to be very accurate and others experiencing significant errors.
• The use of multi-language support on a single keyboard is a desirable feature for some users, but others find it to be a source of errors.
• Some users have noticed a decline in the accuracy of autocorrect and predictive text features over time, possibly due to changes in algorithms or optimization for other metrics.
• The importance of accent and language support is highlighted, with some users experiencing better results with certain accents or languages.
• There are differing opinions on the trade-off between model complexity and performance, with some users preferring simpler models and others advocating for more advanced approaches.