Beyond self-attention: How a small language model predicts the next token (shyam.blog)

Article summary

The article explores how a small language model, trained on a dataset of Shakespeare's plays, predicts the next token in a sequence. The author proposes a theory that each transformer block learns to associate a given prompt with a class of strings found in the training corpus, and outputs a distribution of tokens that follow those strings. The model is able to produce plausible, albeit nonsensical, text that mimics the style of Shakespeare. The author also provides an imperative code implementation of their proposed explanation, which produces similar outputs to the original model.

Main themes

• language models
• transformer architecture
• copyright law
• explainability
• abstraction
• efficiency
• training data
• human understanding
• AI development
• regulation

What commenters say

• The article's findings may have implications for copyright law, as they suggest that language models may be retaining and regurgitating training data rather than truly learning from it.
• The efficiency of the proposed approach is questioned, with some arguing that it is not more efficient than running the original model.
• The role of training data in language models is a subject of debate, with some arguing that it is well-known and fundamental, while others see it as a potentially problematic issue.
• The use of abstraction and high-level explanations is seen as useful for understanding complex systems like language models, but others argue that it can be misleading or oversimplified.
• The relationship between language models and human understanding is complex, with some arguing that trying to understand these models at a high level of abstraction is a dead end.
• The article's findings may have implications for the development of more transparent and explainable AI systems.
• The use of copyright law to regulate language models is seen as potentially problematic, with some arguing that it is a badly-negotiated bargain that favors intermediaries over artists and the public.