OpenAI threatens to revoke o1 access for asking it about its chain of thought (twitter.com)

Article summary

OpenAI has been emailing users who ask about the chain of thought of their model, threatening to revoke access if they continue to do so. The email is in response to repeated inquiries about the model's thought process. This has sparked a discussion about the model's capabilities and limitations. The model's performance on certain tasks, such as STEM-related queries, is notably better than on others, like English composition.

Main themes

• AI model capabilities
• Chain of thought
• Language understanding
• Model training data
• AI-generated content

What commenters say

• The model's chain of thought is not truly a 'thought process', but rather a combination of writings from various people thrown together.
• OpenAI may be hiding the chain of thought to prevent others from training their own models on the output.
• The model's performance is due to its training data, which may include bespoke writing from contractors, rather than any true 'innovation' or 'magic'.
• The use of non-native English speakers in training the model may be responsible for its distinctive writing style, which some find 'corny' or 'abnormal'.
• The model's limitations, such as its performance on English composition tasks, are a result of its focus on other areas, like STEM-related queries.
• The distinction between 'sampling' and 'causality' is important in understanding how the model works, with some arguing that the model is simply learning distributions rather than truly 'thinking'.
• The use of human contractors to generate training data raises questions about the value and authenticity of the model's output.
• The model's 'reasoning' abilities are not truly innovative, but rather a result of its training data and algorithms.