Karpathy’s Pelican (twitter.com)

Article summary

Andrej Karpathy experimented with an LLM, asking it to generate a 3D animation of a scene from the Lord of the Rings based on a text prompt. The LLM produced 5500 lines of code and a render of the scene, although it was somewhat janky. Karpathy notes that LLMs have the potential to create custom worlds and games, but are currently limited by their inability to efficiently perceive videos or play games. This limitation hinders their ability to audit their own work and create more complex outputs.

Main themes

• LLM capabilities
• AI-generated content
• Game development
• Personalized entertainment
• AI limitations

What commenters say

• The current state of AI image generation is not nearly perfect and has significant limitations, such as poor spatial awareness and inability to update images according to user vision.
• Hyper-personalized entertainment may not be desirable or practical, as people often lack clear ideas about what they want and may not be interested in custom content.
• The use of AI-generated content can lead to a homogenization of artistic expression, making it difficult for companies to stand out and potentially leading to a resurgence of human creativity.
• Some argue that the pursuit of hyper-personalized entertainment is a gimmick that will lose appeal after a short period, while others see it as a potential game-changer for the entertainment industry.
• There is a concern that the narrative around AI development is being influenced by the interests of employers and marketers, rather than a genuine pursuit of progress.
• The ability of LLMs to take video as input is a significant open problem that needs to be addressed in order to unlock more complex and interactive applications.