Generative Models: What do they know? Do they know things? Let's find out (intrinsic-lora.github.io)

Article summary

Researchers have developed a model-agnostic approach called Low-Rank Adaptation (LoRA) to recover intrinsic knowledge from generative models, including Autoregressive, GANs, and Diffusion models. This approach allows for the recovery of fundamental intrinsic images, such as normals, depth, albedo, and shading, directly from the models' learned representations. The method is lightweight and requires minimal learnable parameters and labeled data. The findings indicate a positive correlation between the generative model's quality and the accuracy of the recovered intrinsics.

Main themes

• generative models
• intrinsic knowledge
• understanding and intelligence
• qualia and subjective experience
• human vs artificial intelligence
• learning and training
• math and complexity
• interpretability and explainability

What commenters say

• Some argue that generative models learn humanly interpretable representations, while others claim they are just manipulating complex math without true understanding.
• The ability of models to produce correct output most of the time does not necessarily prove understanding, according to some commentators.
• Others propose that understanding can be defined as having a working model of a concept that adapts to new configurations of inputs reliably, even if not always correctly.
• A few commentators suggest that the concept of understanding is poorly defined and may not be strictly binary, with different people understanding things to varying degrees.
• Some argue that models do not truly understand because they do not have qualia, or subjective experiences, and are limited to processing and generating text or images based on their training data.
• Others counter that the presence or absence of qualia is difficult to determine and may not be a necessary condition for understanding.
• The question of whether models can be said to understand concepts like multiplication is also debated, with some arguing that true understanding requires the ability to train oneself and build on existing knowledge.
• The relationship between human intelligence and artificial intelligence is also discussed, with some commentators drawing parallels between the two and others highlighting their differences.