Deep Learning Is Applied Topology (theahura.substack.com)

Article summary

The article discusses the connection between deep learning and topology, suggesting that deep learning can be viewed as applied topology. It explains how neural networks can be seen as transforming and manipulating surfaces, and how this relates to the concept of manifolds. The article also touches on the idea that everything lives on a manifold, and that neural networks can be used to discover and represent these manifolds. This perspective is presented as a way to understand and improve deep learning models.

Main themes

• Deep Learning
• Topology
• Manifolds
• Linear Algebra
• Geometric Deep Learning
• Theory vs. Practice
• Interdisciplinary Connections
• Predictive Theory

What commenters say

• Deep learning is not truly based on topology, but rather on linear algebra and other mathematical concepts.
• The connection between deep learning and topology is useful for understanding and improving models, but it is not a direct or exact relationship.
• The idea that data lives on a manifold is an approximation or a useful fiction, rather than a literal truth.
• The lack of a predictive theory for deep learning means that it is still a largely empirical field, with models being discovered through trial and error rather than being derived from first principles.
• The connection between deep learning and other fields, such as statistical mechanics and control theory, is an area of active research and potential insight.
• The use of geometric and topological concepts in deep learning is a useful tool for understanding and improving models, but it is not a replacement for a deeper theoretical understanding.
• The current state of deep learning is comparable to alchemy, with models being developed through trial and error rather than being derived from a underlying theory.
• The development of a predictive theory for deep learning is an active area of research, with some progress being made in understanding the underlying mechanisms and principles.