Monte-Carlo graph search from first principles (github.com)

Article summary

The article discusses Monte-Carlo graph search, a technique used in game-playing AI, which trades compute for memory by hashing game states to check for visited nodes. This approach can save both compute and memory by avoiding duplicate nodes in the search space. The article explores the background and importance of this technique, and its relation to Tree Search. The discussion reveals that the original Monte-Carlo Tree Search algorithm had randomness, but current implementations often use neural network evaluations instead.

Main themes

• Monte-Carlo graph search
• Game-playing AI
• Tree Search
• Neural network evaluations
• Hashing game states
• Optimizing search algorithms

What commenters say

• The Monte-Carlo Tree Search algorithm has evolved to use neural network evaluations instead of randomness, making it deterministic.
• Graph search can save both compute and memory by avoiding duplicate nodes in the search space, but its effectiveness depends on the specific use case.
• The name 'Monte-Carlo' is misleading, as the algorithm is no longer random, and a different name might be more appropriate.
• The use of neural networks as heuristics can be seen as a form of biased random number generation, similar to Monte Carlo methods.
• The technique is particularly useful in games like Go, where repeated positions can occur frequently, but its applicability to other games like chess is debated.
• Some argue that the algorithm's determinism is not a limitation, as it can still provide good coverage of the search space without true randomness.
• Others suggest that the algorithm's performance can be improved by jointly learning the state representation and search algorithm, rather than using a fixed representation.
• The article's approach is seen as a promising step forward in optimizing search algorithms, but its impact on the field of AI is still a topic of discussion.