History LLMs: Models trained exclusively on pre-1913 texts (github.com)

Article summary

The article discusses the development of History LLMs, large language models trained exclusively on pre-1913 texts, which can provide a window into the past and enable research in the humanities and social sciences. These models are time-locked, meaning they do not have access to information beyond their knowledge-cutoff date, and can be used to explore discourse patterns and understand historical views. The project aims to make the models accessible to researchers, teachers, and the broader public, while also acknowledging the potential for the models to reproduce racist, misogynistic, and other problematic views present in historical texts.

Main themes

• Historical language models
• Time-locked training data
• Humanities and social sciences research
• Bias and problematic views in historical texts
• LLM capabilities and limitations
• Cognitive science and human thought

What commenters say

• The development of History LLMs can provide a unique perspective on historical events and cultural norms by allowing researchers to engage in open-ended dialogue with the past.
• Some commenters argue that LLMs are simply advanced autocomplete engines and do not truly think or understand language, while others see them as more complex systems capable of solving new problems.
• The comparison between human thought and LLMs is seen as both insightful and misleading, with some arguing that it oversimplifies the complexity of human cognition and others finding it a useful framework for understanding LLM capabilities.
• There is disagreement over the significance of LLMs' ability to generate text that is coherent and contextually relevant, with some seeing it as a major breakthrough and others as a limited achievement.
• Some commenters believe that LLMs can be seen as a form of 'autocomplete' that is similar to human thought processes, while others strongly disagree and argue that this comparison is unfounded and misleading.
• The use of reinforcement learning and other training techniques is seen as a way to improve LLMs' performance and capabilities, but also raises questions about their potential biases and limitations.
• There is a need for humility and caution when evaluating the capabilities and limitations of LLMs, and for recognizing the complexity and nuance of human cognition and language use.