Talkie vs Talkie

Talkie is a 13-billion parameter language model trained exclusively on English-language texts published before 1931 — the largest vintage language model built to date. Created by researchers Nick Levine, David Duvenaud (University of Toronto), and Alec Radford (of GPT and DALL-E fame), it represents a novel approach to understanding what training data really does to a model. The research insight is elegant: modern LLMs are so thoroughly contaminated by modern internet data (directly or through distillation) that it's nearly impossible to isolate what the model "knows" from what it absorbed during training. Talkie solves this by hard-cutting the training corpus at 1931 — predating digital computers entirely. This lets the team run controlled experiments impossible with contemporary models, such as teaching the model to write Python from examples alone and measuring how quickly it generalizes. Talkie was trained on ~260 billion tokens of historical text and fine-tuned using direct preference optimization with Claude as judge on structured historical documents (etiquette manuals, letter-writing guides). It's openly available on Hugging Face for research use. It also happens to produce wonderfully formal, slightly anachronistic prose.

Talkie is a 13-billion-parameter language model with an unusual constraint: it was trained exclusively on text written before 1931. That means no internet, no Wikipedia, no modern code — just 260 billion tokens of books, newspapers, journals, patents, and case law from the pre-modern era. The result is a "vintage" LLM that speaks like it's from the early 20th century and has zero knowledge of anything after its cutoff. The model was built by Nick Levine, David Duvenaud, and Alec Radford (yes, one of the original GPT authors) with support from Anthropic and Coefficient Giving. The scientific motivation is rigorous: Talkie enables researchers to cleanly test how models generalize to unfamiliar tasks from examples alone (since it's never seen Python), study future prediction capabilities without data leakage, and understand how training data diversity shapes model dispositions and values. An instruction-tuned version exists, trained on synthetic data derived from historical etiquette manuals and cookbooks, enabling actual conversation. The model is available free on Hugging Face with a live chat demo on their site. A larger variant is planned for summer 2026.