OpenMythos vs Talkie

OpenMythos is a PyTorch reconstruction of the suspected architecture underlying Anthropic's Claude Mythos model, built entirely from published research. Creator Kye Gomez hypothesizes that Mythos uses a Recurrent-Depth Transformer (RDT) — where a subset of transformer layers loops multiple times per forward pass with shared weights rather than stacking unique layers. This allows the model to simulate "thinking" by iterating over the same compute graph, giving it emergent chain-of-thought behavior without explicit CoT prompting. At 770M parameters, the OpenMythos implementation reportedly matches the downstream quality of a 1.3B standard transformer on benchmarks. The architecture combines Multi-Latent Attention for memory compression, LTI (Linear Time-Invariant) stability constraints to prevent training instability during recurrence, Mixture of Experts routing for specialization, and Adaptive Computation Time (ACT) halting to decide when to stop looping per token. The project exploded on GitHub within days — 6.2k stars, 1.2k forks — and Kye's X announcement drove massive engagement (4.1k likes, 4.5k reposts). Community reaction is genuinely divided: AI researchers calling it "the most sophisticated reverse-engineering of an LLM architecture I've seen" while Anthropic has not confirmed or denied any of the architectural claims. This is an educated speculation backed by real engineering, not a marketing exercise.

Talkie is a 13-billion-parameter language model with an unusual constraint: it was trained exclusively on text written before 1931. That means no internet, no Wikipedia, no modern code — just 260 billion tokens of books, newspapers, journals, patents, and case law from the pre-modern era. The result is a "vintage" LLM that speaks like it's from the early 20th century and has zero knowledge of anything after its cutoff. The model was built by Nick Levine, David Duvenaud, and Alec Radford (yes, one of the original GPT authors) with support from Anthropic and Coefficient Giving. The scientific motivation is rigorous: Talkie enables researchers to cleanly test how models generalize to unfamiliar tasks from examples alone (since it's never seen Python), study future prediction capabilities without data leakage, and understand how training data diversity shapes model dispositions and values. An instruction-tuned version exists, trained on synthetic data derived from historical etiquette manuals and cookbooks, enabling actual conversation. The model is available free on Hugging Face with a live chat demo on their site. A larger variant is planned for summer 2026.