Cartridges vs Talkie

Cartridges is an open-source single-GPU PyTorch reproduction of two recent papers on KV-cache compaction for long-context LLM inference: "Cartridges" (lightweight long-context representations via self-study condensation) and "STILL." Both methods address the same bottleneck — KV caches grow linearly with context length and quickly become the dominant memory consumer in long-context inference, making extended context windows impractical on consumer hardware. The Cartridges paper proposes condensing long contexts into compact "cartridge" representations through a self-study phase, trading some context fidelity for dramatic memory reduction. STILL uses a different approach focused on selective layer-wise compression. This repository makes both reproducible on a single consumer GPU — previously these required multi-GPU setups accessible mainly to research labs. KV-cache memory is one of the primary bottlenecks preventing long-context models from running efficiently on local hardware. A working single-GPU reproduction of these techniques is directly useful to anyone building long-context applications outside of cloud environments, and may accelerate community development of hybrid compaction strategies not in the original papers.

Talkie is a 13-billion parameter language model trained exclusively on English-language texts published before 1931 — the largest vintage language model built to date. Created by researchers Nick Levine, David Duvenaud (University of Toronto), and Alec Radford (of GPT and DALL-E fame), it represents a novel approach to understanding what training data really does to a model. The research insight is elegant: modern LLMs are so thoroughly contaminated by modern internet data (directly or through distillation) that it's nearly impossible to isolate what the model "knows" from what it absorbed during training. Talkie solves this by hard-cutting the training corpus at 1931 — predating digital computers entirely. This lets the team run controlled experiments impossible with contemporary models, such as teaching the model to write Python from examples alone and measuring how quickly it generalizes. Talkie was trained on ~260 billion tokens of historical text and fine-tuned using direct preference optimization with Claude as judge on structured historical documents (etiquette manuals, letter-writing guides). It's openly available on Hugging Face for research use. It also happens to produce wonderfully formal, slightly anachronistic prose.