Bonsai-8B vs MemPalace

Bonsai-8B is PrismML's latest model in their BitNet-inspired lineage — an 8.2B parameter language model that has been quantized end-to-end to true 1-bit precision (weights stored as -1 or +1), compressing the entire model to just 1.15 GB. That's roughly 12-14x smaller than a standard FP16 equivalent. Unlike post-training quantization hacks that lose substantial quality, PrismML trained Bonsai-8B with 1-bit arithmetic baked into the forward pass from the start. Benchmark results are competitive for the size class: 63.8 on MMLU, 72.1 on HellaSwag, and 54.2 on GSM8K — while running at 131 tokens/sec on an M4 Pro MacBook and 44 tokens/sec on an iPhone 17 Pro Max. That makes it the fastest locally-runnable 8B model in its weight class on Apple Silicon. The MLX-optimized weights are available on Hugging Face today under Apache 2.0. The significance goes beyond benchmarks. Getting a capable open-weight model to run at interactive speeds on consumer hardware — with no API key, no GPU, no cloud dependency — is a meaningful step toward truly private, offline AI. This follows PrismML's earlier "Ternary Bonsai" (1.58-bit) but represents a cleaner binary architecture that's easier to accelerate on custom silicon.

MemPalace is an open-source persistent memory system for LLMs that takes a philosophically different approach from every summarization-based alternative: it stores conversations verbatim, forever, and retrieves them with semantic precision. Where systems like MemGPT or standard RAG pipelines compress memories into lossy summaries, MemPalace treats exact wording as sacred — because often the specific phrasing of something a user said six months ago is the thing that matters. The storage architecture uses a hierarchical "memory palace" metaphor: people and projects are wings, topics are rooms, individual memories are drawers. Semantic retrieval is scoped to sub-trees rather than doing a flat vector search across everything, which dramatically reduces false positives and improves precision at depth. The system claims a 96.6% score on LongMemEval — the highest publicly reported score among free tools — and integrates with any OpenAI-compatible API endpoint. Verbatim storage does mean storage costs grow linearly with usage, and there's no built-in forgetting mechanism yet (which some see as a bug and others as a feature). But for personal assistants, coding agents, and any application where "you told me X last Tuesday" accuracy matters, MemPalace's approach to memory is architecturally more honest than the alternatives.