Magika vs qmd

Magika is Google's AI-powered file content-type detection library, now available as open source. Unlike traditional magic-byte heuristics (like libmagic), Magika uses a small custom deep learning model that runs in milliseconds on CPU and identifies 200+ file types with approximately 99% accuracy — a significant improvement over rule-based alternatives, especially on binary formats and polyglot files. Available as a CLI (Rust), Python package, and JavaScript/TypeScript library, Magika integrates cleanly into build pipelines, security scanners, and file-processing backends. Google deploys it internally to route hundreds of billions of files per week across Gmail, Drive, and Safe Browsing. It's also integrated with VirusTotal and abuse.ch for malware triage. A research paper was published at ICSE 2025. The practical use cases are broad: malware analysis, upload validation, content pipelines, archival systems, and anywhere you need to trust a file's actual type rather than its extension. The model footprint is small enough to ship with a CLI or embed in a serverless function — no GPU required.

qmd is a lightweight local search engine built by Tobi Luetke, CEO of Shopify, for indexing and querying personal knowledge bases, documentation, and meeting notes — entirely offline. It combines three retrieval approaches in a single pipeline: BM25 full-text search for exact keyword matches, vector semantic search via ONNX-based embeddings, and LLM re-ranking using GGUF models through node-llama-cpp. All three stages run locally with no cloud dependency. The tool ships in multiple deployment modes: a CLI for ad-hoc queries, a Node.js library for programmatic use, an HTTP service for local API access, and — most useful for AI workflows — a native MCP server that lets Claude Code, Cursor, and similar editors query your local knowledge base directly during coding sessions. The hybrid retrieval approach means it handles both "find the exact error message from last week's standup notes" and "what was our decision about the auth architecture" equally well. What makes this notable beyond its technical approach is provenance: Luetke shipped it as a personal tool he actually uses, not a startup product. The GitHub history shows active iteration and he's been talking about it on X. It's a credible signal of where pragmatic AI-augmented knowledge management is heading for technical users who prefer local-first tools.