Structured Output Benchmark vs MinerU2.5

Interfaze's Structured Output Benchmark (SOB) exposes a gap that has been quietly breaking production AI pipelines: models can produce syntactically valid JSON while getting the actual values wrong. SOB measures value accuracy across 21 models using 5,000 text passages, 209 OCR documents, and 115 meeting transcripts — scoring each on seven metrics including value accuracy, faithfulness (grounding vs. hallucination), type safety, and perfect-response rate. The benchmark reveals some sobering findings. Even top models like GPT-5.4 and Claude Sonnet 4.6 achieve ~83% on text but drop to 67% on images and only 23.7% on audio. No single model dominates all modalities — GPT-5.4, GLM-4.7, Qwen3.5-35B, and Gemini 2.5 Flash cluster within one point of each other on text. Perfect response rates (all seven metrics correct) rarely exceed 50% for even the best performers. For developers building data extraction pipelines, agents that read invoices, or any system where "correct JSON" means more than syntactically valid JSON, this is required reading. The dataset is on Hugging Face, the paper is on arXiv, and the playground lets you test your own model's structured output capability directly.

MinerU2.5 is a 1.2-billion parameter vision-language model purpose-built for high-resolution document parsing. From OpenDataLab, it's the latest version of a project that's accumulated 61.5K GitHub stars — which tells you something about how painful document-to-text has been as a category. The model uses a decoupled vision-language architecture for efficient high-resolution processing with state-of-the-art recognition accuracy across tables, formulas, figures, and mixed-layout documents. The core use case is turning messy PDFs, scanned forms, academic papers, and enterprise documents into clean Markdown or structured JSON that LLMs can actually work with. Earlier MinerU versions were already widely adopted for RAG pipeline preprocessing — 2.5 tightens up accuracy on the edge cases that killed earlier tools: rotated pages, dense tables, multi-column layouts, and multilingual content. At 1.2B parameters it's lightweight enough to run locally without a GPU farm, and the Apache 2.0 license means it integrates cleanly into commercial document pipelines. For anyone building RAG applications, AI research assistants, or document intelligence products, this is the preprocessing layer that removes a persistent pain point.