tldr MCP Gateway vs TurboOCR

tldr is a local proxy that sits between your AI coding harness and upstream MCP servers, solving one of the most underappreciated problems in agentic workflows: context bloat from tool schema proliferation. When you connect GitHub MCP, filesystem MCP, and a few others, you can easily be sending 24,000+ tokens of tool schemas to the model before any work begins. Instead of passing all those schemas directly, tldr exposes exactly five wrapper tools to the model: search_tools, execute_plan, call_raw, inspect_tool, and get_result. The model learns which underlying tools exist on-demand through search_tools, then calls them through the proxy. GitHub MCP's 24,473-token schema surface compresses to 3,482 tokens — an 86% reduction. Output responses are further compressed through field stripping, a 4,096-token cap, and a 64KB byte limit. This is a genuinely practical solution for power users running multi-MCP setups who've noticed degraded performance as their tool count grows. The tradeoff is one extra hop of indirection, but the token savings pay for themselves in improved model attention and lower API costs.

TurboOCR is a C++20 OCR server that uses CUDA and TensorRT to process documents at speeds that make Python-based OCR look like a fax machine. The headline number: 270 images per second on FUNSD form datasets with approximately 11ms single-request latency — roughly 50x faster than PaddleOCR's standard Python implementation. It uses PP-OCRv5 models (the same underlying tech as PaddleOCR) but squeezes them through TensorRT FP16 optimization for GPU inference. The server exposes both HTTP and gRPC interfaces from a single binary and handles PDFs natively with four extraction strategies: pure OCR, native text layer extraction, hybrid verification mode, and a "best of both" fallback chain. PP-DocLayoutV3 handles layout detection across 25 document region classes — useful for structured documents where you need to know that a bounding box is a table cell vs. a header vs. a figure caption. A Prometheus metrics endpoint tracks throughput, latency, and GPU memory in real time. Deployment is Docker-first: TensorRT engine compilation happens automatically on first startup. The catch is it requires Linux with an NVIDIA Turing GPU (RTX 20-series minimum) and driver 595+, so it's not a laptop tool. But for enterprise document automation — invoices, forms, medical records — the throughput-to-cost ratio is hard to beat.