evalmonkey vs Honker

evalmonkey is an open-source framework for testing how LLM agents degrade under adversarial conditions. You run your agent against 10 standard datasets (GSM8K, ARC, HellaSwag, etc.) pulled automatically from HuggingFace, then apply chaos profiles that introduce realistic failure modes: malformed JSON schemas, artificial latency spikes, 429 rate-limit errors, context-window overflow, and prompt injection payloads. The key output is a degradation delta — evalmonkey shows you exactly how much your agent's accuracy drops under each failure type versus clean inputs. A model that scores 78% on GSM8K normally but drops to 31% when it gets a 429 mid-chain tells you something crucial about its error-recovery behavior that standard benchmarks completely miss. It supports OpenAI, Anthropic (via Bedrock and direct), Azure, GCP, and any Ollama-hosted model. Corbell-AI published this with a clear thesis: agents break in production for infrastructure reasons, not model reasons — and no existing benchmark tests that. evalmonkey was created today (April 17, 2026) and is still at 3 stars, but the core idea is genuinely novel in the evals space.

Honker is a Rust-built SQLite extension that brings Postgres-style NOTIFY/LISTEN semantics to SQLite without any external broker. It adds cross-process notifications, durable pub/sub channels, task queues with retries and priority, and crontab-style scheduling — all living inside your existing SQLite file. Single-digit millisecond delivery via WAL-file watching instead of polling. The core trick: rather than polling the database on an interval, Honker watches SQLite's Write-Ahead Log (WAL) file with stat(2) calls. When a write lands, listeners wake up immediately. This gives push semantics without Redis, RabbitMQ, or any additional infrastructure. Business logic writes and task enqueues are atomic because they're in the same database. Honker ships as a loadable SQLite extension plus language packages for Python, Node.js, Rust, Go, Ruby, Bun, Elixir, and C++. It's experimental and the API may change, but it's addressing a real pain point: SQLite projects that outgrow simple reads/writes inevitably reach for external messaging, and Honker defers that moment significantly.