evalmonkey vs marimo-pair

evalmonkey is an open-source framework for testing how LLM agents degrade under adversarial conditions. You run your agent against 10 standard datasets (GSM8K, ARC, HellaSwag, etc.) pulled automatically from HuggingFace, then apply chaos profiles that introduce realistic failure modes: malformed JSON schemas, artificial latency spikes, 429 rate-limit errors, context-window overflow, and prompt injection payloads. The key output is a degradation delta — evalmonkey shows you exactly how much your agent's accuracy drops under each failure type versus clean inputs. A model that scores 78% on GSM8K normally but drops to 31% when it gets a 429 mid-chain tells you something crucial about its error-recovery behavior that standard benchmarks completely miss. It supports OpenAI, Anthropic (via Bedrock and direct), Azure, GCP, and any Ollama-hosted model. Corbell-AI published this with a clear thesis: agents break in production for infrastructure reasons, not model reasons — and no existing benchmark tests that. evalmonkey was created today (April 17, 2026) and is still at 3 stars, but the core idea is genuinely novel in the evals space.

marimo-pair is an open-source extension for marimo reactive notebooks that lets you drop AI agents directly into live, running notebook sessions. Unlike traditional AI coding assistants that only see static code, these agents can execute cells, inspect in-memory variables, read dataframes, manipulate UI components, and iterate on your actual live state — not a static snapshot. The tool plugs into Claude Code via a marketplace plugin and supports any agent implementing the Agent Skills standard. An agent that can see and run your notebook opens up genuinely new workflows: "explore this dataframe and tell me what's anomalous," "run this hypothesis test on the data already in memory," or "generate a chart for each of these 12 conditions." It's the difference between an assistant that reads your code and one that works alongside you in your actual environment. Marimo itself is already a compelling React-based replacement for Jupyter — every cell tracks its dependencies so the notebook is always consistent. marimo-pair makes that reactive model collaborative with AI, enabling a new style of human-AI pair programming where the agent shares your full computational context.