Paper2Code vs Waydev

Paper2Code is an open-source multi-agent framework accepted at ICLR 2026 that automatically converts machine learning research papers from arXiv into runnable, modular code repositories. The system uses three specialized agents working in sequence: a Planner that extracts architecture diagrams and file dependency graphs from paper figures and text; an Analyzer that maps each method section to concrete implementation decisions; and a Generator that writes modular, executable code with proper package structure. Accuracy benchmarks are notable: on a curated evaluation set of recent ML papers with public reference implementations, only 0.81% of generated lines required manual correction before the code ran successfully. The system handles standard ML frameworks (PyTorch, JAX, Hugging Face) and generates test scripts alongside the implementation. Papers are ingested via arXiv IDs or PDF upload. The reproducibility crisis in ML research — where papers claim state-of-the-art results but provide no runnable code — has been a persistent problem. Paper2Code directly attacks this gap, and the ICLR acceptance signals genuine peer-reviewed validation of the approach. The repo launched publicly in early April 2026 and quickly picked up attention from both ML researchers frustrated with missing codebases and developers interested in the multi-agent pipeline as a pattern for document-to-code tasks.

Waydev has relaunched as the measurement layer for AI-written code, letting engineering teams track which AI agent wrote which code, tokens consumed per PR, cost-per-shipped-line, and acceptance rates — with a unified comparison dashboard across GitHub Copilot, Cursor, Claude Code, and other AI coding tools. Founded in 2017 and backed by Y Combinator (W21), Waydev spent nine years building engineering analytics infrastructure. The pivot to AI SDLC measurement uses that existing integration surface (GitHub, GitLab, Jira, Linear) to add agent attribution metadata on top of existing flow metrics. The result is the first tool that can answer 'our team spent $4,200 on AI coding tools last month — which $1,000 was actually worth it?' With enterprise engineering budgets now routinely including five-figure monthly AI tooling costs and no standardized way to measure output quality by tool, Waydev's timing is sharp. The YC pedigree and existing customer relationships mean this isn't starting from zero — they're adding a new measurement layer to existing installed base.