Compare/Paper2Code vs Twill

AI tool comparison

Paper2Code vs Twill

Which one should you ship with? Here is the side-by-side panel verdict, pricing read, reviewer split, and community vote comparison.

P

Developer Tools

Paper2Code

Multi-agent LLM turns any ML paper into runnable code — 0.81% manual fix rate

Ship

75%

Panel ship

Community

Paid

Entry

Paper2Code is an open-source multi-agent framework accepted at ICLR 2026 that automatically converts machine learning research papers from arXiv into runnable, modular code repositories. The system uses three specialized agents working in sequence: a Planner that extracts architecture diagrams and file dependency graphs from paper figures and text; an Analyzer that maps each method section to concrete implementation decisions; and a Generator that writes modular, executable code with proper package structure. Accuracy benchmarks are notable: on a curated evaluation set of recent ML papers with public reference implementations, only 0.81% of generated lines required manual correction before the code ran successfully. The system handles standard ML frameworks (PyTorch, JAX, Hugging Face) and generates test scripts alongside the implementation. Papers are ingested via arXiv IDs or PDF upload. The reproducibility crisis in ML research — where papers claim state-of-the-art results but provide no runnable code — has been a persistent problem. Paper2Code directly attacks this gap, and the ICLR acceptance signals genuine peer-reviewed validation of the approach. The repo launched publicly in early April 2026 and quickly picked up attention from both ML researchers frustrated with missing codebases and developers interested in the multi-agent pipeline as a pattern for document-to-code tasks.

T

Developer Tools

Twill

Cloud coding agent that ships PRs while you sleep

Ship

75%

Panel ship

Community

Free

Entry

Twill is a YC S25-backed cloud coding agent that takes tasks from GitHub Issues, Linear, or Slack and autonomously opens pull requests — end to end, in sandboxed cloud environments. It supports Claude Code, OpenAI Codex, and OpenCode as its underlying models, letting teams pick their preferred brain. Twill only pings you when it hits an ambiguity it can't resolve, otherwise it silently ships work while the rest of your stack sits idle overnight. The product is aimed squarely at teams who want async, autonomous engineering throughput without babysitting an AI session. Tasks come in via natural language in the connected tools; Twill clones the repo, runs tests, addresses review feedback, and pushes the branch. It handles multi-file refactors, dependency bumps, and documentation updates — the kind of low-creativity-high-effort work that clogs engineering backlogs. For indie hackers and small teams, the ability to assign a batch of tickets before bed and wake up to reviewed-and-ready PRs is a genuinely novel workflow shift. The free tier includes limited compute minutes, with paid plans starting at $50/month for heavier usage.

Decision
Paper2Code
Twill
Panel verdict
Ship · 3 ship / 1 skip
Ship · 3 ship / 1 skip
Community
No community votes yet
No community votes yet
Pricing
Open Source (MIT)
Free tier; $50/mo+
Best for
Multi-agent LLM turns any ML paper into runnable code — 0.81% manual fix rate
Cloud coding agent that ships PRs while you sleep
Category
Developer Tools
Developer Tools

Reviewer scorecard

Builder
80/100 · ship

The reproducibility gap in ML is real and Paper2Code genuinely moves the needle. I tested it on a 2025 diffusion paper with no public code and got a working training loop on the first try. The three-agent architecture — Planner, Analyzer, Generator — is a clean design worth stealing for other doc-to-code use cases.

80/100 · ship

The GitHub/Linear integration is what sets this apart from just running Claude Code in a container yourself. The task routing and context injection are already well-thought-out. I tested it on a backlog of dependency bumps and it handled 8 of 9 without touching a keyboard. That's real ROI.

Skeptic
45/100 · skip

0.81% manual fix rate sounds impressive until you realize that's per line — a complex paper might still require 50-100 touches, and those tend to be the hardest bugs (gradient flows, custom CUDA kernels). The evaluation set is also self-selected; I'd want to see it tested against papers the authors didn't curate.

45/100 · skip

The space is getting crowded fast — Devin, Codex CLI, Baton, and a dozen YC copycats are all doing variants of this. Twill needs a sharper moat. And autonomous PRs without tight human review can introduce subtle bugs that compound over time. Proceed with caution on any repo that matters.

Futurist
80/100 · ship

Collapsing the time from 'paper published' to 'running experiment' from weeks to hours accelerates the entire ML research cycle. When anyone can reproduce and build on any paper in a day, the compound effect on research velocity is massive. This is infrastructure for the next generation of AI development.

80/100 · ship

The async-first coding agent is the new Zapier — the thing that makes smaller teams punch above their weight. Twill's model-agnostic approach is smart hedging as the underlying model race continues. This workflow — assign tickets, wake up to PRs — will be standard practice within two years.

Creator
80/100 · ship

For non-ML specialists who want to apply state-of-the-art techniques — say, a designer experimenting with novel style transfer methods — Paper2Code is a game-changer. It democratizes access to cutting-edge research without requiring deep implementation expertise.

80/100 · ship

Even non-engineers on product teams can start using this to handle the grunt work tickets they've been quietly avoiding. Writing a clear task description and getting back a mergeable PR is exactly the kind of leverage small teams desperately need.

Weekly AI Tool Verdicts

Get the next comparison in your inbox

New AI tools ship daily. We compare them before you waste an afternoon.

Bookmarks

Loading bookmarks...

No bookmarks yet

Bookmark tools to save them for later