AI tool comparison
LM Studio + Locally AI vs MDArena
Which one should you ship with? Here is the side-by-side panel verdict, pricing read, reviewer split, and community vote comparison.
Developer Tools
LM Studio + Locally AI
LM Studio buys the best iOS local LLM app to go cross-device
75%
Panel ship
—
Community
Free
Entry
LM Studio, the most popular desktop app for running local large language models, has acquired Locally AI — the leading iOS and iPadOS app for on-device inference on Apple Silicon. Locally AI's creator Adrien Grondin is joining LM Studio full-time to lead cross-device native AI experiences. The acquisition signals LM Studio's ambition to own the full local AI stack: macOS, Windows, Linux, and now iPhone and iPad. Locally AI was notable for its deep Apple Silicon integration, using Core ML and Metal Performance Shaders to run models like Llama 3 and Phi-3 natively on A-series and M-series chips. The app had a dedicated following among privacy-conscious users who wanted a clean iOS interface without compromising their data to cloud services. LM Studio brings a larger model library, server mode, and a more mature MLX/GGUF toolchain. For local AI enthusiasts, this is a consolidation play in a space that was starting to fragment across too many single-platform apps. A unified LM Studio experience across desktop and mobile would be a significant UX improvement. It also sets up an interesting competition with Apple's own on-device AI ambitions in iOS 19.
Developer Tools
MDArena
Benchmark your CLAUDE.md files against real PRs to see if they actually help
50%
Panel ship
—
Community
Free
Entry
MDArena is an open-source benchmarking tool that answers a question every Claude Code user eventually asks: do my CLAUDE.md context files actually improve agent performance, or am I just adding tokens? It mines merged PRs from your repository, strips or injects context files, runs your actual test suite, and measures success rates with statistical significance tests. The methodology mirrors SWE-bench: use `git archive` to create history-free checkpoints so agents can't peek at future commits, detect test commands from CI/CD configs automatically, and run paired t-tests to determine whether differences are real or noise. The project was motivated by academic research showing many CLAUDE.md files reduce agent success rates by 20% while consuming more tokens. For any team investing heavily in Claude Code infrastructure, MDArena provides empirical feedback that most developers currently lack. It's a small, focused tool that solves an annoying but real problem in the emerging AI coding workflow.
Reviewer scorecard
“This is the right move for LM Studio. The desktop client is already excellent and Locally AI's Core ML integration is the best iOS inference wrapper available. Combining Grondin's Apple-native work with LM Studio's model management and server mode could produce something genuinely special for local AI power users.”
“I've spent real time crafting CLAUDE.md files with no way to know if they help. A tool that uses my actual test suite against real PRs to measure context file effectiveness is exactly the feedback loop I've been missing. The `git archive` anti-cheat approach shows this was built by someone who's thought carefully about methodology.”
“Acquisitions in open-source adjacent tools often mean the indie app loses what made it great. Locally AI was clean and opinionated; LM Studio is powerful but has more surface area. There's real risk the mobile experience gets de-prioritized once the acquisition honeymoon ends.”
“Benchmarking on merged PRs is circular — the agent is being tested on tasks that were already solved by humans, which may not reflect the actual distribution of tasks you need it for. Statistical significance from your codebase's PR history also doesn't generalize: what works in one repo will vary wildly in another. Interesting research tool, limited practical signal.”
“The race to own the local AI client layer is just beginning. LM Studio is positioning itself as the VLC of AI — runs everything, everywhere, free. If they nail the cross-device sync story (shared model library, shared chats), they become the default for privacy-first AI.”
“Context engineering is becoming a real discipline as AI coding agents proliferate, and right now it's entirely vibes-based. MDArena represents the first step toward empirical context optimization — within two years, running something like this before shipping an agent configuration will be standard practice.”
“Being able to run the same model on my MacBook and iPhone with the same interface is a genuine quality-of-life win. I use local models for confidential creative writing and the iOS gap has always been frustrating. This closes it.”
“The audience here is squarely developer teams with established test suites and PR histories — not a tool for creators or smaller codebases without CI/CD. The value proposition is real, but only lands for teams already deep in Claude Code infrastructure.”
Weekly AI Tool Verdicts
Get the next comparison in your inbox
New AI tools ship daily. We compare them before you waste an afternoon.