AI tool comparison
Sup AI vs Task Bert
Which one should you ship with? Here is the side-by-side panel verdict, pricing read, reviewer split, and community vote comparison.
AI Productivity
Sup AI
Runs 339 LLMs in parallel and downweights the hallucinating ones.
50%
Panel ship
—
Community
Free
Entry
Sup AI is an ensemble AI assistant that runs your query through 339 language models simultaneously, measures per-segment confidence across all responses, and synthesizes a final answer that amplifies agreement and suppresses likely hallucinations. The team claims a 52.15% score on Humanity's Last Exam (HLE) — 7.41 percentage points above the single best model — which, if verified, would make it the highest-scoring system on the benchmark to date. The underlying mechanism works like an LLM panel: each model votes on sub-claims within the response, confidence is estimated by agreement density, and the final output surfaces high-confidence segments while flagging uncertain ones. It's designed to reduce hallucination rate on factual tasks, not improve reasoning per se — the models in the ensemble aren't doing collaborative chain-of-thought, they're voting on outputs. Sup AI was built by Ken Mueller (Stanford, CEO) and Scott Mueller (AI Research Scientist) and launched on Product Hunt today. Pricing starts with $10 in free credits, no auto-charge, with a credit card required to start. The HLE benchmark claim is the headline and will face scrutiny — if verified, this is a meaningful research result. If it's cherry-picked, it's still a usable product with a differentiated architecture.
Productivity
Task Bert
Fully local iMessage AI agent that turns your conversations into tasks
75%
Panel ship
—
Community
Free
Entry
Task Bert is a privacy-first Mac app that acts as a local AI assistant for your iMessage conversations. It runs entirely on-device using local vector embeddings and your own API key (OpenAI or Anthropic), so your messages never touch a third-party server. The assistant can search across your message history, convert casual plans buried in conversations into calendar events and reminders, and surface follow-up nudges for conversations that fell through the cracks. The technical implementation is clean: it uses Hugging Face's nomic-embed-text model for on-device vector embeddings, meaning semantic search across your iMessage history doesn't require cloud calls. When it detects a plan or commitment in a conversation ("let's grab coffee Thursday"), it can write it directly to Apple Calendar and Reminders. The BYOK model puts the user in control — the app acts as orchestration layer, not a data holder. Task Bert targets a real pain point for heavy iMessage users: important follow-ups and plans routinely get buried in high-volume group chats or forgotten in long one-on-one threads. By running locally and integrating natively with Apple's ecosystem, it sidesteps the privacy concerns that have plagued cloud-based messaging assistants.
Reviewer scorecard
“The HLE claim needs independent verification, but the underlying ensemble approach is architecturally sound for factual Q&A tasks. Running 339 models is expensive — pricing will be the gating factor for production use. The $10 free credit is a fair trial.”
“BYOK + on-device embeddings is the right architecture for a messaging assistant. No cold storage of conversations, no vendor lock-in, no trust required. Using nomic-embed-text locally for semantic search is a smart call — it's fast and accurate enough for this use case without GPU hardware.”
“Extraordinary claims require extraordinary evidence. A 7.41 point jump on HLE via ensembling — without publishing methodology — smells like benchmark gaming. The latency of running 339 models in parallel is also a real concern for anything other than async research tasks.”
“Apple's iMessage privacy model creates real friction here — accessing message history requires specific macOS permissions that users are increasingly reluctant to grant after recent privacy scandals. Also, iMessage-only limits this to Apple devices, cutting out anyone running a mixed iOS/Android household. The addressable market is narrower than it looks.”
“Model ensembling is an underexplored direction in the race to reduce hallucination. If Sup AI's approach scales, it could be more durable than fine-tuning individual models — you get the wisdom of the crowd across model families, training data, and architectures simultaneously.”
“The local-first AI assistant is the next major product category. Task Bert is an early proof-of-concept for what happens when you give an AI agent read access to your communication history with proper privacy guarantees. As local inference gets faster, every major messaging platform will have something like this — but the indie versions will always be more trustworthy.”
“For creative work, ensemble outputs tend to regress toward the mean — you get the most-agreed-upon version of something, which is usually the least interesting version. This is a tool for factual accuracy, not creativity. I'd stick with a single strong model for writing.”
“The follow-up nudge feature alone would pay for this tool. I can't count how many creative collabs have died because someone (usually me) forgot to follow up on a message thread. Having an on-device assistant surface those forgotten conversations without sending them to a cloud server feels like a genuinely ethical approach to AI assistance.”
Weekly AI Tool Verdicts
Get the next comparison in your inbox
New AI tools ship daily. We compare them before you waste an afternoon.