LiteRT-LM vs Perplexity Deep Research API

“A unified inference runtime across Android, iOS, browser, and IoT with function calling support is exactly what the edge AI ecosystem has been missing. The WebAssembly path alone opens up private on-device AI in any browser without installing anything. Ship this immediately.”

“The primitive here is clean: one API call returns a cited, multi-step research report instead of you stitching together a crawler, a chunker, a retriever, and a summarizer yourself. The DX bet is depth-as-a-parameter, which is the right call — you specify how deep the research goes and pay accordingly, rather than configuring a pipeline. The moment of truth is whether the citation metadata is structured enough to render in your own UI, and from the docs it looks like it is — sources come back with URLs and relevance signals, not just inline footnotes. A competent engineer could approximate this with Tavily plus GPT-4o plus a Redis queue, but the latency and reliability gap is real enough that the abstraction earns its price. Ships because it collapses a genuinely annoying multi-service integration into a single endpoint with predictable output schema.”

“Edge inference is still severely constrained — even quantized Gemma 3B on a phone gives you a noticeably worse experience than cloud APIs. Google's history with edge AI frameworks is also mixed: TensorFlow Lite, ML Kit, MediaPipe all launched with fanfare and then got inconsistent maintenance.”

“Direct competitor here is Exa plus any frontier model with web access, or just OpenAI's Deep Research endpoint — yes, OpenAI has one too, and that's the threat this review has to acknowledge upfront. Where Perplexity has a real edge is citation density and source freshness; their crawler is genuinely good and the cited-report format is more structured than what you get back from a raw GPT-4o search call. The scenario where this breaks is high-volume enterprise workloads where session-depth pricing compounds fast — a product that runs 500 research queries a day will see costs balloon in ways that a flat-rate subscription wouldn't. Twelve-month prediction: OpenAI ships 90% of this natively into the Responses API with better model quality, and Perplexity has to compete on price and source breadth. What would have to be true for me to be wrong: Perplexity's web index turns out to be meaningfully fresher and wider than what OpenAI can access, which is not implausible given their search-first architecture.”

“This is infrastructure for the next decade. When models run on-device with no latency and no data leaving the device, entirely new categories of ambient, private AI become possible. LiteRT-LM is the missing runtime layer for that world — and Google open-sourcing it means the ecosystem builds around it rather than around Apple.”

“The thesis here is falsifiable: within three years, knowledge work applications will be expected to answer questions with cited, multi-step research rather than static retrieval — and building that capability in-house will be as absurd as building your own search index. That's a credible bet, not a vibe. What has to go right: enterprise buyers have to accept AI-generated research as sufficient for high-stakes decisions, and Perplexity's citation model has to remain trusted enough that downstream liability doesn't kill the use case. The second-order effect that nobody's talking about: if this API succeeds, it accelerates the commoditization of analyst-tier research tasks at the application layer — which reshapes what junior knowledge workers get hired to do, not just what tools they use. Perplexity is on-time to the 'research as infrastructure' trend, not early; the window before the major model providers close the gap is 12-18 months. If this tool wins, it becomes the research substrate for a generation of B2B SaaS products the same way Stripe became the payment substrate — the infrastructure nobody builds themselves.”

“Offline AI for creative apps is a game-changer — imagine Procreate or Figma with on-device generative features that work on a plane. The browser WebAssembly support means I can prototype these ideas without an app store or backend. Very excited about the creative possibilities here.”

“The buyer here is a product or engineering team at a company that wants research-enriched features — competitive intelligence dashboards, due diligence tools, automated briefing products — without owning the infrastructure. That buyer has a real budget and a clear make-vs-buy calculus. The pricing architecture is usage-based, which aligns with value when research sessions are sparse but becomes a liability if a customer's use case is high-frequency; I'd want to see volume tiers or committed-use discounts before betting a product on this. The moat is the web index and the citation quality — Perplexity has been building that index for years and it's legitimately differentiated from a raw LLM call. The platform risk is real: if OpenAI or Anthropic bundles equivalent search grounding into their standard API pricing, this margin story gets uncomfortable fast. Ships because the wedge is real and the buyer is defined, but the pricing architecture needs enterprise tiers before this scales cleanly.”