LiteRT-LM vs Skrun

LiteRT-LM is Google's production-grade, open-source inference framework for deploying Large Language Models on edge devices — phones, IoT hardware, Raspberry Pi, and desktop machines without cloud connectivity. Launched April 7, 2026 alongside Gemma 4 support, it enables developers to run Gemma, Llama, Phi-4, Qwen, and other models entirely locally via a simple CLI or embedded SDK. The framework handles the hard parts of edge inference: memory-mapped per-layer embeddings, 2-bit and 4-bit quantization, NPU acceleration for Qualcomm and MediaTek chipsets (early access), and cross-platform support spanning Android, iOS, Web, and desktop. Gemma 4's E2B variant runs under 1.5GB RAM on some devices, making full LLM functionality viable on mid-range hardware. What makes LiteRT-LM significant is the agentic angle. It's one of the first frameworks to support multi-step agentic workflows running completely on-device — function calling, tool use, vision and audio inputs — without a single network request. For developers building privacy-sensitive apps or offline-capable agents, this changes the calculus entirely.

Skrun is an open-source tool that wraps agentic skills — the discrete, reusable capabilities you build for AI agents (web search, data extraction, file transformation, API calls) — into deployable REST APIs with a single command. The idea is that skills you build for one agent context shouldn't be locked to that agent's runtime. With Skrun, you define a skill once with a standard function signature, and get a hosted endpoint with automatic request validation, retry logic, rate limiting, and an OpenAPI spec generated automatically. The project addresses a real architectural tension in the current AI tools ecosystem: agent skills are written in a dozen different formats (LangChain tools, MCP tools, function call JSON, OpenAI tool specs) and are essentially stranded assets — they only work within their specific orchestration framework. Skrun normalizes this by wrapping any skill definition format and exposing it as a framework-agnostic HTTP endpoint that any agent or pipeline can call. This appeared on Hacker News with a small but thoughtful discussion focused on the "skills as microservices" architectural pattern. Critics noted that adding HTTP round-trips to every tool call introduces latency; proponents argued that the composability and reusability benefits outweigh the cost. The early version focuses on stateless skills; stateful/conversational skill deployment is on the roadmap.