Claude 4 API: Tool Use Streaming & Prompt Caching vs Firecrawl v2

“The primitive here is clean: incremental tool-call deltas over SSE, and a cache-control header you attach to prompt segments to pin them server-side. The DX bet is that complexity lives in the HTTP layer, not in a new SDK abstraction — you opt in per-request, no new mental model required. The moment of truth is calling `stream=true` on a tool-use request and watching partial JSON arguments arrive before the model finishes thinking, which actually matters for agent loops where you want to dispatch work early. This is not a weekend-script replacement — implementing correct incremental JSON parsing for partial tool arguments plus a reliable distributed cache with 2M token capacity is a real engineering problem Anthropic has solved for you. The specific decision that earns the ship: cache invalidation is explicit and cache hits are reflected in the usage object, so you can actually measure what you're saving instead of guessing.”

“The primitive is clean: pass a URL and a Zod-style JSON schema, get back structured data — LLM handles the DOM-to-schema mapping so you never write another XPath selector. The DX bet is that schema-first extraction beats selector maintenance over time, and for anything with irregular or frequently-changing markup, that bet is almost certainly correct. The moment of truth is the first `extract` call — if your schema comes back populated with the right fields, you're sold; if the LLM hallucinates a field or silently omits a nested object, you're debugging against a black box. The weekend alternative (Playwright + cheerio + GPT-4o with a JSON mode prompt) gets you 80% of the way there in 200 lines, but Firecrawl earns its keep on JS-rendered pages and rate-limit handling that would take a week to replicate properly. The specific technical decision that earned the ship: they expose the schema contract at the API surface, not buried in a prompt string — that's the right abstraction.”

“Direct competitors are OpenAI's cached completions and Google's context caching in Gemini 1.5 — both shipping for months — so Anthropic is catching up, not leading. The specific scenario where this breaks: cache hit rates depend entirely on prompt structure, and developers who dynamically compose system prompts (inserting user-specific context at the top) will see near-zero cache utilization and pay full price while assuming they're saving money. The prediction: this feature doesn't get killed — it becomes table stakes infrastructure and Anthropic wins by having the largest cache window (2M vs. competitors' current limits). What would have to be true for me to be wrong: OpenAI ships a 10M token cache window before Anthropic's ecosystem matures, commoditizing the advantage. Still a ship because the streaming tool-use delta is genuinely differentiated — no competitor has clean partial-argument streaming for tool calls yet, and that changes agent loop architecture in ways that matter.”

“Category is LLM-powered web extraction; direct competitors are Apify's AI scrapers, Browserless with a GPT layer, and — honestly — OpenAI's operator-style browsing for structured tasks. Firecrawl v2 earns the ship specifically because the hosted MCP endpoint solves a real pain point: every agent framework team is reinventing web-fetch-plus-parse right now, and having a single reliable endpoint that returns structured JSON rather than raw markdown is legitimately useful. Where it breaks: any extraction job at scale where the LLM token cost per page starts eating your margin — the credit model obscures this until you're in production. What kills this in 12 months: Anthropic and OpenAI both ship native tool-use browsing with structured extraction as a first-class feature at effectively zero marginal cost. For Firecrawl to survive that, they need deep enough workflow integration and reliability track record that switching is painful — they're not there yet, but they have a credible path.”

“The thesis this bets on: by 2027, the dominant AI application architecture is a persistent agent with a large, stable context (tools, memory, instructions) that gets reused across thousands of user interactions — making context I/O cost the primary unit economics lever, not generation cost. The dependency that has to hold: agents don't collapse back to stateless chatbots, and context windows keep growing faster than per-token prices fall. The second-order effect nobody's talking about: prompt caching at 2M tokens makes it economically viable to give every enterprise user a fully-loaded, role-specific agent context at request time — which shifts competitive differentiation from 'who has the best model' to 'who has the best cached context corpus,' effectively making knowledge curation the new moat. This tool is riding the trend of context-window expansion-as-infrastructure, and it's on-time, not early — but the streaming tool-use primitive is ahead of the curve on agent loop efficiency. The future state where this is infrastructure: every production agentic system has a cache manifest the same way it has a CDN config.”

“The buyer is the engineering team at any company running Claude in production with long system prompts or multi-step agents — this comes out of the AI infrastructure budget, not a new budget line, which means no procurement friction. The pricing architecture is sound: cache reads at ~90% discount means the savings are real and measurable in the first billing cycle, which creates immediate retention — developers who restructure prompts to maximize cache hits are now architecturally coupled to Anthropic's caching implementation. The moat question is the honest one: this is infrastructure that OpenAI and Google will match, so the defensible position isn't the feature itself but the ecosystem of developers who've restructured their codebases around it. What survives a 10x model price drop: the streaming tool-use architecture, because that's about latency, not cost. The specific business decision that makes this viable is pricing cache reads as a separate SKU — it lets Anthropic capture value from high-volume production workloads without losing price-sensitive experimenters.”

“The buyer is a developer or small engineering team pulling it from an existing tool budget — likely DevOps or infrastructure spend — which is fine, but the credit-based pricing model is a trap: it's opaque enough that teams under-estimate production costs and hit a wall at the Standard tier before they've built switching costs. The moat question is the real problem here: the extraction quality depends entirely on the underlying LLM provider, the JS rendering layer is table stakes, and the MCP server is one open-source repo away from being replicated. When model costs drop 10x, Firecrawl's margin on credits compresses unless they've built proprietary training data or reliability infrastructure that actually differentiates — and nothing in the v2 announcement signals that. I'd want to see a clear enterprise tier with SLA guarantees and a data retention story before calling this a durable business rather than a well-executed API wrapper.”

“The job-to-be-done is sharp: get structured data from any URL without writing a parser, and v2 delivers on that in a single API call with a schema argument — no product tour, no configuration screen, you're at value the moment you see populated JSON. The product is complete enough to replace the current solution for teams currently stitching together Playwright, BeautifulSoup, and a GPT call, which is genuinely a large population. The opinion baked into the product is correct: the schema is the interface, not the CSS selector — that's the right bet on how developers want to express intent. The one gap that keeps this from a higher score: error handling and confidence signals on extracted fields are underdeveloped; when the LLM misses a field or returns a best-guess value, the API gives you no structured way to know, which means you're writing defensive validation code that the product should own.”