Contents
In brief
Chrome 138+ exposes the Prompt API: await LanguageModel.create() runs Gemini Nano on-device—no server, API key, or network round-trip. The model (~4 GB) ships with browser updates. Production still needs a hosted fallback; many users will never qualify.
What happened
Flags and the spec matured since 2024; by May 2026 the API is in Stable plus an Origin Trial for production origins. HN demos and extensions (comment rewriting, live subtitle translation) show real shipping, not lab toys.
Minimal flow:
const session = await LanguageModel.create({
systemPrompt: "You are a concise technical assistant.",
});
const reply = await session.prompt("Summarize this in one sentence.");
Streaming uses session.promptStreaming() with for await; cancel via AbortSignal.
Why it matters
This is a distribution channel hosted vendors cannot match on price for light tasks: nothing to install, just Chrome. It competes with local servers and open weights for “cheap AI,” but with zero user setup.
Honest limits:
| Area | Reality |
|---|---|
| Context | ~4K in / ~1K out |
| Languages | English first |
| Hardware | ~4 GB VRAM or 16 GB RAM + 4+ cores |
| Disk | ~22 GB free for model cache |
| Weak at | Long RAG, codegen, multi-step reasoning |
Mozilla warns about prompt lock-in to Nano quirks—fallback protects Firefox/Safari users and future model updates.
In practice
- Poll
LanguageModel.availability()(downloadable,downloading,available). - Expose both paths as one
AsyncIterable<string>so UI stays identical. - Point fallback at a cheap hosted model, not flagship pricing.
- Do not overclaim privacy—browser UI for “on-device AI in use” is still immature.
- Treat JSON output as best-effort; truncate long pages before prompting.
Rule of thumb: <500 tokens in, <200 out, recoverable mistakes → Nano; else hosted.
Takeaway
The Prompt API is an autocomplete-class LLM, not ChatGPT in the tab. Ship one feature with fallback, measure on-device hit rate, then decide API economics.