Why run AI in the browser

Server inference gives environment control and room for large models. The bill is GPU rent, network delay, shipping documents to someone else’s host, and scaling for spikes. For forms, receipts, and internal paperwork, “where did the photo go?” often matters more than “how smart is the model?”

Client path: the first visit downloads weights (or hits cache), then inference stays local. The server stores results, permissions, and orchestration — it does not have to see the raw image. The practical question: can an ordinary React site become an AI-system client? Yes — if the task fits model size, cold start, and WebGPU/WASM support on your audience’s devices.

WebGPU in plain terms

Neural nets are mass matrix math. CPUs excel at varied sequential work. GPUs excel at thousands of similar parallel steps. Roughly: one worker doing a thousand ops in sequence versus a thousand tiny workers at once.

Before WebGPU, browsers had JavaScript on CPU, WebAssembly, and WebGL. WebGL is graphics-first; ML hacks it through textures and shaders — awkward and limited. WebGPU is the successor with a more direct model of modern GPUs and compute support: what ML runtimes need.

Separate the layers:

JavaScript
   ↓
ML framework / runtime
   ↓
WebGPU
   ↓
platform GPU API
   ↓
GPU

WebGPU alone does not classify images or read text. It lets a runtime execute a compute graph on the user’s hardware.

Transformers.js and the ONNX link

Transformers.js (npm @huggingface/transformers) is Hugging Face’s JavaScript library for running models in the browser and in Node. Pipelines cover text classification, embeddings, speech recognition, image classification and detection, OCR-style tasks, text generation, and other Transformer- and vision-based models exported to ONNX.

What ONNX is (briefly)

ONNX (Open Neural Network Exchange) is an open format for a neural network graph: nodes (ops), tensors, constants. It is not another training framework and not a PyTorch replacement. Training usually stays in PyTorch or a similar stack; you export to ONNX so one artifact can run on many engines.

ONNX Runtime (ORT) is the engine that reads that graph and executes it: on a server (CPU/CUDA), in Node, or in the browser (ONNX Runtime Web with WASM / WebGPU backends). Transformers.js sits on ORT Web: you call a friendly pipeline, and an ONNX graph runs underneath.

Minimal chain:

Trained model (PyTorch, etc.)
        ↓
   export to ONNX
        ↓
ONNX Runtime (Web / server)
        ↓
   WebGPU / WASM / CUDA …

Without that link, “our CRNN in the browser” does not glue together: React does not run .pt files directly. Export, opsets, dynamic axes, and Python parity are covered in ONNX: from PyTorch to Runtime.

Model lifecycle:

First run → download files → browser cache → local inference

Weight size hits cold start hard: hundreds of megabytes on mobile is a product decision, not a demo footnote.

Roles in the stack:

Full diagram:

React / JavaScript
        ↓
Transformers.js
        ↓
`ONNX Runtime Web`
        ↓
   WebGPU or WASM
        ↓
     GPU / CPU

WebGPU, WASM, and server: what to pick

Product ladder:

WebGPU → else WASM → if too heavy/slow → Server inference

Do not pick a backend for fashion. Pick it for the task, p95 latency on real audience devices, and data policy.

First model and device: 'webgpu'

Install:

npm install @huggingface/transformers

Minimal text classification:

import { pipeline } from '@huggingface/transformers';

const classifier = await pipeline(
  'text-classification',
  'Xenova/distilbert-base-uncased-finetuned-sst-2-english',
);

const result = await classifier('Hello world');

After pipeline, the library resolves the model card, downloads ONNX files, prepares the runtime, picks a backend, runs inference, and returns the result. To force GPU:

const pipe = await pipeline(
  'image-classification',
  'onnx-community/mobilenetv4_conv_small.e2400_r224_in1k',
  { device: 'webgpu' },
);

The same pattern works for embeddings and ASR — see Hugging Face’s WebGPU guide. device: 'webgpu' is not a guarantee: you need API support, a working driver, and operators the runtime can run on that backend. Wrap init in detection plus a WASM (or server) fallback.

React: where to keep the model and why a Worker

Do not create a pipeline on every render. Use lazy init, a singleton service, or a useAIModel() hook that loads once and exposes status: idle → loading → ready → error. While weights download, the UI shows progress and stays usable. Returning to the OCR screen should reuse a warm instance, not re-download.

UI shape:

React UI → useAIModel() → Model Service → Transformers.js → ORT → WebGPU

Inference and large tensor work on the main thread can freeze scrolling and clicks. Move load and inference into a Web Worker: the main thread sends an ArrayBuffer or OffscreenCanvas, the worker returns results, timings, and error codes. Same idea as other heavy client work — see Web Workers for client compute.

Plan cancellation: the user picks another photo while inference runs. The worker should ignore stale requestIds, or the UI will show the previous file’s result.

Computer vision and OCR in the browser

This stack is not only NLP. A typical vision pipeline:

Image → preprocessing → detect / crop → classify or OCR → result

Cases: documents, receipts, forms, diagrams, technical sheets. Classic server OCR uploads the file. Browser OCR keeps the image on-device:

Browser → Image → Preprocessing → AI model → Text

Pros: data stays local, lower network latency, no GPU instance cost, offline after cache. Cons: model size, device requirements, sandbox limits, harder to ship very heavy weights.

For measurement forms, “find table → find cell → crop → compact OCR” beats shipping the whole scan into a giant LLM. Same discipline as CRNN on UZT measurements and handwritten digits — with the runtime shifted toward the client.

Your CRNN, YOLO, and document vision

A custom PyTorch model can reach the browser like this:

PyTorch → trained CRNN → export ONNX → ORT Web → (Transformers.js or direct ORT) → WebGPU

Check that target execution providers support every op; browser preprocessing matches Python (normalization, channel order, size); postprocessing (CTC, argmax) matches a golden set. “0.98 in Colab, garbage in the tab” is almost always preprocess or quantization mismatch — not “WebGPU magic”.

Detector path:

Image → YOLO → boxes → crop → OCR

YOLO answers “where”, OCR answers “what”. Both can stay local if sized down. A client-side document vision pipeline is realistic for internal tools that must not leave the workstation. Detector engineering is covered in the YOLO pillar.

When it helps, model size, and quantization

Good fits: OCR on small crops, classification, object detection, embeddings, compact vision models, speech, local tools, offline-first apps.

Poor fits: very large LLMs, huge vision nets, high RAM/VRAM tasks, unsupported ops, unbearable cold start.

Delivery paradox:

Server:  10 MB JS + 5 GB model on the server
Browser: 10 MB JS + 500 MB model on the user

Users pay in bandwidth and disk. Mitigate with quantization, compression, cache (Cache Storage / IndexedDB), lazy load, CDN, and several small models instead of one giant.

Quantization, simplified:

FP32 → FP16 → INT8 → smaller size/memory → often faster

It is a size–speed–quality trade-off, not free lossless shrink. Measure accuracy on your validation set after each step.

Privacy, offline-first, and server comparison

Local inference reduces shipping raw documents to an inference host. That is not absolute safety: weights and dependencies still download; the sandbox limits but does not erase supply-chain risk; a tampered model artifact is its own threat. Control weight sources and integrity (hashes, your CDN).

Offline-first after cache warm-up:

React → Local AI Model → WebGPU → GPU
(network may be offline)

PWA, Cache Storage, and IndexedDB hold UI and weights. Comparison:

How to measure and mistakes that sink projects

Count more than “it ran”. Track model size (disk and memory), download vs cache load time, cold first inference, warm inference, preprocessing/postprocessing share, tab memory peaks, and WebGPU success rate. Without “first visit vs return” splits you optimize the wrong path: return users already have weights in Cache Storage.

Lab report example:

Model loading (cold network):  2.8 s
Model loading (cache hit):     0.4 s
First inference:               450 ms
Warm inference:                 85 ms
Model size:                    120 MB
WebGPU success rate (lab):      9/10 devices

Common mistakes: treating Transformers.js as “the neural net” and WebGPU as an ML framework; shipping a huge model “because the server can”; running inference on the UI thread; no WASM fallback; ignoring cold start; sending the whole photo when a crop would do; calling an LLM where a detector plus OCR suffice; comparing browser quality to Python without pinning quantization.

Production skeleton:

React → AI Service → Web Worker → Transformers.js → ORT Web → WebGPU
Fallback: WebGPU → WASM → Server API

A large LLM ≠ all AI. For documents this often wins:

YOLO → OCR → Classifier → Rules

instead of “image → giant LLM → JSON”. Server LLMs still belong where broad reasoning is needed — beside the client path, not instead of it. See three layers of the LLM stack.

Ahead: more mature WebGPU, APIs like WebNN, WASM SIMD, quantized and multimodal models, privacy-preserving apps. Separate what ships today (device: 'webgpu') from browser roadmaps, or planning turns into slide theater.

Mini project: AI OCR in a tab

Build a scratch app that:

1. Loads an image and shows a preview.
2. Checks navigator.gpu / tries device: 'webgpu'.
3. Loads a compact model with a progress indicator.
4. Preprocesses and, if possible, finds a document region.
5. Runs OCR → JSON in the UI.
6. Prints load time and warm inference.
7. Switches to a WASM fallback.

Target architecture:

Image → React → Web Worker → Model → WebGPU → OCR → JSON → UI

That proves the stack on your devices before promising a client “all AI in the browser”.

FAQ

Further reading

• YOLO engineering and OCR linkage
• Compact CRNN for measurements
• Handwritten digits: CNN, CTC, TrOCR
• Three layers of the LLM stack
• Web Workers for heavy client tasks
• Docs: Running models on WebGPU

Follow-up ideas: ONNX: from PyTorch to Runtime, ML frameworks, YOLO in the browser, CRNN on WebGPU, quantization without myths, offline-first AI.

Conclusion

WebGPU gives the browser GPU compute. Transformers.js simplifies running models. ONNX Runtime Web binds weights to execution providers. Together they keep OCR, detection, and compact vision on the user’s device — with honest fallback, a model-size budget, and a Worker around the UI. The future of web AI is not “browser only” or “server only”, but client inference for sensitive and frequent work paired with a server path for heavy models and control.