Simon Willison ported Moebius — a 0.2B-parameter image inpainting model — from PyTorch/CUDA to run entirely in the browser using ONNX and WebGPU. Weights live on Hugging Face (~1.3GB). Inference happens client-side in Chrome, Firefox, and Safari. The port was executed primarily through Claude Code while Willison worked on other tasks in parallel.
The project is a datapoint on two trends: browser ML is viable for small models, and coding agents can carry multi-step conversion work with minimal hand-holding.
Why move inference to the browser
Server-side inference for image models implies GPU hosting, cold start latency, upload bandwidth for images, and privacy questions when user photos leave the device. Client-side inference trades those costs for download size and device GPU requirements.
For a 0.2B model, the trade often favors the browser:
- Privacy — pixels never leave the machine.
- Marginal cost — no per-request GPU bill after assets are cached.
- Latency after load — no round trip once weights sit in CacheStorage.
The cost is upfront: users download over a gigabyte of weights on first visit. Willison mitigated with browser caching so repeat visits skip the full transfer.
Think of it like installing a desktop app once versus streaming video forever. Heavy first load, then local execution.
PyTorch to ONNX to WebGPU
PyTorch models do not run natively in browsers. The conversion pipeline:
- Export to ONNX — a portable graph format many runtimes consume.
- Validate numerical parity — spot-check outputs against PyTorch on sample inputs. Conversion bugs are common.
- Run via ONNX Runtime Web — compiled for WebAssembly and WebGPU backends.
- Wire browser I/O — canvas or WebGL textures for image input/output, CacheStorage for weight persistence.
WebGPU provides GPU acceleration where available; WASM falls back on devices without WebGPU support. The demo targets modern browsers explicitly.
Each step is mechanical but tedious — exactly the sort of multi-file refactor coding agents handle when given a clear target and test criteria.
Agent-assisted porting in practice
Willison describes reviewing little of the generated code directly — running the demo, checking outputs, iterating on failures. That workflow assumes:
- Automated verification — visual diff on inpainting results, console errors as signals.
- Clear done criteria — "inpaint this mask in browser" is testable; "make it elegant" is not.
- Parallel human attention — agent loops run while the human works elsewhere, not unattended forever.
Agent-assisted ports succeed when success is observable without reading every line — tests, demos, or numerical checks.
This is not "vibe coding" as abandonment of engineering judgment. It is delegating translation layers — ONNX export scripts, bundler config, cache headers — while the human owns acceptance criteria.
Size and performance realities
1.3GB is large for a web page. Techniques that matter:
- CacheStorage — persist weights across sessions after first download.
- CDN hosting on Hugging Face — reliable large-file delivery.
- Progressive loading UI — users tolerate download with clear progress; they do not tolerate silent hangs.
- Model size ceiling — 0.2B works; 7B does not fit this pattern without quantization and aggressive streaming research.
Browser ML today is a niche for small models and patient users — not a replacement for datacenter GPUs on frontier workloads.
| Stage | Server inference | Browser inference |
|---|---|---|
| First visit | Low (no model download) | High (GB-scale weights) |
| Repeat use | Per-request GPU cost | Near-zero marginal |
| Privacy | Data leaves client | Data stays local |
| Device reqs | Minimal | Modern GPU + WebGPU |
| Model size limit | Datacenter scale | Sub-billion practical |
What this means for builders
Prototype browser ML on sub-billion models first. Prove the UX and conversion pipeline before committing to larger architectures.
Treat ONNX export as a test-gated step. Always compare outputs to PyTorch on fixed fixtures after conversion.
Plan for first-load UX. A gigabyte download needs progress, retry, and cache strategy — not a blank page.
Use agents for boilerplate conversion. Export scripts, webpack config, and glue code are high-leverage delegation targets if acceptance tests exist.
Know the ceiling. Browser inference complements server inference; it does not replace it for large models or low-latency cold starts on first visit.
Conclusion
Willison's Moebius port shows that a meaningful neural network can run entirely in the browser with ONNX, WebGPU, and aggressive caching — and that coding agents can execute much of the conversion grind when humans hold the test harness. For builders, the actionable lesson is narrower: match deployment surface to model size and privacy requirements. Some inference belongs on the server. Some belongs in the tab. The tooling to choose deliberately now exists on both sides.