AI Agents
Topshe: voice AI that runs entirely in your browser
· Updated · 6 min read
S
Get AI Agent RFP Template
The RFP I use for agent discovery — scope, eval, and pricing.
No spam, unsubscribe anytime. I only email teardowns.
The short answer
Topshe is a voice assistant that never calls home. Named after the teenage assistant in Satyajit Ray's Feluda stories — the one handed the research, the planning, and the grunt work — it runs wake word to speech-to-text to a quantised Qwen 2.5 LLM inside the browser over WebAssembly. No server, no API key, no data leaving the phone.
Say "Topshe, what's on my calendar?" into your phone, put it in airplane mode, and ask again — it still answers. Most people assume a voice AI assistant must phone a cloud; the entire point of this project is that it doesn't. The whole stack runs locally: React 19 shell, Web Speech API for input, a quantised Qwen 2.5 1.5B Q4 model inferring through WebAssembly (via wllama), offline-capable as a PWA. The whole model binary is ~900 MB cached in IndexedDB — you download it once.
The name is the spec
Topshe is Tapesh Ranjan Mitter, the teenage assistant in Satyajit Ray's Feluda detective stories — the one handed the research, the notes, and tomorrow's stake-out plan while the detective thinks. Feluda thought; Topshe researched, planned, and kept records. That division of labour is the product spec: a junior assistant you can shout random tasks at, who quietly researches, plans, and remembers — and never gossips, because nothing ever leaves the device.
What this Topshe took as his tasks
- Research — WebSearch, weather, stock, and news tool handlers; ask for the current Sensex or tomorrow's forecast, no server call.
- Plan — todos, reminders, focus timer, and alarms with voice snooze; a "today plan" list greets you in the morning.
- Records — calendar and email via OAuth, conversation history in IndexedDB, and a simple RAG embedding pipeline so "remember that" persists on-device.
- Registry — every capability hangs off a tiny tool-registry event bus rather than a server route.
The repo is public: github.com/saswatasg/topshe. Ten commits in, the personality system, contacts, clipboard, files, and REPL shipped to make the assistant's "random tasks" land somewhere.
The pipeline, end to end
- Wake word — Picovoice Porcupine (WASM) runs continuous keyword spotting on the audio stream. "Topshe" triggers; ~50 ms latency on device.
- Speech-to-text — Web Speech API (browser native) or Whisper.cpp WASM fallback. Streaming partials feed the LLM as you speak.
- Intent parse + tool dispatch — the quantised Qwen 2.5 1.5B Q4 (~900 MB,
wllamaruntime) receives the transcript, decides if a tool is needed, emits a structured call. - Tool registry — a 200-line event bus maps tool names to handlers:
web_search,weather,stocks,news,calendar,email,todo,reminder,timer,rag_remember,rag_recall. Each handler is pure client-side or OAuth-to-your-own-accounts. - Response synthesis — the model composes a natural-language answer from tool results.
- Text-to-speech — Web Speech Synthesis or Piper WASM for offline TTS.
No network hop. No API key. The only bytes leaving the device are your explicit OAuth grants to Google/Outlook calendars.
The latency budget (why each stage is sized the way it is)
The whole loop has to feel conversational, which in practice means staying under ~2 seconds from end-of-speech to first spoken word. Where the time actually goes on an M-series laptop vs a mid-tier Android:
| Stage | Laptop (M2) | Mid-tier phone | Notes |
|---|---|---|---|
| Wake word detect | ~50 ms | ~80 ms | Porcupine WASM, always-on |
| STT (Web Speech) | streaming | streaming | partials feed the LLM early |
| LLM first token | ~300 ms | ~900 ms | Qwen 2.5 1.5B Q4 via wllama |
| LLM full response | ~1.2 s | ~3.5 s | ~40 tokens typical reply |
| TTS start | ~100 ms | ~200 ms | Web Speech Synthesis |
The phone column is why the model is 1.5B and not 7B. The latency budget picked the model, not the leaderboard.
Why edge, really
- Privacy without a story. Nothing leaves the device — no transcripts recorded by a vendor, no telemetry, no "improve the model" checkbox.
- $0/month with real inference, not an API token funnel. You own the compute.
- PWA — works as an app, no install, offline capable. Install to home screen, runs in airplane mode.
Edge vs cloud, honestly scored
| Dimension | In-browser (Topshe) | Cloud API assistant |
|---|---|---|
| Privacy | Nothing leaves device | Vendor sees every prompt |
| Marginal cost | $0 | Per-token, forever |
| Reasoning depth | Small-model ceiling | Frontier-model quality |
| Offline | Full function | Dead |
| First-run setup | ~900 MB download | Instant |
| Latency (simple ask) | Sub-second local | Network + inference |
The right answer for most products is a hybrid — local for intent parsing and private data, cloud for heavy reasoning. Topshe deliberately refuses the hybrid to prove where the pure-edge ceiling is.
The agent personality system (the v2 leap)
Version 1 was a tool-caller. Version 2 added a personality system — a small system prompt plus a few-shot memory block that persists across sessions in IndexedDB. The assistant now:
- Remembers your name, preferred name for it ("Topshe" / "assistant"), and tone preference (terse / conversational).
- Maintains a "running context" of the last 5 turns compressed into a summary, so "remember that" works without blowing the context window.
- Has a REPL mode — type
/to drop into a raw prompt sandbox for debugging tool calls. - Exposes contacts, clipboard, and file read/write (via File System Access API) as tools, so "send this to Anirudh" or "save this as a note" work end-to-end.
This turned "a voice tool" into "an agent you talk to daily." The personality system is ~300 lines of prompt + storage logic — small enough to audit, big enough to feel like a consistent character.
The honest tradeoffs
- Model size vs quality. Quantisation shaves latency but taxes clever summarisation; the assistant is snappier than deep, by design. A 7B model would hallucinate less but wouldn't fit on many phones in-browser.
- Memory ceiling. Not every mobile device runs a 7B model comfortably in-browser; we size the model to the device (auto-detect RAM, pick 1.5B or 3B).
- Wake-word environments. WASM audio wakes fast in quiet rooms; speaker noise, AC hum, and cafe chatter still puzzle it. Porcupine helps but isn't magic.
- OAuth is the only cloud. Calendar and email need your explicit grant. If you don't connect them, those tools simply don't appear — graceful degradation.
What it demonstrated
Edge LLMs are production-viable for small systems with clear intents — a light assistant, a noise tool, a vault-adjacent copilot. They are not (yet) full cloud-grade chat without a compute fallback.
TGB Hunt and Topshe bookend the same taste: I keep reaching for the smallest system that gives the product to the user. Topshe is the smallest.
What I'd do differently
- Ship wake-word tuning before claiming "hands-free" — the demo works, the hallway doesn't.
- Benchmark on a mid-tier phone. We demoed on silicon that flattered the WASM numbers; a Snapdragon 7-series shows 2–3× latency.
- Build a WASM Whisper fallback for STT when Web Speech is unavailable (Safari, some Androids).
The projects page holds the live demo link and the open-source repo — 10 commits, v2 personality + contacts + clipboard + files + REPL.
Frequently asked questions
Can a large language model run in a browser?+
Yes for small quantised models. Topshe runs Qwen 2.5 1.5B Q4 through WebAssembly on a phone; bigger models need a machine with more RAM.
Why run AI in the browser instead of an API?+
Privacy and $0 marginal cost — no prompt leaves the device, no vendor API bill on a quiet day.
Is browser voice AI practical for daily use?+
Today: yes for dictating and light tasks. Browser WebAssembly ticks under a second on modern devices; heavy reasoning still offloads.