Semantics on the device
A neural voice with 82 million parameters already runs inside this browser tab. The question is whether a model that understands word sense can join it — offline, inside a 150-millisecond frame, without a download nobody would accept.
What this piece argues
- An 82M-parameter voice already runs in-browser, which proves useful models fit
- Scored context windows are fast and explainable, and fail on irony and metaphor
- The real constraints are latency, determinism, download size and battery
- It runs locally or it does not ship — the alternative is uploading what you read
There is a version of this app in which the machine understands the sentence. Not pattern-matches it, not scores a window of neighbouring words against a table — understands, in the narrow and unromantic sense of knowing which meaning of a word is in play here. It does not exist. But the reason it does not exist has moved, in the last few years, from physics to engineering.
The existence proof is already inside the product. KittenTTS nano, an open-weight neural voice with 15 million parameters, runs in the browser on your own machine, producing eight voices, with nothing uploaded. Five years ago that sentence would have described a server. It now describes a tab. If a generative speech model fits, a much smaller model that only has to answer a single classification question — which sense of this word is this — fits comfortably, at least on the arithmetic.
Arithmetic is the easy part. What follows is the rest of it: what we do today, where it breaks, what a local model could plausibly repair, and the four constraints that make this much harder than a capability demo suggests. None of it is in the product. Some of it may never be.
What we do today, and why it is not stupid
Every sense in the curated lexicon carries context cue words. When a target word arrives, the app scores those cues against an eight-word window either side, with frequency rank breaking ties. bank in a sentence containing river, grassy and water resolves to a sloping edge; bank near loan and walked to resolves to an institution. Around 125 of the most treacherous words in English are handled this way, and everything else falls through five resolver tiers to a procedural mark that cannot fail.
Three properties of that method are worth defending before replacing it. It is fast — a scoring pass over sixteen tokens costs effectively nothing. It is explainable: you can ask why a picture appeared and get a list of the cues that fired, which matters when the picture is wrong. And it is deterministic, so the same sentence produces the same image on every machine, every time. Those are not accidental virtues. They are most of the reason the thing is trustworthy at all.
Where an eight-word window fails
It fails in four families, and we can name all of them. Irony, where the cues in the sentence point at the meaning the writer is not using. Metaphor, where the argument collapsed wants a structural image and not a building. Technical reuse, where a common word is doing specialist work — a kernel in an operating system, a ring in algebra, capital in economics — and the surrounding eight words are ordinary English. And long-range reference, where the sense was fixed by a definition four paragraphs back and there is no mechanism in a sliding window that can reach it. We go through the mechanics of the whole problem in two banks, one word.
A local sense model would not replace the deterministic path. It would sit behind it. The scoring pass carries the easy cases, which are the overwhelming majority, at zero cost and full explainability; the model is asked only when the scores come back flat, when the window contains no cues at all, or when the word is one of the known-hard classes. That division matters for reasons that are practical rather than architectural: it keeps the compute budget small, it keeps most images explainable, and it keeps the failure surface narrow enough to inspect.
The trajectory, without dates
Several things have been moving in the same direction at once, and it is the combination rather than any one of them that changed the picture.
- Quantisation. Storing weights at lower precision cuts model size several-fold with modest quality loss, which turns a download you would never ask a reader to accept into one you might.
- Distillation. Training a small model to imitate a large one, on the narrow task you actually need, rather than shipping general capability you will never call.
- WebGPU. Access to the machine’s graphics hardware from inside a page, which is the difference between a model that runs and a model that runs in time.
- Small embedding models. Sense disambiguation does not need a chat model. It needs vectors good enough to separate meanings, which is a far cheaper object.
- WASM runtimes. Compiled inference engines that run the same way in every browser, which is what makes “it works on your machine” a claim rather than a hope.
We are deliberately not attaching a year to any of it. The honest position is that each of these lines has moved steadily and none of them has a schedule, and that a forecast with a date on it is a marketing device rather than an estimate.
The constraints people skip
Capability demos are usually run without a clock, on a laptop plugged into the wall, with the download already cached. Our situation is none of those. Here is what the model would actually have to satisfy.
The latency figure is the one that does most of the damage. At 400 words per minute a word occupies 150 milliseconds, and the icon has to be resolved before the word appears, not after — an image that arrives late is not a late image, it is the wrong image paired with the next word. Reading ahead in the text buys headroom, and a couple of frames of lookahead gives a budget nearer a quarter of a second, but the budget is bounded by the fact that a reader can rewind, jump, or change pace mid-passage and the stream has to keep up.
| Constraint | The budget | Why it bites |
|---|---|---|
| Latency | ≈150 ms per word at 400 wpm | Resolution must complete before the word is shown, not after it |
| Determinism | Same sentence, same picture, twice | A picture that changes between runs makes the whole channel untrustworthy |
| Download | Once, then cached forever | A 200 MB download for a free reading app is not a trade-off, it is a refusal |
| Battery | Phone, on a train, unplugged | Per-word inference across a chapter is thousands of calls, not one |
| Locality | Runs on device or does not ship | The alternative is uploading everything anyone reads |
Determinism deserves more attention than it usually gets. A model that samples will occasionally produce a different sense for the same sentence, and a reader who sees a different picture on the second pass does not conclude that the model is probabilistic. They conclude the tool is unreliable, and they are not wrong to. Greedy decoding, fixed seeds and a cache keyed on the sentence all help, and none of them is free.
A picture that changes between runs isn’t a probabilistic feature. It reads as a broken one.
Why determinism is not optional
The line we have already drawn
One constraint is not an engineering trade-off but a commitment, and it is the one most likely to cost us the capability. The app is a single HTML file with no account, no backend and no telemetry. Documents are parsed on your own machine — PDF, Word, EPUB, HTML, plain text — and nothing leaves it. That property is why people are willing to put a medical letter, a legal bundle or their own unfinished writing through it.
The easy version of everything above is an API call. Send the sentence, receive the sense, done in 80 milliseconds with a model nobody has to download. It would work today. It is also, in practice, a system that uploads every word its users read, and there is no configuration of that we are willing to ship. So the ceiling on this feature is whatever fits on the device. If a good local sense model never becomes small enough, the feature never arrives, and we would rather lose the feature than the property.
The version we can actually imagine
It is smaller and duller than the one that sounds exciting. Not a model that reads the document and understands the argument — a classifier a few megabytes across, quantised, distilled onto a fixed sense inventory, invoked on perhaps one word in twenty, running behind a deterministic path that still does the ordinary work. It would fix a fraction of the wrong pictures. It would fix the ones that are most visibly wrong, which matters more than the fraction suggests, because a badly wrong image does not merely fail to help; it competes with the sentence you are reading.
That is the whole prize: fewer sentences where the second channel argues with the first. It will not raise anyone’s reading rate, because nothing raises anyone’s reading rate. It would make the images right slightly more often, on exactly the material where being wrong is most expensive. We think that is worth several years of work, and we would rather describe it at that size than at the size it would take to make a good announcement.
A note on what this is. Signal is written in-house by the team that builds Reader Inc., so treat it as an argument rather than a review. Nothing here is medical, psychological or educational advice, and the app is not a treatment, therapy or diagnosis for any condition. Where we describe research we describe it in general terms; where we are reasoning past the evidence we say so. The app is free, runs entirely on your own device, and ships with a comprehension test switched on — which means you can check every claim we make against your own reading rather than taking our word for it.
Read next
Two banks, one word
Every image in the stream depends on a decision made in a quarter of a second: which sense of this word did the sentence mean. Get it right and nobody notices. Get it wrong and you have injected a meaning that was never there.
Nothing left blank
A picture dictionary that covers ninety-four per cent of English fails on the words that were carrying the meaning. Five resolvers, tried in order, with a last one that cannot fail — and an honest account of what the last one is worth.
The next decade of reading machines
Forecasts about reading technology usually describe the screen and forget the reader. Here is an attempt at the opposite — one constraint that will not move, five that will, and the only frontier we think is worth competing on.