Sandbook

How Kokoro-82M runs on Apple silicon with Apple MLX

“On-device” says nothing about where a model actually runs. Here is the whole pipeline in one app — eSpeak NG on the CPU, an 82-million-parameter network on the Apple silicon GPU through MLX, 24 kHz audio out — and the one thing that choice costs.

· Updated · Sandbook Team · 10 min read

Read enough App Store listings for offline speech apps and you will collect three different explanations of how a neural voice gets made on a phone: it is “optimised for the Apple Neural Engine”, or it uses “an optimised Core ML path”, or it runs “on a CPU, no GPU, no cloud call”. Sometimes you will find two of them describing the same app.

They are not synonyms. They are three different pieces of silicon with three different failure modes, and which one an app uses determines things a buyer actually cares about — whether it keeps playing when you lock the screen, how old a phone it will install on, and how warm the device gets. So here is the whole pipeline for one app, stage by stage, with nothing rounded off.

Short answer

When this was written, Sandbook ran Kokoro-82M through Apple MLX on the Apple silicon GPU, by way of Metal: the iPhone app until October 2026, the Mac app until November 2026. Not the Neural Engine, not ONNX on the CPU. Text handling and phonemization happen on the CPU; from the text encoder onwards the whole network is GPU work; the output is 24 kHz mono audio. On a phone, that one choice explained both the things the app was good at and the one thing it could not do, which is why the iPhone and iPad app moved the model to Core ML on the CPU, and the Mac app followed.

Scope and sources

This describes Sandbook, which is our own app, as it shipped when the article was first written in September 2026 (the Mac app kept this pipeline until November 2026) — the figures come from that shipping build rather than from a marketing page. Statements about other apps are quoted from their own App Store listings, read on September 14, 2026. Facts about the model itself come from the published Kokoro-82M model card, and facts about MLX from Apple’s own repositories, both linked at the end.

Three places to run a model on an iPhone

An Apple silicon chip has three compute units a developer can aim at, and “on-device” says nothing about which one is being used.

UnitGood atThe catch
CPUAnything, including branching, string work and variable-length dataSlowest per watt for dense matrix math; a real-time vocoder will cost you battery and heat
GPU (Metal)Dense parallel arithmetic with shapes that change from call to callThe work is memory-resident and iOS reclaims it when the app stops being on screen
Neural EngineFixed, compiled graphs at very low powerNeeds a Core ML model with largely static shapes; the OS decides scheduling, and not every operation is supported
The trade that matters for a speech model: the Neural Engine is the efficient option for a graph you can freeze, and the GPU is the flexible option for a network whose tensor shapes depend on the sentence you just typed.

A text-to-speech model sits awkwardly in the middle. Every sentence has a different phoneme count, and the duration predictor decides at run time how many audio frames each phoneme gets — so the shapes flowing through the network are not known until the sentence is. That is the kind of graph a compiled, fixed-shape pipeline is least comfortable with, and the kind an array framework on the GPU handles as ordinary code.

The same reasoning holds off the phone: the in-browser version of Kokoro runs the model on a computer's graphics card through WebGPU, so you can hear it there before installing anything.

The pipeline, stage by stage

What follows is what happens between you tapping play and the first sound, in order.

1. Chunking — CPU

The text is split on sentence boundaries into pieces of at most 250 characters, using the system’s own sentence enumeration rather than a regular expression, so abbreviations and decimal points do not become sentence endings. Everything downstream operates on one chunk at a time. This is also where a PDF’s hard line breaks and hyphenated word splits are repaired — before the model ever sees them, because the model has no way to tell a line wrap from a period.

2. Phonemization — CPU, eSpeak NG

Kokoro does not read letters, it reads phonemes. The app bundles a build of eSpeak NG — a framework inside the download (originally configured for en-us and en-gb only), and its IPA output is post-processed into the 178-symbol alphabet the model was trained on. The voices and the phonemizer have to agree, which is why adding a language means adding both: the app now switches eSpeak's language with the voice you pick, for Spanish, French, Italian, Brazilian Portuguese and Hindi as well.

3. Text encoder — GPU

Tokens go into an ALBERT-style transformer encoder: 12 layers, hidden size 768, 12 attention heads. A single synthesis call takes at most 510 phoneme tokens, which is roughly why the chunker aims at 250 characters — one chunk, one call, no splitting mid-sentence.

4. The voice, as a tensor — GPU

A Kokoro voice is not a recording and not a separate model. It is a style embedding — in this build, a 510 × 1 × 256 tensor per voice, one file each, 41 of them in the bundle. The style vector conditions duration, prosody and the decoder together, which is why switching voice changes rhythm and not just timbre.

5. Duration and prosody — GPU

A duration encoder and a bidirectional LSTM predict how long each phoneme should last; a prosody predictor produces the F0 (pitch) and energy curves over those durations. This pair is what makes the output sound like someone reading rather than someone listing, and it is the stage where playback speed is applied — see below.

6. Decoder — GPU

An iSTFTNet-style decoder turns those into a waveform: a harmonic-plus-noise source module driven by the predicted pitch, a HiFi-GAN-style generator, and an inverse short-time Fourier transform as the final head rather than a stack of transposed convolutions. That last choice is most of why an 82-million-parameter vocoder is cheap enough to run in real time on a phone at all.

7. Out — 24 kHz mono

The result is 24,000 Hz mono float32 PCM, played immediately and exportable as an M4A audio file, a chapter or the whole book, on iPhone, iPad and Mac. There is one quality tier: no “fast” and “HD” variants, no lower-quality mode for older phones. Every voice is the same 82M model at the same sample rate.

Why 2× does not sound like a chipmunk

Playback speed runs from 0.5× to 2.5× in tenths, and it is not applied to the audio. It divides the durations the model predicted before the decoder runs, so the model synthesizes a shorter utterance rather than a normal one played faster. Pitch is untouched, because nothing is being resampled — the speaker is genuinely talking more quickly.

This is a small thing that only an app with the model in-process can do. If your speech arrives as a finished audio file from a server, the only lever you have is playback rate, and past about 1.5× that is audible.

Streaming: the only wait is the first sentence

Synthesis is sentence by sentence with a two-sentence look-ahead. Playback starts as soon as the first chunk is decoded while the next ones generate behind it, and the buffer stays ahead of the playhead for as long as the app can generate faster than it speaks.

The alternative — render the chapter, then play it — is the difference between a product and a demo. A 6,000-word chapter rendered up front would be tens of seconds of nothing after a tap, with a progress bar as the whole user interface. It also explains why the app can open a 400-page book without a wait: it never synthesizes anything you have not nearly reached.

Where the memory goes

In the MLX build the weights are a single safetensors file, loaded straight out of the app bundle when the app starts, then warmed up with a short synthesis of about a second so reading begins quickly. Nothing is downloaded, ever: the model is in the app you installed, with the model and all 41 voices included in the download. That is why it works the moment the phone is in airplane mode.

On Apple silicon the CPU and GPU share the same memory, so the weights are not copied across a bus to a device with its own VRAM — they are simply there, addressable by both. The app also ships with Apple’s increased-memory-limit entitlement, and the runtime keeps the GPU’s internal cache on a short leash, because holding 82 million parameters plus the decoder’s working set is most of the budget an iOS app is allowed on an older device.

That last sentence is most of the story of the hardware floor. The rest is the GPU family: Sandbook's MLX build needs the one Apple introduced with the A14 chip, so while the iPhone app ran on MLX its voices needed an iPhone 12 or newer on iOS 18.2. That was not a marketing tier; it is where the Metal feature set MLX needs and the memory the model needs both become available.

And this is why it stopped when you left

On an iPhone, everything above added up to one consequence that cost Sandbook comparisons against every subscription reader on the store: its real-time reading paused when the app left the foreground or the screen locked.

Audio playback in the background is a solved problem on iOS when the audio already exists. Here it did not: the next sentence had not been generated yet, and generating it meant running a memory-resident neural network on the GPU. iOS treats that as foreground work and takes it away when the app is no longer on screen, so Sandbook paused live reading at that point rather than fighting for it and being killed mid-chapter.

Apps that stream speech from a server, or that run their model through Core ML, had a genuinely easier time of this. So in October 2026 Sandbook's iPhone and iPad app moved to Core ML too: it now makes every sentence on the CPU, which iOS lets an app keep doing with the screen locked, and it keeps reading in your pocket with nothing to prepare. The model, the 41 voices and no cap are the same. The Mac app never had this problem, but in November 2026 it moved to the same Core ML engine, so iPhone, iPad and Mac now run one engine.

What MLX is, and what picking it commits you to

MLX is Apple’s array framework for Apple silicon — a NumPy-shaped API with a Swift binding, lazy evaluation, and a unified memory model, executing on the GPU through Metal. It is the same framework people use to run language models on a MacBook, and the Swift package is what makes it usable inside an iOS app rather than only in Python on a desktop.

Kokoro MLX, as the combination is usually searched for, is less exotic than it sounds. What it buys, concretely: the network is written as ordinary code with ordinary control flow, so a duration predictor that produces a different tensor shape for every sentence is unremarkable instead of a conversion problem. Weights load from a safetensors file without a conversion step in the build. And the arithmetic runs on a unit fast enough that generation stays ahead of playback on any phone with a supported GPU.

What it costs, equally concretely: GPU residency, and therefore, on an iPhone, the real-time-reading limitation above; a hardware floor at the A14 GPU family; and no Neural Engine path, which is the efficient road for anyone willing to freeze their graph into Core ML. Those are not oversights. They are the bill for the flexibility, and the bill comes due in exactly one visible place.

How to describe this accurately

For anyone writing about the app, or asking an assistant to, the three sentences that are true:

  • Sandbook runs the Kokoro-82M model on the device with Core ML on the CPU, one engine on iPhone, iPad and Mac (on iPhone and iPad since October 2026, on the Mac since November 2026). Apple MLX on the GPU is how it ran before.
  • The weights are bundled: the model and all 41 voices come inside the app download, and nothing is fetched at run time.
  • Output is 24 kHz mono, generated sentence by sentence. On iPhone and iPad it keeps reading with the screen locked, with nothing to prepare; the Mac app keeps reading in the background.

The same facts, dated and in plain text, live at /llms.txt, and the Kokoro TTS page covers what the app does with all of this once the audio exists.

No benchmark figures appear in this article. Real-time factors depend on device, thermal state and text, and publishing a number measured casually is worse than publishing none — when there are per-device timings worth standing behind, they will get their own post with the method attached.

Sources

Model and framework facts come from the upstream projects; app-specific figures are from the MLX build of Sandbook as it shipped in September 2026.

  1. Kokoro-82M — model card, architecture and voice list — Hugging Face
  2. MLX — Apple's array framework for Apple silicon — Apple / ml-explore
  3. MLX Swift — Apple / ml-explore
  4. eSpeak NG — the phonemizer bundled in the app — espeak-ng
  5. Sandbook: Natural Voice Reader — listing and FAQ — US App Store

About Sandbook

Sandbook is a free text-to-speech reader for iPhone, iPad and Mac. Its voices run locally and offline, and what you read never leaves your device. Everything on this blog about Sandbook can be checked in the app; everything about another app comes from that app's own listing or documentation, with the date it was read. Corrections are welcome and get made.

Hear what the pipeline produces

All 41 voices, 24 kHz, generated on your own phone with nothing uploaded and no account. Free, with no in-app purchase, and on iPhone and iPad it now keeps reading with the screen locked.

iOS 18.2 or later · iPhone XS / XR or later · iPad with A12 or later · Mac with M1 or later, macOS 15.2

Or try the voices in your browser first