Zyphra's ZONOS3-Mini clones voices in real time on a phone

Felix Laurent 9 Nov 2026 5 min read

Models

Zyphra has released ZONOS3-Mini, an open-source text-to-speech model with voice cloning that runs in real time on a phone. The San Francisco lab published it on November 9, 2026, with weights under the Apache 2.0 license. The model can copy a voice from a short clip and lets users control emotion and pace down to single words.

ZONOS3-Mini is a sparse mixture-of-experts model with 800 million parameters, of which about 100 million are active for each token. Zyphra's previous model, ZONOS2, used 8 billion parameters. The new one is ten times smaller and, by Zyphra's own tests, sounds better.

What ZONOS3-Mini can do

The model covers two jobs. It can clone an existing voice from a reference recording of up to 60 seconds, with no transcript needed. It can also build a new character by setting age, gender, accent and vocal texture.

Delivery controls sit on top of that. Users can steer pace, emotion, projection and liveliness, with presets such as happy, angry, scared, raspy or deep. Zyphra does this with activation steering, which nudges the model's internal signals in a chosen direction instead of retraining it.

Word-level control works in two ways. During streaming, the model tracks which word it is speaking and applies settings to that span. After generation, a separate edit model can redo a single phrase. Zyphra's example whispers only the words "I didn't say" in a line. This model follows the VoiceCraft recipe and needs Qwen3-ASR for the transcript.

Zyphra lists the limits plainly. The controls "do not guarantee a specific emotion" in its words. Pace steering can shift loudness and pitch, and there is no breathiness control. Text normalization covers nine languages, including English, Mandarin, Japanese, Spanish and German.

The bigger claim is about where all this runs.

How ZONOS3-Mini runs on a phone

On a Pixel 9, the model starts speaking in under 570 milliseconds inside a browser, according to Zyphra's blog. On an M3 MacBook Air, time to first audio is under 150 milliseconds. Zyphra's X thread gives looser figures of under 800 and 450 milliseconds for the same devices, without explaining the gap.

On a server, the numbers are sharper. On one Nvidia H100, median time to first audio is 80 milliseconds with eight requests at once. Peak throughput reaches 457 seconds of audio per second of compute.

Two changes make that possible. Only about an eighth of the weights work on each step, with 64 experts and two picked per token. The model also uses a new codec, ZAC-1, that turns speech into far fewer tokens. Zyphra says this cut the tokens needed per second of speech from 774 to 150.

The old codec, DAC, ran at 86 frames per second and is hard for a model to predict. ZAC-1 runs at 15 frames per second at 44.1 kHz. Zyphra says a model trained on it made ten times fewer word errors than one trained on DAC.

The weights ship with runtimes for most hardware:

Browser: WebGPU, with no install, cached after the first visit
Apple Silicon: an MLX server
CPU and other GPUs: a C++ ggml server and command-line tool
Servers: day-one support in SGLang, with an OpenAI-compatible speech endpoint

Training took 112 Nvidia H100s for 2.5 days, followed by DPO and GRPO post-training. Speed alone does not sell a voice model, so Zyphra also published a quality test.

How ZONOS3-Mini compares with other TTS models

Zyphra built a new benchmark for the launch, ZTTS2-Eval. It covers read speech in 35 languages and recordings from the wild in 20 languages. It scores word error rate, speaker similarity, rhythm and pitch.

From these, Zyphra built its own TTS Capability Index over 420 combinations of metric, test set and language. On that index, Zyphra says ZONOS3-Mini ranks first among open models, ahead of Fish S2 Pro and MOSS TTS v1.5. The company published no score values, only charts and listening samples.

Zyphra's chart: ZONOS3-Mini quality score about 0.8 vs 1.0 for OmniVoice, and 457 s of audio per second vs about 16 on one H100 — Approximate values read from Zyphra's Pareto chart, November 9, 2026

The chart in Zyphra's thread shows more nuance. It plots quality against speed for 12 open models on one H100. OmniVoice scores higher on quality, but it returns audio only when a clip is finished and takes about four seconds. VoxCPM2 and Fish Audio Drama 3 sit close to ZONOS3-Mini on quality, but they produce four to six times less audio per second.

Among models as fast as ZONOS3-Mini, the gap is wide. Qwen3-TTS at 0.6B and 1.7B parameters starts slightly faster but scores far lower. ZONOS2 at 8B lands below the new model on both axes.

All of these results come from Zyphra's own test. No outside group has rerun them yet, and the listening comparison with ElevenLabs and Cartesia is a set of samples. The model also enters a crowded field.

Other on-device voice models

Small open voice models have multiplied since early 2025. Kokoro, with 82 million parameters, runs on a CPU but cannot clone voices. Kyutai's Pocket TTS, released in January 2026, clones voices with 100 million parameters on a laptop CPU.

Alibaba's Qwen3-TTS came out in January 2026 in 0.6B and 1.7B versions under Apache 2.0. Resemble's Chatterbox Turbo has 350 million parameters and adds a built-in watermark. Fish Audio's S2 Pro, released in March, uses a research license that requires a separate deal for commercial use.

ZONOS3-Mini combines cloning, fine delivery control and an Apache 2.0 license in a model built for phones. Few rivals offer all three. Zyphra's own path shows how fast the field moved to get here.

Zonos models so far

Zyphra released Zonos-v0.1 in February 2025 as two 1.6B models trained on about 200,000 hours of speech. A reporter at The Register cloned their own voice from a 24-second clip and fooled friends and family at first listen. Rendering about 50 words took nine to ten seconds on a workstation GPU.

ZONOS2 followed on June 12, 2026, as an 8B mixture-of-experts model with 900 million active parameters. It trained on more than six million hours of audio and ran only on Nvidia GPUs under Linux. ZONOS3-Mini keeps the open license and moves the same idea down to a phone.

Voice is one part of Zyphra's wider plan. The company also builds ZAYA language models trained on AMD chips and Maia, an AI agent for teams. Forbes reported in May that it was raising $500 million at a valuation of at least $5 billion, with AMD committed.

Is ZONOS3-Mini worth trying

For game studios, narration tools and voice assistants, ZONOS3-Mini removes the main cost of cloud voice. A character can speak on the player's device with no API bill, no network delay and no audio leaving the phone. Apache 2.0 allows commercial use without the license deals that Fish Audio requires.

There are open questions. The quality ranking is Zyphra's own, and the phone latency figures differ between the blog and the thread. The CUDA server pulls about 15 GB of checkpoints, and the repo says some may need a Hugging Face token. Zyphra also mentions no watermark or consent check for cloning, which matters for a tool this easy to run. For developers who need an open voice model on device today, ZONOS3-Mini is the strongest option Zyphra has shipped. It is worth testing against Qwen3-TTS and Pocket TTS on real hardware.