Pocket TTS Guide: Voice Cloning in Your Browser

Pocket TTS is Kyutai’s 100M-parameter text-to-speech model built for CPUs. Unlike cloud TTS APIs, the whole pipeline — tokenizer, text conditioning, latent autoregression, flow-matching refinement, and the Mimi audio codec — runs on your own hardware. OfflineTTS integrates it in the browser with voice cloning by microphone recording or audio upload, 8 built-in voices, streaming playback, and WAV/MP3 export.

Try Pocket TTS with voice cloning in OfflineTTS — no account, no API key, no uploads to a synthesis server.

Quick Facts

What Makes Pocket TTS Different

Most browser TTS models either run a single neural network end to end (Kokoro, Piper) or chain several ONNX graphs with a straightforward encoder-decoder structure (Supertonic). Pocket TTS is different under the hood:

1. Text conditioning. A SentencePiece tokenizer feeds a conditioner that produces a 1024-dimension embedding sequence.
2. Latent autoregression. A streaming transformer predicts one 32-dimension latent frame at a time, maintaining a KV cache across steps — the same streaming pattern used by large language models, but on audio latents.
3. Flow-matching refinement. Each predicted latent is refined with a short LSD (Latent Score Distillation-style) flow-matching loop instead of a fixed decoder step, which is why the model stays small (100M parameters) while producing natural prosody.
4. Mimi audio codec. The refined latents are decoded by Mimi into 24kHz waveforms in 1920-sample frames (80ms each), which is exactly why streaming playback works: the browser can play each 80ms frame as soon as it is generated instead of waiting for the whole file.

This architecture is why Pocket TTS stays usable on plain CPUs: the official tech report measures roughly 6x real-time on an Apple M4. In practice WASM speed still depends on the processor, browser, memory, and text length — measure on your intended device before planning long or batch work.

Voice Cloning: Record or Upload

Cloning is the headline feature, and the OfflineTTS integration supports two input methods:

• Record your voice. Click the microphone button, speak up to 10 seconds of clean audio, and stop. The clip is captured locally and processed immediately.
• Upload an audio clip. Use a WAV, MP3, or other browser-decodable file up to 10 seconds.

Either way the flow is identical: the clip is downmixed to mono, resampled to 24kHz, and passed through the mimi voice encoder, which produces a speaker embedding. That embedding conditions the text-to-speech model so your next generation speaks in the cloned voice.

After cloning, the reference clip appears in a listen-and-compare panel — play the reference, generate text, and A/B the result against the original to judge similarity before you export.

Getting a Good Clone

Cloning quality depends far more on the reference than on the settings:

• Keep it short and clean. 3–10 seconds of single-speaker audio with minimal background noise, reverb, or music gives the best results.
• Use natural, expressive speech. Flat or whispered references transfer flat or whispered delivery.
• Avoid overlapping audio. The encoder averages speaker characteristics; a reference with two speakers produces a blend, not either voice.
• Test with your real script. Names, numbers, and unusual words in the final text are the real test — a pleasant sample sentence proves nothing by itself.

Consent matters. Only clone voices you have permission to use, disclose synthetic audio where platforms or law require it, and review the upstream model terms before commercial deployment. The 8 built-in voice names (alba, javert, etc.) are catalog labels from the model’s conditioning states — not verified identities of named people.

The 8 Built-in Voices

Treat these as discovery labels, not a quality ranking. Compare the same passage in several voices before committing to one.

How Browser Inference Works on OfflineTTS

The OfflineTTS integration uses ONNX Runtime Web on WebAssembly for the default path:

text → SentencePiece tokenizer → text conditioner
     → streaming FlowLM (autoregressive latent frames + KV cache)
     → flow-matching refinement (LSD step)
     → Mimi decoder → 24kHz audio frames
clone: reference audio → mimi encoder → speaker embedding → conditioning

Generation is streamed internally: each 80ms audio frame is decoded as soon as the autoregressive loop produces it. The default playback mode buffers frames and plays the complete result through the shared waveform player; an experimental WebGPU path (jax-js) plays frames as they arrive for true real-time output on capable devices.

Backend choice

• WASM (default) — the proven ONNX path. Supports cloning (record + upload), all 5 languages, and works on any browser with WebAssembly.
• WebGPU — an experimental jax-js path with faster real-time streaming on Chrome/Edge with a GPU. Currently English-only with the 8 built-in voices; cloning on this path is not available yet.

Switch backends with the selector in the model status bar. Changing backends reloads the model on the next run.

Privacy and Caching

• Text and reference audio stay on-device. Neither your script nor your recorded/uploaded clip is sent to a synthesis API.
• Model files come from Hugging Face. The ONNX bundles, tokenizer, and built-in voice states are downloaded at runtime and cached in the browser’s IndexedDB. After the first load, later visits reuse the cache.
• Site analytics remain separate network activity. OfflineTTS uses standard analytics on all pages; the TTS page’s network boundary is documented in the Privacy Policy.
• Cloning is local. The mimi encoder runs entirely in the browser — your reference audio never leaves the device.

Pocket TTS vs Kokoro, Piper, and Supertonic

Choose Pocket TTS when you want to clone a voice (yours, or with consent, someone else’s) entirely in the browser, or when a compact CPU model with streaming output matters. Choose Supertonic for broader language coverage, Kokoro for the largest voice catalog, or Piper for the smallest CPU footprint.

Troubleshooting

• “Custom voice is not available on the WebGPU path” — switch the backend to WASM; cloning currently runs on the ONNX engine.
• Cloned voice sounds off — re-record with a cleaner, shorter, single-speaker reference; avoid music and background noise.
• Model download is slow — the first load fetches up to ~198MB. Later visits use the browser cache.
• Generation is slow on an old laptop — WebAssembly inference speed depends on the CPU. Try short sentences and measure before planning long or batch work.
• Audio is truncated — very long text is chunked; keep paragraphs short and well-punctuated so chunk boundaries land cleanly.

Reproducible Cloning Test

1. Record or upload a 5–10 second clean reference of the target voice.
2. Listen to the reference in the compare panel; note its pitch and pacing.
3. Generate a test sentence with names and numbers, e.g. “Hi, I’m Alex. My number is 555-0142.”
4. Compare the output with the reference; adjust the reference if similarity is poor.
5. Export WAV for further editing or MP3 for sharing, and keep the reference clip with the production file.

Start with a Real Workflow

Pocket TTS is a practical option whenever a consistent voice matters more than a huge catalog: personal narration, demo reels, game characters, accessibility prototypes, or voice-consistent series drafts. Because cloning and synthesis both run locally, iteration is free and private.

Open the Pocket TTS workspace and clone your voice.