Hi HN — I’ve been working on a real-time conversational AI, and one of the biggest problems is speech latency. As I was experimenting with various TTS systems I stumbled upon Pocket-TTS by Kyutai. It runs on the CPU which made it a great candidate since it keeps speech synthesis away from GPU resources.
As I was using it I noticed that it didn't solve the latency issues fully due to leading silence and artifacts generation (quite common with all these systems). So I started poking under the hood with the help of agents to try and improve it.
On m4 max, the new browser build runs around ~14x realtime on longer utterances, and around ~3.5x realtime on an iPhone 16 Pro. The native CPU path is faster still: roughly 32–33x realtime warm, with first useful audio around 30ms on my benchmarks. Other systems will vary; my Windows browser results were slower but still usable. This does not introduce a new model or claim better voice quality. The goal was to preserve the intended PocketTTS quality while making it fast enough for low-latency interactive scenarios.
I decided to open source it in case it's useful for anyone else, or at least as a demo of what things that are finally possible now that the barrier of trying stuff has been lowered.
EDIT: Keep in mind it downloads 67mb of data on load.
Are you doing this to unlock natural human-to-model conversations? I was speaking to an AI-demo-assistant yesterday and I think I get whay you are talking about when you say the delay has to be really small.
Yes, you can see an early demo video in the page, of a very early version (I decided not to post the final wip yet).
I’m building an AI gaming companion. Essentially an AI NPC (a speaking dog, think dogmeat of fallout on steroids) that is grounded in the game world, can perceive what’s happening, perform actions, follow complex multi-step instructions, and strategize with the player. I want it to feel natural, so I 'm aiming for real audio conversationality (no push to talk & wait), so audio is important, but also... every millisecond trimmed from audio can be dedicated to heavier AI systems (caching helps heavily there, but extracting player's intent before it even reaches an LLM is probably the biggest trick).
Budgets are pretty tight, ~400ms for everything. I have optimized qwen-3-asr and added streaming detection to it, as well as pocket-tts (this) and qwen-3-tts, with a goal of ~70ms for audio for both detection/creation and then ~350ms for the planning.
It's interesting that gpt-live was just announced today, planning to make more things open source as well as the learnings while building this.
Hi HN — I’ve been working on a real-time conversational AI, and one of the biggest problems is speech latency. As I was experimenting with various TTS systems I stumbled upon Pocket-TTS by Kyutai. It runs on the CPU which made it a great candidate since it keeps speech synthesis away from GPU resources.
As I was using it I noticed that it didn't solve the latency issues fully due to leading silence and artifacts generation (quite common with all these systems). So I started poking under the hood with the help of agents to try and improve it.
On m4 max, the new browser build runs around ~14x realtime on longer utterances, and around ~3.5x realtime on an iPhone 16 Pro. The native CPU path is faster still: roughly 32–33x realtime warm, with first useful audio around 30ms on my benchmarks. Other systems will vary; my Windows browser results were slower but still usable. This does not introduce a new model or claim better voice quality. The goal was to preserve the intended PocketTTS quality while making it fast enough for low-latency interactive scenarios.
I decided to open source it in case it's useful for anyone else, or at least as a demo of what things that are finally possible now that the barrier of trying stuff has been lowered.
EDIT: Keep in mind it downloads 67mb of data on load.
this looks (and sounds) like a pretty cool thing!
Are you doing this to unlock natural human-to-model conversations? I was speaking to an AI-demo-assistant yesterday and I think I get whay you are talking about when you say the delay has to be really small.
What is the bigger project you're working on?
Yes, you can see an early demo video in the page, of a very early version (I decided not to post the final wip yet).
I’m building an AI gaming companion. Essentially an AI NPC (a speaking dog, think dogmeat of fallout on steroids) that is grounded in the game world, can perceive what’s happening, perform actions, follow complex multi-step instructions, and strategize with the player. I want it to feel natural, so I 'm aiming for real audio conversationality (no push to talk & wait), so audio is important, but also... every millisecond trimmed from audio can be dedicated to heavier AI systems (caching helps heavily there, but extracting player's intent before it even reaches an LLM is probably the biggest trick).
Budgets are pretty tight, ~400ms for everything. I have optimized qwen-3-asr and added streaming detection to it, as well as pocket-tts (this) and qwen-3-tts, with a goal of ~70ms for audio for both detection/creation and then ~350ms for the planning.
It's interesting that gpt-live was just announced today, planning to make more things open source as well as the learnings while building this.
This works great! even on a phone with 2GB of RAM finally, somebody made something useful.