Your team just demoed a voicebot to the boss. The voice was smooth, the answers were correct, and everyone nodded in approval. Two weeks later in production, integrated into the real call center, a customer calls in asking "where is my order" — and there is 1.8 seconds of silence before the bot speaks. The customer asks "hello?". The bot starts answering the old question, overlapping the "hello". Both sides talk over each other. The customer hangs up and presses 0 to reach a human.
The problem isn't the voice. By 2026, Vietnamese TTS voices have long passed the uncanny valley. The problem is the silence. In human-to-human conversation, the gap between two turns is on average only about 200ms — the human brain is extremely sensitive to this delay. If a voicebot is 1.5 seconds slow, no matter how good the voice is, the customer immediately knows it's a machine, and worse: it feels like talking to someone inattentive.
2026 perception threshold and an 800ms budget
The current standard figure, measured from when the customer finishes speaking to when the bot emits the first sound:
- Under 800ms: smooth, almost like natural conversation.
- 800–1,200ms: acceptable, slightly "thinking long" but not broken.
- Over 1,500ms: clearly a machine. Customers start saying "hello?", repeating themselves, or hanging up.
800ms sounds like a lot, but when broken down into each stage of a traditional pipeline, it's all gone:
| Stage | Typical latency | Notes |
|---|---|---|
| VAD + endpointing (detecting when the user finishes speaking) | 200–500ms | How long to wait for silence to consider a "sentence finished"? |
| STT (final transcript commit) | 100–300ms | Streaming helps, but the final result still requires waiting |
| LLM (time-to-first-token) | 200–800ms | Additional time if tool-calling is used |
| TTS (time-to-first-audio) | 80–300ms | Streaming TTS is already quite fast |
| Telecommunications network + telephony stack | 100–300ms | SIP trunk, codec, jitter buffer — things WebRTC demos lack |
Accumulating "everything is a bit slow" scenarios puts you at 1.2–2 seconds. This is why the classic STT→LLM→TTS pipeline usually falls into the 600–1,500ms range, and most real-world implementations sit in the upper half of that range.
The biggest budget eater isn't the LLM as many think, but endpointing: the bot must decide "has the user finished speaking" by waiting for silence. Waiting 700ms to be safe costs 700ms before doing anything. Waiting 300ms causes the bot to interrupt when the user is just hesitating while reading an order code.
Legacy pipeline vs native speech-to-speech
Third-generation platforms (2025–2026) solve this problem in two directions.
Direction 1: optimizing the pipeline to its limit. Still STT→LLM→TTS but everything is streaming and overlapping: STT returns continuous partial transcripts, LLM starts reasoning on partials before the user finishes speaking, TTS emits audio from the first token, and a dedicated turn-taking model is used instead of pure silence-based VAD. Retell AI announced sub-600ms using this approach; Telnyx pushed it down to sub-200ms round-trip by owning the telecommunications infrastructure — placing inference next to the media server to avoid routing through the internet.
Approach 2: native speech-to-speech. Model receives audio and returns audio, with no intermediate text step. Time-to-first-audio is ~100ms — fast enough that latency issues almost disappear. But there is a significant trade-off:
| Streaming pipeline | Speech-to-speech | |
|---|---|---|
| Latency | 400–800ms (well-optimized) | ~100ms TTFA |
| Content control | High — text is visible, can be filtered, and logged | Low — audio is direct, difficult to guardrail |
| Tool-calling (lookup, CRM) | Mature, reliable | Improving but less stable |
| Voice/language switching | Arbitrary TTS composition | Model-dependent |
| Cost | Granular control over each stage | Usually more expensive, fewer options |
For customer service use cases requiring order data retrieval, customer authentication, and ticket logging — i.e., heavy tool-calling — an optimized streaming pipeline remains the more pragmatic choice at this time. Speech-to-speech is suitable for open conversations with low business logic requirements.
Turn-taking and barge-in: harder than latency
Latency is measurable. Turn-taking is not, and it is what kills the experience.
Barge-in — customer interrupts while bot is speaking — mandatory. Bot reads a 30-second return policy while the customer already knows the answer by the 5th second but cannot interrupt; that is torture. True barge-in means: stop TTS within ~100–200ms, flush the audio buffer pushed into the carrier's jitter buffer (this part is often forgotten — the bot "has stopped" but the customer still hears 1 second of residual audio), and the LLM must understand the context of "where I was interrupted."
Distinguish interruption from noise even harder. Customers call from Vietnamese streets: motorbike sounds, horns, street vendors, neighbor's TV. Naive VAD will treat a horn as the customer speaking, stopping the bot mid-sentence. Conversely, if a customer says "um... wait" in a low voice, it must be recognized. This is why the new generation of platforms no longer uses pure energy-based VAD but runs a dedicated turn-taking model — a small model specialized in predicting "has the customer finished speaking" based on both prosody and content, instead of just counting milliseconds of silence.
And intentional silence: the customer is searching their wallet for an order code. 4 seconds of silence. If the bot interrupts with "Are you still there?" at the 2nd second, it's a failure. Logic is needed: if the bot just requested information that requires looking up, the waiting threshold must be extended.
Vietnamese: where to give up and use DTMF
Vietnamese STT in 2026 is good for casual conversation, but still weak in the most critical business scenarios:
- Phone numbers: "khong chin ba ba..." — is "ba" the number 3 or a filler word? Customers speak fast, swallow digits, or read in pairs.
- Proper names: Dung/Dụng/Dung, Ngân/Ngần — a single tone error results in the wrong person, failing the CRM lookup.
- Order ID: "SPX-VN-0847..." mixing Latin characters, numbers, and hyphens — a nightmare for any STT.
Pragmatic approach: don't try to win with AI on these points. DTMF Fallback — "please press the numbers on your keypad" — for numeric input. 100% accurate, customers have been used to this since the IVR era, and it's faster than a bot re-confirming "is it nine three three...". For order IDs, look up orders via the authenticated phone number and list them for the customer to select, instead of forcing them to read the code. A good voicebot is one that knows when not to use voice.
Checklist for real-world measurement, not demo measurement
Latency figures in a WebRTC demo from an office laptop are meaningless. Real-world measurement means:
□ Gọi qua SIM thật (Viettel/Vina/Mobi), qua SIP trunk production,
không phải WebRTC nội bộ
□ Đo p95, không phải p50 — khách nhớ cuộc gọi tệ nhất,
không nhớ cuộc trung bình
□ Đo end-to-end: mốc bắt đầu là khách NGỪNG nói,
mốc kết thúc là audio ĐẦU TIÊN tới tai khách
□ Đo lúc cao điểm: LLM provider nghẽn giờ hành chính,
TTFT có thể x2-x3 so với đêm
□ Test barge-in: ngắt lời bot ở giây 2, đo bao lâu bot im hẳn
□ Test tiếng ồn: gọi từ ngoài đường, bật quạt, mở TV
□ Test số + tên riêng tiếng Việt: 50 số điện thoại,
50 tên có dấu dễ nhầm — đếm tỷ lệ đúng
□ Ghi log từng chặng (endpointing / STT / LLM / TTS / network)
để biết chặng nào ăn budget khi p95 vỡ
If the p95 end-to-end on the actual telecommunications line exceeds 1,200ms, do not launch. Fix the architecture first, not the prompt.
This industry is repeating the exact lesson of web performance from fifteen years ago: users don't praise a fast website, they just abandon a slow one. Voicebots are the same — customers never praise a bot for responding quickly, they just hang up when it is slow. Voice is what you demo to the boss; latency is what your customers actually hear.
