The Tamil voice AI stack in 2026 — what actually works in production
Building voice AI that understands and speaks Tamil is now feasible at production quality. Here is the exact stack we run, the failure modes we hit, and the numbers we care about.
Two years ago, "Tamil voice AI" meant a chatbot that read Tamil words with an American accent and heard them wrong. That's no longer true. Here's the stack that we run in production, the parts that break, and what we look at day to day.
The stack
- STT (speech → text): Google Cloud Speech-to-Text with
latest_longmodel anduseEnhanced: true, primaryta-IN, alternatesen-IN,hi-IN,en-US. - LLM: Gemini 3.1 Flash Lite streaming — cheap, fast, handles Tanglish (mixed Tamil-English) well when prompted right.
- TTS (text → speech): Sarvam Bulbul v3 with speaker
neha. Better than any other production Tamil TTS we've tested. - Voice pipeline: browser MediaRecorder for capture, Web Audio API for playback, AudioContext unlock pattern for iOS Safari.
- Silence detection: AnalyserNode + 1.4s silence threshold. Simpler than VAD, works well for turn-taking.
What breaks
1. Transliteration bleed. The LLM will spell brand names in Tamil script if we're not careful. "Hantas AI" becomes "ஹாண்டாஸ் ஏஐ". Fix: explicit system-prompt rule — "never write English brand names in Tamil script."
2. Sentence boundaries. Tamil sentences don't always end in "." —
। and ॥ show up. Our TTS-chunker handles both.
3. iOS Safari audio. AudioContext won't start outside a direct
tap handler. So we unlock on the mic-tap, keep the context alive, and
reuse it for TTS playback. Same trick for speechSynthesis.
4. Recording MIME. Chrome speaks audio/webm, Safari
speaks audio/mp4. Try MIME candidates in order or you'll get silent
audio.
5. Language detection. If a Tamil speaker starts an English word, the LLM sometimes flips locale mid-reply. We route language on the input Unicode range, not on model guess — much more stable.
The numbers we track
- First-word latency — user stops speaking → user hears reply's first word. Target: under 1s.
- STT WER on our own eval set — 300 clips of real Trichy Tamil, mixed with Tanglish. Target: under 12%.
- TTS mean opinion score from real users — target: 4.2/5 on Tamil, 4.4/5 on English.
- Cost per conversation — currently around ₹1.20 per 5-minute session.
What's next
Streaming STT (partial transcripts as user speaks) and lower-latency TTS. We ship what our clients need — voice AI is genuinely a solved problem for Tamil now.
Building voice AI in an Indian language? We ship this every week — hantasai.com/book-demo.