All posts

EngineeringSeptember 18, 2026 · 6 min read

How live speaker diarization actually works (and why it's hard)

"Who said what" sounds trivial for humans and brutal for machines. A look at streaming diarization, code-switching, and the failure modes we designed Turnlogue around.

Ask a human to transcribe a meeting and they'll do two things at once without noticing: write down the words, and track who's saying them. Ask a machine to do the same and you've handed it two genuinely hard problems. The words are the famous one. The who — speaker diarization — is the one that quietly breaks most transcription products.

Streaming makes it harder

Offline diarization is a luxury problem: you have the whole recording, so you can cluster all the voice embeddings at once, notice that segments 4 and 37 sound like the same person, and label accordingly. Live diarization gets none of that. The audio arrives in small chunks, decisions have to be made in milliseconds, and there's no going back — the user is reading the transcript during the meeting.

Turnlogue streams audio to a speech engine that attaches a speaker label to every finalized phrase. That's the raw material. But raw labels are just "Speaker A" and "Speaker B" — the product work is everything around them.

Three failure modes we designed around

1. The merged turn

Two people talking in quick succession — one finishes, the other starts within the same breath — can arrive as one utterance. Merge them into a single line and the transcript reads like a monologue. Turnlogue merges only same-speaker fragments (< 1.5s apart), so a rapid exchange keeps its two voices.

2. The language switch

Real multilingual conversations aren't "ten minutes of Spanish, then ten minutes of English." Speakers switch mid-sentence. Auto-detect modes cover a fixed set of languages well — and degrade quietly into gibberish for everything else. That's why Turnlogue pairs auto-detect (great for the major languages, handles code-switching) with an explicit source picker (35+ languages, guaranteed accuracy) — and, when a live auto-detected session drifts outside the covered set, flags it and re-runs the audio with the right model.

3. The dropped connection

A browser tab crashing mid-meeting shouldn't cost the record. Every finalized turn is persisted server-side the instant it arrives — rejoin the session and the transcript is exactly where you left it. The client is disposable; the record isn't.

What "good enough" means

A transcript that's 97% word-accurate but attributes every other line to the wrong speaker is worse than useless in a legal or medical setting — the words are right and the record is wrong. Diarization quality, language detection honesty (flagging uncertainty instead of guessing), and crash-safety are why we treat the transcript as a log, append-only and immutable, rather than a document being edited in place.

If you've ever re-listened to a meeting recording to settle who agreed to what — that's the problem this post describes. Turnlogue solves it while the meeting is still happening.

Put it to work on your own conversations

Live sessions, 10-hour uploads, translations into 50+ languages. Free plan included.

Start free