Transcribing a recording with more than one speaker

Diarisation is a separate model, not a setting

Speech recognition answers what words were spoken. Speaker diarisation answers how many distinct voices there are and which stretch belongs to which. They are independent problems, and the second is harder because it has no vocabulary to constrain it.

Products that show you Speaker 1 and Speaker 2 are running a second model that clusters voice characteristics across the recording, then stitching its segment boundaries onto the transcript's. That stitching is where the visible errors come from, which is why a label often flips mid-sentence.

Whisper does not do this. It returns text and timestamps and no speaker information of any kind, so there is nothing here to switch on. Adding it would mean shipping a second model to the browser alongside the 45MB or 80MB of speech weights, for a feature that would still be wrong often enough to need checking.

Be suspicious of free automatic labels

Automatic diarisation is good under conditions most real recordings do not meet: distinct voices, little crosstalk, and either one microphone per speaker or a good room. A call with two people on laptop microphones and one on speakerphone is close to the worst case.

The failure mode is not an obvious error, it is a confident wrong label. The transcript reads as clean dialogue and attributes a sentence to the wrong person, which in a record of a client call or a disciplinary meeting is worse than no labels at all. Unlabelled text makes you check the audio. A wrong label stops you checking.

So the question is not whether a tool offers labels. It is whether you are going to verify them, because a label nobody verified is not evidence of anything.

Labelling by hand, and why it is fast

For two or three speakers this is a few minutes of work. Transcribe normally, then move through the lines against the waveform: a speaker change almost always sits at a pause, which is visible as a dip. Type the name into the start of the first line after each change.

It is quick because you are not relistening to the whole recording. You are looking at a picture of it and playing only the few seconds either side of a boundary you are unsure about. On a half-hour two-person interview that is typically twenty or thirty edits.

It is also when you catch the names the model got wrong, because you are already moving through the transcript boundary by boundary rather than reading it as prose.

• Use a consistent prefix, such as the name followed by a colon, so the text stays searchable and an assistant reading it later can tell speakers apart
• Label the change, not every line. A reader assumes continuity until told otherwise
• Do it before fixing wording. Boundaries are easier to see while the text is still exactly what was heard

When to use something else

Past about four speakers, or when you need labels across many hours, hand labelling stops being sensible and a tool with real diarisation earns its cost. That means a hosted service, which means uploading the recording, which for some recordings settles the question in the other direction.

If the recording is one you are not permitted to upload, the realistic options are hand labelling a transcript produced locally, or running a diarisation model yourself on your own machine outside the browser. Neither is convenient. Both keep the file where it is.