How do I identify the same speakers automatically across many recordings?

Identifying the same speakers across many recordings takes two separate steps: diarization, which splits one recording into “who spoke when” with anonymous labels, and speaker identification, which matches those voices to known people and keeps that mapping across files. Most transcription tools stop after the first step and forget everything between requests. Audioscrape does the second step: name a voice once in your workspace and every later upload comes back with that name attached, so “Speaker 2” never has to be worked out again.

Diarization versus identification

Diarization answers “how many people spoke and when did each one talk” for one file. Its labels are local to that file; Speaker 1 in Monday’s call and Speaker 1 in Tuesday’s are unrelated. Identification answers “which of these is Maria” and requires a stored representation of Maria’s voice to compare against. Without it, every recording is a fresh puzzle, and searching “what did Maria say across all our calls” is impossible. With it, the name becomes a filter.

How to do it with Audioscrape

1. Upload recordings to a workspace. Drag files onto your workspace dashboard, or have an agent or script send them to POST /api/datasets/{id}/items (or the transcribe_audio MCP tool). Each recording is transcribed and diarized in the background. Reference: Upload API.

2. Enroll speakers by naming them. Open a finished transcript. Speakers appear as separated voices with generic labels. Assign real names to the people who will recur: team members, a regular client, the council members if you are working with meetings. Bulk rename is available in the transcript editor, so naming one label fixes every segment of that voice in the file. This is the enrollment step; nothing else is needed.

3. Upload the next recording. When it finishes, enrolled speakers are named automatically. Anyone the workspace has not met before still gets a generic label, which you can name when it matters. Over time the set of named voices grows and the manual work shrinks to new people only.

4. Search by speaker. Now speaker: "Maria Chen" pricing works across every recording in the workspace, from the website, from an agent over MCP (search_audio), or from POST /api/search with a speakers filter. Each hit carries the timestamp and a link to the moment.

5. Fix mistakes when they happen. If two people are merged on a noisy call, or one person is split into two, correct the labels in the transcript editor. Corrections are kept with the workspace, so the fix carries forward. If a label was wrong, mark it unknown rather than leaving a wrong name in place.

The public corpus. For podcasts and public meetings, Audioscrape already does this at scale: hosts and recurring guests are named across shows by an evidence-gated classifier, and each person has a page listing their attributed appearances. You do not enroll anyone there; you can report a mis-attribution from any person page.

What you get

• Speaker identities that persist across every recording in the workspace, without re-labeling.
• Named-speaker filters in search and in agent tool calls.
• Per-recording speaker lists in get_episode_overview and get_transcript.
• Speaker-level entity context: which people, companies and topics each person tends to talk about.
• Identities are scoped to the workspace. A voice enrolled in one workspace is not used in another, and nothing about your speakers is used to train models.

Limits and honest caveats

• Audio quality limits everything downstream. Distant microphones, phone lines, heavy compression and crosstalk all degrade both diarization and identification. Two people with very similar voices on a poor recording can be confused.
• The first recording of any speaker is always anonymous. Identification needs an enrolled voice to compare against; there is no way to name someone the system has never heard.
• Diarization is batch, not live. Short recordings finish in minutes; long ones take longer, and a queue can add to that.
• Speaker name attribution in the public corpus is automatic and can be wrong; the report button exists for that reason.
• Voice characteristics used for identification are biometric data in some jurisdictions. Make sure you have the right to record and process the people in your recordings. Upload only audio you own or are permitted to use.
• Free accounts include 30 transcription minutes per month, which is enough to enroll a couple of speakers on short test files, not to run a recording archive.

Doing it without Audioscrape

You can run a diarization tool on each file, then build your own speaker-embedding store, compare new voices against it, and maintain the thresholds and corrections yourself. Teams that have done this describe it as a product in its own right. If speaker identity across files is what you need, use a system that keeps it.

Plans and limits: /pricing. Facts for AI agents: /docs/for-agents.