Synthesis / Representation
verifiedSpeaker Embedding
A fixed-length vector that captures who is speaking rather than what they said. The same vector recognises a speaker across utterances and conditions a synthesiser to sound like them, which is why one representation serves verification and cloning at once — a duality worth noticing before building either.
Trained with a margin-based classification loss over speakers, then used for utterances from speakers never seen in training, which works because the space is organised by voice rather than by identity list. That generalisation is exactly what makes a few seconds of audio enough to clone a voice, so the capability and the misuse come from the same property rather than from a design choice.
Pool frame-level features over time into a single vector, and train so that same-speaker pairs are closer than different-speaker pairs by a margin. Verification thresholds a cosine similarity, and the equal-error rate is where false accepts and false rejects cross — one number that hides which of the two a deployment can tolerate.
enrolment-seconds holds 50% of the budget; rest holds the remaining 50%.
Seconds of enrolment audio against the single utterance being matched, in seconds. Drag the enrolment up to watch the estimate steady — a few seconds is enough, which is the whole security problem in one bar.
Reviewed by opendroid · 2026-08-18
- arXiv:2301.02111 — Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers