Speech processing

Voice Activity Detection Benchmark: What to Measure

Set voice-detection onset and offset controls using clipped words, false triggers, and response delay.

A voice assistant needs to hear the first word without waking up for every noise. Pick voice activity detection (VAD) settings by measuring clipped starts, false triggers per hour, and reply delay together. A different microphone or room can change the choice. This is a method, not a BuildAI VAD test.

Choose limits from the interaction

In this example, the team allows at most 2 clipped first words in 100 spoken turns, at most 1 false trigger per hour of ordinary room audio, and a detected speech start within 300 ms for 95% of turns. The first limit reflects how often a user would need to repeat themselves; the second reflects how often an unwanted wake-up is tolerable. These are product choices, not standard VAD targets.

Label audio and test settings

Record held-out sessions on the target microphones with quiet speech, music, keyboard noise, distant voices, interruptions, and silence. Mark where speech begins and ends; agree whether breaths and laughter count. Tune several onset and offset thresholds on development audio, then lock a candidate for the final set. The pyannote VAD pipeline exposes onset and offset controls. Keep the downstream recognizer fixed and count the words it loses after segmentation.

Worked example: an onset sweep

Suppose 100 spoken turns and 10 hours of background audio are used for a first comparison. The table uses hypothetical results. Lowering onset catches more quiet starts but can wake on noise. Offset still needs its own sweep because ending a segment early can cut off the final word.

Onset settingClipped first wordsFalse triggers in 10 hStarts within 300 msDecision
Low0/10028 (2.8/h)99/100Too many wakes
Middle1/1008 (0.8/h)97/100Passes pilot limits
High6/1002 (0.2/h)90/100Clips and delays speech
Illustrative onset sweep. The middle setting passes all three example limits on this small pilot set.

Try the middle setting on new rooms and speakers. One clipped word in 100 and eight triggers in 10 hours are uncertain estimates. Add more hours of ordinary audio and quiet speakers before promising a low trigger rate. If a microphone change raises misses, repeat the setting search. If no setting passes, add a wake-word or push-to-talk path instead of hiding the failure in an average detection score.

Record microphone, room, onset and offset settings, labeled speech intervals, clipped words, false triggers per hour, and start delay. Keep audio-level and recognizer-level results side by side.

Download the results sheet. Fill one row for each test item; keep the item-level errors, not just the average.

Sources

  1. pyannote.audio VAD controls
  2. pyannote metrics reference
← All articlesOur article standard →