Speech processing

How to Set an Audio Alert Threshold

Turn sound-classifier errors into missed events and false alerts per hour, then choose an alert threshold.

If a sound classifier triggers an alert, its useful output is an acceptable number of missed events and false alerts per hour. Choose the threshold that meets both limits in the environment where it will run. This guide is a method, not a BuildAI test.

Start with the alert cost

Ask the operations team how many alerts people can investigate and how many true events they can miss. In this example they can handle at most two false alerts per hour across one monitored stream and require at least 90% of real alarms to be caught. Those limits come from staffing and safety review; your limits may differ. Count the ordinary sound windows in production. Here there are 600 non-alarm windows per hour. The false-alert estimate is false-positive rate × 600. A 1% false-positive rate therefore means about six false alerts each hour.

Build a test that matches the stream

Label alarm and non-alarm windows from the real microphones and rooms. Include quiet alarms, similar sounding noises, clipping, and overlapping events. Split recordings by session so nearby windows from one event cannot appear on both sides of the split. Keep a final set aside until thresholds are chosen. Track counts by location and condition. Precision, recall, and the confusion matrix have standard definitions in scikit-learn's classification guide; for this alert, the hourly burden is easier for the team to act on than macro F1.

Worked example: sweep the threshold

Suppose the tuning set contains 100 alarm windows and 1,000 ordinary windows. The figures below are illustrative. “Missed” means alarms below the threshold; the hourly estimate uses 600 ordinary windows, so it assumes this test set resembles production.

ThresholdAlarms caughtOrdinary windows flaggedEstimated false alerts/hourDecision
0.3096/10020/1,00012Too many alerts
0.5592/1003/1,0001.8Passes both limits
0.7583/1001/1,0000.6Misses too many alarms
Illustrative threshold sweep. The 0.55 setting meets at least 90% alarm recall and at most two false alerts per hour.

Choose 0.55 for a new held-out test. A quieter site could see fewer false alerts; a busier one could see more. If the cost of a miss rises, the team may need a second human review path rather than simply lowering the threshold and flooding operators.

Check that the result is stable

Three false positives in 1,000 ordinary windows is a thin estimate of a low error rate. Collect many more normal hours and show the range across sites and days before promising an alert burden. Review every miss, especially quiet alarms. Record the threshold, test hours, event counts, device, environment, alerts per hour, and response time. Do not change the threshold on the final set and then report that set as an independent result.

Download the results sheet. Fill one row for each test item; keep the item-level errors, not just the average.

Sources

  1. scikit-learn: classification metrics
  2. scikit-learn: confusion matrix
← All articlesOur article standard →