Investigating voiced and unvoiced regions of speech for audio deepfake detection

Authors: Ganesh Sivaraman, Hemlata Tak, Elie Khoury

Published: 2026-08-25 14:49:55+00:00

Comment: Accepted in IEEE ICASSP 2025

AI Summary

This study investigates the distinct roles of voiced and unvoiced speech regions in detecting audio deepfakes. By analyzing speech into these components and training a graph attention-based detection system independently on each, the research demonstrates that unvoiced regions are significantly more effective at discriminating synthetic from bonafide speech. Fusing the scores from both voiced and unvoiced components further enhances deepfake detection performance.

Abstract

Deep neural network based deepfake detection systems have achieved high levels of accuracy on benchmark datasets and competitions. However, most models lack interpretability. It is challenging to extract reasoning from the network that can convince the human evaluator to trust the decision. Humans often rely on acoustic cues like unnatural pitch jitter, robotic intonation, acoustic artifacts, and unnatural sounding fricatives to judge the quality of the synthetic audio. This study explores the role played by the voiced and unvoiced regions of speech in discriminating synthetic from bonafide speech. A measure of signal periodicity is used to analyze speech into voiced and unvoiced components. Then, the graph attention based AASIST detection system is trained independently on each component. This work compares the accuracy of deepfake detection system using voiced and unvoiced components and analyzes the results on the MLAAD dataset. Our results show that unvoiced regions are particularly more effective in distinguishing synthetic (deepfake) speech from bonafide, and achieves an equal error rate of 6.62%. When combined with voice regions through score-level fusion, the overall performance improves further, yielding a 5.82% EER, a relative improvement of 49% over the baseline system that uses the full audio.

Key findings

Unvoiced regions are particularly more effective in distinguishing synthetic speech, achieving an Equal Error Rate (EER) of 6.62%. When voiced and unvoiced regions are combined through score-level fusion, the overall performance improves to a 5.82% EER, representing a 49% relative improvement over using the full audio. The average spectra of bonafide and synthetic unvoiced segments show a clearer departure in higher frequencies compared to voiced segments, suggesting more discriminative cues in unvoiced regions.

Approach

The authors analyze speech signals into voiced and unvoiced components using probabilistic YIN (pYIN) for segmentation and Web-RTC based speech activity detection (SAD). A graph attention-based AASIST detection system is then trained independently on the full audio, SAD-segmented audio, voiced components, and unvoiced components, and also on a score-level fusion of voiced and unvoiced components.

Datasets

MLAAD (Multi-Lingual Audio Anti-Spoofing Dataset)

Model(s)

AASIST (Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks) with a RawNet2-based encoder.

Author countries

USA

← Previous