I Hear, Therefore I Trust: A Socio-Technical Investigation of Humans as Synthetic Speech Detectors

Authors: Lelia Erscoi, Tomi Kinnunen

Published: 2026-05-27 07:16:02+00:00

Comment: To be included in Odyssey 2026: The Speaker and Language Recognition Workshop, Session 4.2, 23-26 June, Lisbon, Portugal

AI Summary

This paper investigates human detection of synthetic speech as a socio-technical process, employing a localization task with 47 participants. They evaluated authentic, fully synthetic, and partially synthetic utterances under varying trust cues like instructional framing, affective priming, and provenance labeling. The study found that utterance authenticity was the primary factor in detection accuracy and perceptual quality, while external trust cues had no significant main effect on detection.

Abstract

Automatic deepfake detection has received considerable research attention, yet the socio-technical environment in which humans actually encounter synthetic speech remains poorly understood. We investigate voice deepfake detection as a perceptual and contextual process, presenting a localization task in which 47 participants marked suspected synthetic segments across authentic, fully synthetic, and partially synthetic utterances under three manipulated trust cues: instructional framing, affective priming, and provenance labeling. Participants provided quality ratings on mechanicalness, expressiveness, intelligibility, clarity, calmness, and confidence of evaluation. Utterance class was the primary determinant of detection accuracy and perceptual quality; trust cues produced no main effects but motivated detection behavior. Fully synthetic speech was detected at below-chance levels. Quality ratings tracked utterance type, indicating implicit discrimination where overt detection failed.

Key findings

Fully synthetic speech was detected at below-chance levels, and participants were systematically overconfident in their evaluations. While utterance authenticity strongly dictated detection accuracy and perceived quality, contextual trust cues did not significantly improve human detection performance. This suggests that humans are unreliable deepfake detectors and cannot be the sole gatekeepers against synthetic speech.

Approach

The researchers designed a human-centered experiment where 47 participants performed a localization task, marking suspected synthetic segments in audio utterances. They manipulated three contextual trust cues (instructional framing, affective priming, provenance labeling) and collected perceptual quality ratings and confidence levels from participants.

Datasets

LlamaPartialSpoof, LibriTTS (as source for speakers), International Soundscape Database, Open Affective Standardized Image Set (OASIS)

Model(s)

UNKNOWN

Author countries

Finland

← Previous