Deep-Fake CAPTCHA: Mitigating Next-Generation Social Engineering Attacks

Authors: Guy Frankovits, Lior Yasur, Fred M. Grabovski, Yisroel Mirsky

Published: 2026-09-10 11:35:11+00:00

Comment: Expanded work from the original ASIA CCS paper on DF-CAPTCHA (now evaluates video deepfakes too)

AI Summary

This paper introduces DF-CAPTCHA, an active defense mechanism against real-time deepfake impersonation in voice and video calls. Unlike passive detection methods, DF-CAPTCHA challenges callers with tasks difficult for deepfake systems but easy for humans, verifying responses based on realism, identity consistency, task completion, and response time. The framework significantly improves deepfake detection across both audio and video modalities compared to passive methods, offering a robust defense against social engineering attacks.

Abstract

This paper presents DF-CAPTCHA, an active defense against real-time deepfake impersonation in voice and video calls. Instead of passively searching for artifacts, DF-CAPTCHA prompts the caller to perform simple challenge-response tasks that are easy for humans but difficult for current real-time deepfake systems to generate convincingly. The framework verifies the response using four criteria: realism, identity consistency, task completion, and response time. We evaluate the approach across both audio and video modalities using user studies and experiments with real-time deepfake models. Results show that people often struggle to distinguish real-time deepfakes from authentic media, while DF-CAPTCHA substantially improves detection performance over passive methods, reaching high accuracy in both modalities. These findings suggest that active challenge-based verification is a practical and robust defense against next-generation social engineering attacks based on real-time deepfakes.

Key findings

User studies reveal that humans often struggle to distinguish real-time deepfakes from authentic media, both in audio and video. DF-CAPTCHA substantially improves detection performance over passive methods, achieving high accuracy (91-100% in audio, 87-100% in video) by forcing deepfake systems to generate artifacts during challenges. This active defense is shown to be a practical and robust solution against next-generation real-time deepfake attacks.

Approach

DF-CAPTCHA employs a challenge-response mechanism where callers are prompted to perform specific tasks. The system then analyzes the caller's response across four criteria: realism (identifying artifacts), identity consistency (matching pre-challenge identity), task completion (ensuring the task was performed), and response time. This active approach forces deepfake models outside their optimal operating range, making their generated content more easily detectable.

Datasets

Custom-recorded datasets of 20 English-speaking volunteers for both speech and challenge responses (Dreal, Dfake, Dreal,r, Dfake,r) for both audio and video. Additionally, ASVspoof-DF and RITW datasets were used for training audio realism models.

Model(s)

For audio realism: SpecRNet, One-Class (ResNet-18 backbone), GMM-ASVspoof, PC-DARTS, Local Outlier Factor (LOF). For video realism: RECCE, FFD, SRM, SPSL, Xception, Capsule-Forensics, UCF, F3Net, CORE. For audio task verification: GMM classifier. For video task verification: ResNet-3D. For audio identity verification: ECAPA-TDNN. For video identity verification: ResNet-18.

Author countries

Israel

← Previous