Do Not DeepFake Me: Privacy-Preserving Neural 3D Head Reconstruction Without Sensitive Images

Authors: Jiayi Kong, Xurui Song, Shuo Huai, Baixin Xu, Jun Luo, Ying He

Published: 2023-12-07 07:41:10+00:00

AI Summary

This paper proposes a novel two-stage 3D facial reconstruction method that protects privacy by avoiding the use of sensitive facial images. It achieves detailed geometric accuracy comparable to methods using full images, while making the reconstructed geometry resistant to DeepFake applications and facial recognition systems. The approach utilizes non-sensitive rear-head images for initial geometry and refines it with processed privacy-removed gradient images.

Abstract

While 3D head reconstruction is widely used for modeling, existing neural reconstruction approaches rely on high-resolution multi-view images, posing notable privacy issues. Individuals are particularly sensitive to facial features, and facial image leakage can lead to many malicious activities, such as unauthorized tracking and deepfake. In contrast, geometric data is less susceptible to misuse due to its complex processing requirements, and absence of facial texture features. In this paper, we propose a novel two-stage 3D facial reconstruction method aimed at avoiding exposure to sensitive facial information while preserving detailed geometric accuracy. Our approach first uses non-sensitive rear-head images for initial geometry and then refines this geometry using processed privacy-removed gradient images. Extensive experiments show that the resulting geometry is comparable to methods using full images, while the process is resistant to DeepFake applications and facial recognition (FR) systems, thereby proving its effectiveness in privacy protection.

Key findings

The proposed method successfully reconstructs detailed 3D head geometry with accuracy comparable to state-of-the-art techniques using full images, as evidenced by Chamfer distance metrics. Crucially, the resulting geometry and the reconstruction process are demonstrated to be highly resistant to 2D facial recognition systems and DeepFake applications, significantly enhancing privacy protection.

Approach

The method employs a two-stage neural reconstruction process. The first stage uses privacy-neutral images (e.g., rear-head images) to establish a basic geometric structure. The second stage refines this geometry using privacy-protected images, which are processed facial images converted into gradient information, without relying on full RGB data.

Datasets

FaceScape, High-Fidelity 3D Head (H3DS)

Model(s)

UNKNOWN

Author countries

Singapore

← Previous