Wallcamera: Reinventing the Wheel?

Authors: Aurélien Bourquard, Jeff Yan

Published: 2024-07-22 19:46:27+00:00

AI Summary

This paper argues that the 'Wallcamera' research from MIT CSAIL, which extracts and amplifies invisible signals from wall reflections in video for activity recognition, is based on the same key insight as their previously published concept of Differential Imaging Forensics (DIF). The authors contend that DIF predates Wallcamera and has broader applications beyond activity recognition, including personal identifiable information recovery and deepfake detection.

Abstract

Developed at MIT CSAIL, the Wallcamera has captivated the public's imagination. Here, we show that the key insight underlying the Wallcamera is the same one that underpins the concept and the prototype of differential imaging forensics (DIF), both of which were validated and reported several years prior to the Wallcamera's debut. Rather than being the first to extract and amplify invisible signals -- aka latent evidence in the forensics context -- from wall reflections in a video, or the first to propose activity recognition following that approach, the Wallcamera's actual innovation is achieving activity recognition at a finer granularity than DIF demonstrated. In addition to activity recognition, DIF as conceived has a number of other applications in forensics, including 1) the recovery of a photographer's personal identifiable information such as body width, height, and even the color of their clothing, from a single photo, and 2) the detection of image tampering and deepfake videos.

Key findings

The paper demonstrates that DIF can extract and amplify invisible signals from wall reflections in video, enabling activity recognition and the recovery of specific information on subjects, such as body height, width, and clothing color. While Wallcamera achieved finer granularity in activity recognition using CNNs, DIF proved compelling results with simple signal processing and offers broader forensic applications like deepfake detection.

Approach

Differential Imaging Forensics (DIF) involves acquiring a reference baseline image or video (without the object of interest), then performing a comparative analysis by computationally extracting and amplifying the differences between an image/video of interest and the corresponding reference. This makes faint or invisible visual evidence perceptible.

Datasets

UNKNOWN

Model(s)

No deep learning models were used in the core DIF approach described; simple signal processing techniques like temporal averaging, subtraction, spatial filtering (e.g., 2D Gaussian filter), and contrast normalization are employed.

Author countries

USA, UK

← Previous