Could you become more credible by being White? Assessing Impact of Race on Credibility with Deepfakes

View on arXiv ← Back to list

Authors: Kurtis Haut, Caleb Wohn, Victor Antony, Aidan Goldfarb, Melissa Welsh, Dillanie Sumanthiran, Ji-ze Jang, Md. Rafayet Ali, Ehsan Hoque

Published: 2021-02-16 10:05:11+00:00

Comment: 10 pages, 5 figures

AI Summary

This paper investigates how altering perceived race using Deepfakes and GANs influences credibility in computer-mediated conversations. The authors used AI to manipulate facial features and skin tone in still images and videos, while keeping audio constant, and measured the impact on credibility via crowd-sourced surveys. They found that subtle racial manipulation in videos significantly increased perceived credibility, though this effect was not observed with static images.

Abstract

Computer mediated conversations (e.g., videoconferencing) is now the new mainstream media. How would credibility be impacted if one could change their race on the fly in these environments? We propose an approach using Deepfakes and a supporting GAN architecture to isolate visual features and alter racial perception. We then crowd-sourced over 800 survey responses to measure how credibility was influenced by changing the perceived race. We evaluate the effect of showing a still image of a Black person versus a still image of a White person using the same audio clip for each survey. We also test the effect of showing either an original video or an altered video where the appearance of the person in the original video is modified to appear more White. We measure credibility as the percent of participant responses who believed the speaker was telling the truth. We found that changing the race of a person in a static image has negligible impact on credibility. However, the same manipulation of race on a video increases credibility significantly (61\\% to 73\\% with p $<$ 0.05). Furthermore, a VADER sentiment analysis over the free response survey questions reveals that more positive sentiment is used to justify the credibility of a White individual in a video.

Key findings

Changing the perceived race in a static image had a negligible impact on credibility. However, subtly manipulating a person's appearance in a video to appear more White significantly increased their perceived credibility (from 61% to 73%). Furthermore, a VADER sentiment analysis showed that participants used more positive language when justifying the credibility of a perceived White individual in a video.

Approach

The researchers employed a GAN architecture (CycleGAN) to generate distinct racial representations for still images and DeepFaceLab to subtly alter racial appearance in videos. These manipulated visual stimuli, paired with a constant audio clip, were presented to over 800 crowd-sourced participants on Amazon Mechanical Turk. Participants evaluated speaker credibility, and their free-response justifications were analyzed using VADER sentiment analysis.

Datasets

UR Lying dataset, Chicago Face Dataset (CFD)

Model(s)

UNKNOWN

Author countries

United States

← Previous