REALEDIT: Reddit Edits As a Large-scale Empirical Dataset for Image Transformations

View on arXiv ← Back to list

Authors: Peter Sushko, Ayana Bharadwaj, Zhi Yang Lim, Vasily Ilin, Ben Caffee, Dongping Chen, Mohammadreza Salehi, Cheng-Yu Hsieh, Ranjay Krishna

Published: 2025-02-05 21:35:48+00:00

Comment: Published at CVPR 2025

AI Summary

This paper introduces REALEDIT, a large-scale image editing dataset sourced from Reddit, featuring authentic user requests and human-made edits. It addresses the limitations of existing datasets, which often use artificial edits, by providing ecologically valid data for training more effective image editing models. By finetuning a model on REALEDIT, the authors achieve substantial performance gains in image editing and demonstrate its utility in improving edited image detection.

Abstract

Existing image editing models struggle to meet real-world demands. Despite excelling in academic benchmarks, they have yet to be widely adopted for real user needs. Datasets that power these models use artificial edits, lacking the scale and ecological validity necessary to address the true diversity of user requests. We introduce REALEDIT, a large-scale image editing dataset with authentic user requests and human-made edits sourced from Reddit. REALEDIT includes a test set of 9300 examples to evaluate models on real user requests. Our results show that existing models fall short on these tasks, highlighting the need for realistic training data. To address this, we introduce 48K training examples and train our REALEDIT model, achieving substantial gains - outperforming competitors by up to 165 Elo points in human judgment and 92 percent relative improvement on the automated VIEScore metric. We deploy our model on Reddit, testing it on new requests, and receive positive feedback. Beyond image editing, we explore REALEDIT's potential in detecting edited images by partnering with a deepfake detection non-profit. Finetuning their model on REALEDIT data improves its F1-score by 14 percentage points, underscoring the dataset's value for broad applications.

Key findings

Their REALEDIT model significantly outperforms baselines on real-world image editing tasks, achieving up to 165 Elo points higher in human judgment and 92% relative improvement on the VIEScore metric. Finetuning an edited image detection model (UFD) on REALEDIT data improved its F1-score by 14 percentage points on in-the-wild images. The dataset analysis reveals that real user requests differ substantially from synthetic datasets, focusing more on subtle 'remove' and 'enhance' operations, particularly involving people and media.

Approach

The authors introduce REALEDIT, a dataset of 57K image editing examples sourced from Reddit with real user requests and human-made edits. They finetune an InstructPix2Pix model, replacing its Stable Diffusion decoder with OpenAI’s Consistency Decoder, on this dataset to create a new image editing model. Additionally, they demonstrate the dataset's utility for detecting edited images by finetuning an existing Universal Fake Detect (UFD) model on REALEDIT data.

Datasets

REALEDIT, InstructPix2Pix, MagicBrush, Emu Edit, AURORA, SDEdit, HIVE, Null-text Inversion, GenAI Arena, TrueMedia.org (in-the-wild images). For deepfake training baseline: DiffusionDB, StyleGAN2-FFHQ, Stable-Diffusion-Face, CelebA-HQ, COCO-Train-2017, Flickr-Faces-HQ (FFHQ).

Model(s)

InstructPix2Pix (backbone), OpenAI's Consistency Decoder (for their model), Universal Fake Detect (UFD) (for edited image detection), GPT-4o (for instruction refinement and captioning), LLaVA-Next (for captioning). Baselines: AURORA, HIVE, InstructPix2Pix, MagicBrush, Null-text Inversion, SDEdit, TurboEdit, Edit Friendly.

Author countries

United States

← Previous