Diffusion Deepfake

1Yeungnam University, South Korea 2University of Surrey, UK 3Nanjing University of Posts and Telecommunications, China Equal Contribution*

Abstract

Recent progress in generative AI, primarily through diffusion models, presents significant challenges for real-world deepfake detection. The increased realism in image details, diverse content, and widespread accessibility to the general public complicates the identification of these sophisticated deepfakes. Acknowledging the urgency to address the vulnerability of current deepfake detectors to this evolving threat, our paper introduces two extensive deepfake datasets generated by state-of-the-art diffusion models. Our comprehensive evaluation reveals the struggle of existing detection methods, often optimized for specific image domains and manipulations, to effectively adapt to the intricate nature of diffusion deepfakes, limiting their practical utility. To address this critical issue, we investigate the impact of enhancing training data diversity on representative detection methods. This involves expanding the diversity of both manipulation techniques and image domains. Our findings underscore that increasing training data diversity results in improved generalizability. Moreover, we propose a novel momentum difficulty boosting strategy to tackle the additional challenge posed by training data heterogeneity. This strategy dynamically assigns appropriate sample weights based on learning difficulty, enhancing the model's adaptability to both easy and challenging samples. Extensive experiments on both existing and newly proposed benchmarks demonstrate that our model optimization approach surpasses prior alternatives significantly. Code and datasets will be released.

Collection Process Image

Collection process for the proposed DiffusionDB-Face and JourneyDB-Face datasets. Green Border: Images that were kept for the following round; Red Border: Images that were deleted after filtering.

Your Alt Text

Dataset Preview

Train Strategy FF++ CelebDFv2 UADFV Avgconv Deepfakeface DFDB-Face JDB-Face Avgdiff Avgall
FF++ 0.89 0.66 0.50 0.68 0.35 0.67 0.48 0.50 0.59
CelebDFv2 0.50 0.59 0.40 0.49 0.25 0.49 0.23 0.32 0.41
UADFV 0.50 0.33 0.49 0.44 0.42 0.26 0.49 0.39 0.42
Deepfakeface 0.47 0.23 0.37 0.35 0.68 0.23 0.51 0.47 0.42
DFDB-Face 0.72 0.71 0.47 0.63 0.73 0.73 0.57 0.68 0.65
JDB-Face 0.42 0.47 0.29 0.39 0.25 0.47 0.67 0.46 0.35
Multi-domain 0.85 0.75 0.50 0.70 0.39 0.43 0.51 0.39 0.43
+KD 0.84 0.81 0.50 0.71 0.72 0.76 0.57 0.68 0.70
+DW w/o mom. 0.78 0.53 0.50 0.60 0.57 0.63 0.58 0.59 0.59
+MDB (ours) 0.95 0.82 0.49 0.75 0.79 0.98 0.98 0.91 0.84
Comparison of conventional single-domain and our multi-domain training. Avgconv, Avgdiff, Avgall : Average accuracy on the conventional/diffusion/all datasets.

BibTeX

cite