TIGP (SNHCC) -- 基於深度偽造影片與圖像之模型歸屬分析技術
- 講者汪新 先生 (中央研究院 資訊科學研究所)
邀請人：TIGP (SNHCC) - 時間2025-05-26 (Mon.) 14:00 ~ 16:00
- 地點資訊所新館106演講廳
摘要
The rapid proliferation of DeepFake videos, created using advanced AI face-swapping techniques, underscores the urgent need for robust forensic tools to attribute these manipulated videos to their originating generative models. Model attribution is critical not only for tracing the source of DeepFake content„ but also for enabling the development of targeted countermeasures and proactive defenses. Unlike traditional real/fake classification approaches, model attribution leverages unique model-specific artifacts, providing deeper insights into the generation processes of DeepFake videos.
This dissertation explores the model attribution problem by formulating it as a multiclass classification task, leveraging diverse datasets, including DFDM, GANGen-Detection, FF++, and FAVCeleb. To address this challenge, we introduce two complementary frameworks designed to capture both spatial and temporal dependencies in DeepFake videos. The first framework, the Capsule-Spatial-Temporal (CapST) model, integrates a modified VGG19 for feature extraction with Capsule Networks and a spatio-temporal attention mechanism. This approach captures intricate feature hierarchies and temporal dependencies, enabling robust model attribution. The second framework, the Attention-Driven Neural Network (ADNN), combines convolutional neural networks (CNNs) with a Long Short-Term Memory (LSTM)-based temporal attention mechanism, achieving superior attribution performance with fewer computational resources.
Extensive experiments on multiple datasets demonstrate the efficacy and generalizability of the proposed approaches. The CapST model achieves substantial improvements in the categorization of DeepFake videos, leveraging video-level fusion to aggregate in- sights across frames for precise predictions. Similarly, the ADNN framework outperforms stateof-the-art methods in accuracy and efficiency, showcasing its robustness in challenging scenarios. Together, these contributions pave the way for more accurate, resource- efficient, and scalable forensic applications that address the evolving threats posed by Deep-Fake technologies.
This dissertation explores the model attribution problem by formulating it as a multiclass classification task, leveraging diverse datasets, including DFDM, GANGen-Detection, FF++, and FAVCeleb. To address this challenge, we introduce two complementary frameworks designed to capture both spatial and temporal dependencies in DeepFake videos. The first framework, the Capsule-Spatial-Temporal (CapST) model, integrates a modified VGG19 for feature extraction with Capsule Networks and a spatio-temporal attention mechanism. This approach captures intricate feature hierarchies and temporal dependencies, enabling robust model attribution. The second framework, the Attention-Driven Neural Network (ADNN), combines convolutional neural networks (CNNs) with a Long Short-Term Memory (LSTM)-based temporal attention mechanism, achieving superior attribution performance with fewer computational resources.
Extensive experiments on multiple datasets demonstrate the efficacy and generalizability of the proposed approaches. The CapST model achieves substantial improvements in the categorization of DeepFake videos, leveraging video-level fusion to aggregate in- sights across frames for precise predictions. Similarly, the ADNN framework outperforms stateof-the-art methods in accuracy and efficiency, showcasing its robustness in challenging scenarios. Together, these contributions pave the way for more accurate, resource- efficient, and scalable forensic applications that address the evolving threats posed by Deep-Fake technologies.