The UM team’s solution framework
A team from the Faculty of Science and Technology (FST) of the University of Macau (UM) won the championship and a prize of RMB100,000 at the Global Multimedia Deepfake Detection Challenge held during the 2024 Inclusion Conference on the Bund, after competing with nearly 10,000 participants from over 7,000 teams from more than 20 countries and regions. These teams came from renowned institutions and companies, including the Massachusetts Institute of Technology, Stanford University, Peking University, the National University of Singapore, Microsoft, Google, ByteDance, the Institute of Automation of the Chinese Academy of Sciences, Microsoft Research Asia, and the Agency for Science, Technology and Research in Singapore.
Deepfake technology utilises advanced deep learning algorithms to create highly realistic facial animations and voice synthesis effects by learning from large amounts of video and image data, also known as ‘AI face-swapping’ technology. This technology not only makes it more difficult for the public to determine the authenticity of videos but also increases the risks of fraud, pornography, and other illegal activities. Given the growing threat of ‘AI face-swapping’ fraud, it is vital to cultivate professionals in this area. The competition focused on practical exercises in ‘AI face-swapping’, which helped foster industry-academia collaboration, talent development in artificial intelligence, and the development of solutions to effectively counter deepfake attacks in various scenarios, thereby safeguarding industries and society.
The winning team consisted of FST students Chen Yiming, Li Fengpeng, Li Kemou, Wong Ka Him, Li Zheng, Chen Xiangyu, Song Binbin, Xu Shuning, and Liu Jun; and UM alumnus Wu Haiwei. They trained for the competition under the guidance of Zhou Jiantao, head of the Department of Computer and Information Science. The team’s winning solution not only introduced a data type-aware unsupervised clustering method to reclassify datasets for simulating cross-domain testing scenarios, but also developed feature-level adversarial training algorithms and type augmentation algorithms. These innovations enhanced data types and guided detection models to learn high-generalisation deepfake features adaptable to multiple types and scenarios. This technology helps identify deepfake content, effectively prevent fraud caused by the misuse of deepfake technology, and protect industries and society from its negative impact. The related research findings are available on: https://github.com/HighwayWu/DeepFakeDefenders.
The competition was organised by Ant Group and presented by Ant Group Digital Technologies, with academic collaboration from institutions including the School of Cyber Science and Technology at the University of Science and Technology of China, and the China Society of Image and Graphics. The competition was hosted on the renowned data science competition platform Kaggle.
