Publications/Speech Processing, Recognition & Audio Security
Conference · 2021
Using Self-Attention DNNs to Discover Phonemic Features for Audio Deep Fake Detection
Hira Dhamyal, Ayesha Ali, Ihsan Ayyub Qazi, Agha Ali Raza
ASRU 2021, Cartagena, Colombia
Summary
As synthetic speech becomes more natural, we asked which speech sounds still give fakes away. Synthesizers do not model the physical vocal tract, so we hypothesized that some sounds are reproduced less faithfully than others. We trained a deep network with self-attention on the ASVSpoof2019 dataset and read its attention weights: it relies most on certain fricatives, nasals, a vowel, and a stop consonant. Using only the 16 most-attended phonemes, the detector reaches an 11.98% equal error rate on development data, better than any other class of sounds. The result points to compact, interpretable cues for deepfake detection.
Cite this paper
Hira Dhamyal, Ayesha Ali, Ihsan Ayyub Qazi, Agha Ali Raza. Using Self-Attention DNNs to Discover Phonemic Features for Audio Deep Fake Detection. ASRU 2021, Cartagena, Colombia.
@inproceedings{dhamyal2021self,
author = {Hira Dhamyal and Ayesha Ali and Ihsan Ayyub Qazi and Agha Ali Raza},
title = {{Using Self-Attention DNNs to Discover Phonemic Features for Audio Deep Fake Detection}},
booktitle = {ASRU 2021, Cartagena, Colombia},
year = {2021},
url = {https://aghaaliraza.com/publications/2021--ASRU--USING_SELF_ATTENTION_DNNS_TO_DISCOVER_PHONEMIC_FEATURES_FOR_AUDIO_DEEP_FAKE_DETECTION.pdf}
}
- Dissecting ASR Failures in Low-Resource South Asian LanguagesInterspeech 2026, Sydney, Australia
- WER We Stand: Benchmarking Urdu ASR ModelsCOLING 2025, Abu Dhabi, UAE
- Conversations in the Wild: Data Collection, Automatic Generation and EvaluationComputer Speech & Language, vol. 89, 2025
- Deepfake Defense: Constructing and Evaluating a Specialized Urdu Deepfake Audio DatasetFindings of ACL 2024, Bangkok, Thailand
- Speech Emotion Recognition for the Urdu LanguageLanguage Resources and Evaluation (LRE), 2023