Agha Ali Raza

Publications/Speech Processing, Recognition & Audio Security

Conference · 2021

Using Self-Attention DNNs to Discover Phonemic Features for Audio Deep Fake Detection

Hira Dhamyal, Ayesha Ali, Ihsan Ayyub Qazi, Agha Ali Raza

ASRU 2021, Cartagena, Colombia

Summary

As synthetic speech becomes more natural, we asked which speech sounds still give fakes away. Synthesizers do not model the physical vocal tract, so we hypothesized that some sounds are reproduced less faithfully than others. We trained a deep network with self-attention on the ASVSpoof2019 dataset and read its attention weights: it relies most on certain fricatives, nasals, a vowel, and a stop consonant. Using only the 16 most-attended phonemes, the detector reaches an 11.98% equal error rate on development data, better than any other class of sounds. The result points to compact, interpretable cues for deepfake detection.

Cite this paper

Hira Dhamyal, Ayesha Ali, Ihsan Ayyub Qazi, Agha Ali Raza. Using Self-Attention DNNs to Discover Phonemic Features for Audio Deep Fake Detection. ASRU 2021, Cartagena, Colombia.

@inproceedings{dhamyal2021self,
  author = {Hira Dhamyal and Ayesha Ali and Ihsan Ayyub Qazi and Agha Ali Raza},
  title = {{Using Self-Attention DNNs to Discover Phonemic Features for Audio Deep Fake Detection}},
  booktitle = {ASRU 2021, Cartagena, Colombia},
  year = {2021},
  url = {https://aghaaliraza.com/publications/2021--ASRU--USING_SELF_ATTENTION_DNNS_TO_DISCOVER_PHONEMIC_FEATURES_FOR_AUDIO_DEEP_FAKE_DETECTION.pdf}
}

More on this theme

All 12 works on this theme →