Publications/Natural Language Processing & LLMs for Low-Resource Languages
Journal · 2026
Large Language Models Show Dunning-Kruger-Like Effects in Multilingual Fact-Checking
Ihsan Ayyub Qazi, Zohaib Khan, Abdullah Ghani, Agha Ali Raza, Zafar Ayyub Qazi, Wassay Sajjad, Ayesha Ali, Asher Javaid, Muhammad Abdullah Sohail, Abdul H. Azeemi
Scientific Reports 16, 7594 (2026), a Nature Portfolio journal · IF 3.9
Summary
Can AI check facts across the world's languages? We evaluated nine language models on 5,000 claims assessed by 174 professional fact-checking organizations in 47 languages. Smaller, more accessible models were highly confident despite lower accuracy, while larger models were more accurate but less confident, a pattern resembling the Dunning-Kruger effect. Performance gaps were largest for non-English languages and for claims from the Global South. Because organizations with fewer resources often rely on smaller models, this could widen existing information inequalities. The study provides a multilingual benchmark and evidence for policies on equitable AI-assisted fact-checking.
Cite this paper
Ihsan Ayyub Qazi, Zohaib Khan, Abdullah Ghani, Agha Ali Raza, Zafar Ayyub Qazi, Wassay Sajjad, Ayesha Ali, Asher Javaid, Muhammad Abdullah Sohail, Abdul H. Azeemi. Large Language Models Show Dunning-Kruger-Like Effects in Multilingual Fact-Checking. Scientific Reports 16, 7594 (2026), a Nature Portfolio journal.
@article{qazi2026large,
author = {Ihsan Ayyub Qazi and Zohaib Khan and Abdullah Ghani and Agha Ali Raza and Zafar Ayyub Qazi and Wassay Sajjad and Ayesha Ali and Asher Javaid and Muhammad Abdullah Sohail and Abdul H. Azeemi},
title = {{Large Language Models Show Dunning-Kruger-Like Effects in Multilingual Fact-Checking}},
journal = {Scientific Reports},
volume = {16},
pages = {7594},
year = {2026},
url = {https://doi.org/10.1038/s41598-026-39046-w}
}
- IRTIQA: Evolving in English, Evaluated in Urdu — Does Agent Self-Evolution Transfer Across Languages?Findings of EACL 2027, Athens, Greece
- UrduCultureQA: A Culturally Grounded Benchmark for Urdu Question Answering and Alignment EvaluationFindings of EACL 2027, Athens, Greece
- Pak3H: Evaluating the Cost of Cultural Mismatch in LLM Alignment with a Human-Contextualized Urdu BenchmarkEMNLP 2026 (Main), Budapest, Hungary
- Evaluating Robustness of LLM-Based Ambient Scribes for SOAP Note GenerationCHIL 2026 (Conference on Health, Inference, and Learning), PMLR vol. 333
- Kahaani: A Multimodal Co-Creative Storytelling SystemEACL 2026 Student Research Workshop, Rabat, Morocco