Publications/Natural Language Processing & LLMs for Low-Resource Languages
Workshop · 2024
Challenges in Urdu Machine Translation
Abdul Basit, Abdul Hameed Azeemi, Agha Ali Raza
LoResMT Workshop at ACL 2024, Bangkok, Thailand
Summary
Machine translation has improved dramatically, but still struggles with low-resource languages such as Urdu. We evaluate English-to-Urdu translation with four very different systems: a general large language model, a bilingual translation model, a model covering 200 languages, and IndicTrans2, a specialist for Indic languages. The specialist significantly outperforms the rest. By comparing training data and methods, we explain the differences, identify the specific translation errors that remain, and suggest how Urdu translation systems can be improved.
Cite this paper
Abdul Basit, Abdul Hameed Azeemi, Agha Ali Raza. Challenges in Urdu Machine Translation. LoResMT Workshop at ACL 2024, Bangkok, Thailand.
@inproceedings{basit2024challenges,
author = {Abdul Basit and Abdul Hameed Azeemi and Agha Ali Raza},
title = {{Challenges in Urdu Machine Translation}},
booktitle = {LoResMT Workshop at ACL 2024, Bangkok, Thailand},
year = {2024},
url = {https://aghaaliraza.com/publications/2024--ACL--Challenges_in_Urdu_Machine_Translation.pdf}
}
- IRTIQA: Evolving in English, Evaluated in Urdu — Does Agent Self-Evolution Transfer Across Languages?Findings of EACL 2027, Athens, Greece
- UrduCultureQA: A Culturally Grounded Benchmark for Urdu Question Answering and Alignment EvaluationFindings of EACL 2027, Athens, Greece
- Pak3H: Evaluating the Cost of Cultural Mismatch in LLM Alignment with a Human-Contextualized Urdu BenchmarkEMNLP 2026 (Main), Budapest, Hungary
- Evaluating Robustness of LLM-Based Ambient Scribes for SOAP Note GenerationCHIL 2026 (Conference on Health, Inference, and Learning), PMLR vol. 333
- Kahaani: A Multimodal Co-Creative Storytelling SystemEACL 2026 Student Research Workshop, Rabat, Morocco