Publications/Natural Language Processing & LLMs for Low-Resource Languages
Conference · 2018
Urdu Word Segmentation using Conditional Random Fields
Haris Bin Zia, Agha Ali Raza, Awais Athar
COLING 2018, Santa Fe, NM, USA · CORE A
Summary
Before a computer can process text, it must know where words begin and end. In Urdu this is surprisingly hard: spaces are often omitted between words and inserted inside them, a consequence of how letters in the Arabic script join. We present a word segmentation system based on conditional random fields with orthographic, linguistic, and morphological features. It learns to predict both word boundaries and sub-word boundaries, achieving F1 scores of 0.97 and 0.85 respectively on a manually annotated corpus. Code and corpus are publicly available.
Cite this paper
Haris Bin Zia, Agha Ali Raza, Awais Athar. Urdu Word Segmentation using Conditional Random Fields. COLING 2018, Santa Fe, NM, USA.
@inproceedings{zia2018urdu,
author = {Haris Bin Zia and Agha Ali Raza and Awais Athar},
title = {{Urdu Word Segmentation using Conditional Random Fields}},
booktitle = {COLING 2018, Santa Fe, NM, USA},
year = {2018},
url = {https://aghaaliraza.com/publications/2018--COLING--Urdu_Word_Segmentation_using_Conditional_Random_Fields.pdf}
}
- IRTIQA: Evolving in English, Evaluated in Urdu — Does Agent Self-Evolution Transfer Across Languages?Findings of EACL 2027, Athens, Greece
- UrduCultureQA: A Culturally Grounded Benchmark for Urdu Question Answering and Alignment EvaluationFindings of EACL 2027, Athens, Greece
- Pak3H: Evaluating the Cost of Cultural Mismatch in LLM Alignment with a Human-Contextualized Urdu BenchmarkEMNLP 2026 (Main), Budapest, Hungary
- Evaluating Robustness of LLM-Based Ambient Scribes for SOAP Note GenerationCHIL 2026 (Conference on Health, Inference, and Learning), PMLR vol. 333
- Kahaani: A Multimodal Co-Creative Storytelling SystemEACL 2026 Student Research Workshop, Rabat, Morocco