Publications/Efficient Machine Learning: Data Selection, Pruning, Active & Federated Learning
Conference · 2023
Data Pruning for Efficient Model Pruning in Neural Machine Translation
Abdul Hameed Azeemi, Ihsan Ayyub Qazi, Agha Ali Raza
Findings of EMNLP 2023, Singapore · CORE A*
Summary
Large translation models can be compressed by pruning away parameters, but the pruning process itself is computationally heavy, requiring repeated rounds of training. We show that it can be made cheaper by also pruning the training data. Selecting the hardest-to-learn examples, identified by their training loss, works better than other data selection methods when combined with movement pruning on Romanian-English and Turkish-English translation. Training on this subset reduces the steps needed to converge and the overall training time, and our experiments shed light on how data pruning and model pruning interact.
Cite this paper
Abdul Hameed Azeemi, Ihsan Ayyub Qazi, Agha Ali Raza. Data Pruning for Efficient Model Pruning in Neural Machine Translation. Findings of EMNLP 2023, Singapore.
@inproceedings{azeemi2023data,
author = {Abdul Hameed Azeemi and Ihsan Ayyub Qazi and Agha Ali Raza},
title = {{Data Pruning for Efficient Model Pruning in Neural Machine Translation}},
booktitle = {Findings of EMNLP 2023, Singapore},
year = {2023},
url = {https://aghaaliraza.com/publications/2024--EMNLP--Data_Pruning_for_Efficient_Model_Pruning_in_Neural_Machine_Translation.pdf}
}
- Language Model-Driven Data Pruning Enables Efficient Active LearningFindings of EACL 2026, Rabat, Morocco
- To Label or Not to Label: Hybrid Active Learning for Neural Machine TranslationCOLING 2025, Abu Dhabi, UAE
- A Survey on Data Selection for Efficient Speech ProcessingIEEE Access, vol. 13, 2025
- Representative Subset Selection for Efficient Fine-Tuning in Self-Supervised Speech RecognitionNeurIPS 2023 Workshop (ENLSP), New Orleans, USA
- Learning Fast and Slow: Towards Inclusive Federated LearningECML PKDD 2023, Turin, Italy