Publications/Efficient Machine Learning: Data Selection, Pruning, Active & Federated Learning
Workshop · 2023
Representative Subset Selection for Efficient Fine-Tuning in Self-Supervised Speech Recognition
Abdul Hameed Azeemi, Ihsan Ayyub Qazi, Agha Ali Raza
NeurIPS 2023 Workshop (ENLSP), New Orleans, USA
Summary
Fine-tuning self-supervised speech recognition models requires large amounts of labeled audio. Which subset should be used when resources are limited? Surprisingly, pruning methods designed for images do no better than random selection here. We introduce COWERAGE, which picks a representative subset that covers the full range of difficulty, measured by each example's word error rate early in training. With wav2vec 2.0 and HuBERT on three datasets, it improves word error rate by up to 17% relative to existing pruning methods and random sampling, because covering that range also brings in phonemically diverse examples.
Cite this paper
Abdul Hameed Azeemi, Ihsan Ayyub Qazi, Agha Ali Raza. Representative Subset Selection for Efficient Fine-Tuning in Self-Supervised Speech Recognition. NeurIPS 2023 Workshop (ENLSP), New Orleans, USA.
@inproceedings{azeemi2023representative,
author = {Abdul Hameed Azeemi and Ihsan Ayyub Qazi and Agha Ali Raza},
title = {{Representative Subset Selection for Efficient Fine-Tuning in Self-Supervised Speech Recognition}},
booktitle = {NeurIPS 2023 Workshop (ENLSP), New Orleans, USA},
year = {2023},
url = {https://aghaaliraza.com/publications/2023-NeurIPSWS--Representative_Subset_Selection_for_Efficient_Fine-Tuning_in_Self-Supervised_Speech_Recognition.pdf}
}
- Language Model-Driven Data Pruning Enables Efficient Active LearningFindings of EACL 2026, Rabat, Morocco
- To Label or Not to Label: Hybrid Active Learning for Neural Machine TranslationCOLING 2025, Abu Dhabi, UAE
- A Survey on Data Selection for Efficient Speech ProcessingIEEE Access, vol. 13, 2025
- Data Pruning for Efficient Model Pruning in Neural Machine TranslationFindings of EMNLP 2023, Singapore
- Learning Fast and Slow: Towards Inclusive Federated LearningECML PKDD 2023, Turin, Italy