My research interests span the areas of Speech and Natural Language Processing, Speech-based Human-Computer Interaction, and Information & Communication Technologies for Development (ICT4D). My research can be broadly categorized into four questions:

  1. How can we rapidly disseminate useful development-related information to under-served populations interactively, organically, equitably and at scale?
  2. How can we automatically quantify and measure changes in beliefs and behavior of the target populations that result from large-scale information dissemination campaigns?
  3. How can we enable social connectivity and community-support for under-served populations?
  4. What are the core localization and resource-availability challenges that impede Speech & Natural Language Processing for local languages of the developing world and how can we tackle these challenges?

These questions form a nexus around the central theme of my research: “Use of technology to alleviate inequitable access to information and connectivity in the world”.

Next, I contextualize and explain the above-mentioned directions of my research.

1. Viral Spread of Development-related Information to Under-served Populations

The Internet is a gateway to information and connectivity mostly for literate, affluent, and tech-enabled people. It is their key to awareness and engagement, social connectivity and communal support, e-government services, education, job opportunities, health care, news, citizen journalism, and entertainment. However, with 4.4 billion offline individuals, more than half of the world remains deprived of this facility [1]. My aim is to enable equitable information access and social connectivity for under-connected and under-served populations throughout the developing world, including low-literate, low-income, tech naïve, offline, visually impaired, linguistically and socially marginalized populations, geographically remote communities, and oral cultures [2].

Speech-based services available over simple and feature phones have been shown to overcome the primary barriers of access and inclusion for such populations. However, nearly a decade of research has revealed several secondary barriers to connectivity and information access including user-training, motivation and spread. Target populations need to be trained to use even the simplest of speech-based telephone services and explicit training is not a scalable solution. Users are not motivated enough to replace their traditional though often unreliable information channels with modern technology. Finally, there is no easy way to advertise these services to populations that are under-connected by definition. My work has shown that grounding speech-based, telephone services in simple forms of entertainment and person-to-person asynchronous messaging creates very powerful motivators that induce engagement, self-training, trust and viral spread of these services among primarily under-served populations. Such entertainment services can then be used as viral conduits for spreading core development-related information services (health, job opportunities, public service announcements, emergency disaster response and recovery). This creates a pathway to reach these populations rapidly and effectively.

To achieve the above, I created a telephone-based speech service called Polly that allows users to record their voice, modify it using funny voice modifications and send the original or modified recording to their friends. This extremely simple form of entertainment greatly appealed to our target users. Seeded with 5 initial users, within a year and without any explicit advertisement, Polly virally reached 165,000 users who engaged in 636,000 calls. At its peak, Polly was spreading to 1,000 new users every day [3, 4]. Polly was used primarily by low-educated young men from all over Pakistan for entertainment and other creative uses like voicemail, group messaging and telemarketing. Its viral spread crossed gender and age boundaries, attracting a lot of blind users but remaining primarily in the same socio-economic strata. As part of its first deployment, I used Polly as a viral vehicle to connect users to entry-level and low-skilled job opportunities. An audio promotion of the job ads was placed on Polly and 34,000 users of Polly started using the job ads service. The 728 job ads available in the system were played more than 386,000 times [4]. This novel approach created a paradigm shift in the domain of information services for under-served populations. My novel work on Polly [4] was appreciated by the scientific community and media alike. In 2013, my Polly paper received the Best Paper Award at the prestigious ACM CHI conference. This was followed by media coverage by renowned outlets including the Morning Edition on NPR [5], the Link [6] and Tartan at CMU [7], MIT Tech Review International [8], Express Tribune [9], The Times of India [10], IEEE Signal Processing Society [11], Science Newsline [12], Pittsburgh Post-Gazette [13], Yahoo Finance, India [14], among others. My doctoral adviser Roni Rosenfeld and I also received a grant worth 1 Million Euros from GIZ to use Polly to further spread job and skill training opportunities among hard-to-reach populations in Pakistan.

Motivated by the success of Polly in Pakistan, I worked on the idea of using it as a vehicle to spread useful information in other developing countries. In India, I teamed up with Babajob.com, a commercial job portal, to launch Polly in Bangalore [15] and virally spread information about entry-level and informal jobs. I also used Polly to spread the word about a citizen-radio-over-phone platform, Jharkhand Mobile Vaani (JMV) in India. My first attempt at using Polly to disseminate emergency disaster response information took place in 2014 when Ebola broke out in several West African countries. It threatened to engulf a larger population if effective measures were not taken to inform the public and prevent its spread. I launched Polly in Guinea in collaboration with and funded by the US embassy in Conakry to spread reliable information about Ebola [16]. This variant of Polly, called Polly-Sante’ (Polly-Health), spread audio health messages about prevention, symptoms, cure, and the aftermath of Ebola in eleven local languages. These messages originated from healthcare organizations including the Center for Disease Control (CDC), USA.

In 2015, I started project Rah e Maa (path of a mother) that employs Polly to spread reliable maternal health information among low-literate expectant fathers in Pakistan using a hotline called Super Abbu (Super Dad) [17, 18]. The hotline targets fathers because they are the primary decision-makers in Pakistani households. Super Abbu allows its open community of users to record health questions that are answered by volunteer doctors. Users can also share their experiences (after content validation) with their peers. Super Abbu reached 21,000 users (96% of them men) in just two months, uncovering a pent-up demand for maternal health information and giving the target population an agency to anonymously access culturally sensitive yet lifesaving reliable information. Super Abbu was supported via a USD 100,000 grant from UNICEF Innovations Fund. This work has recently been featured on the BBC [19] and our paper comparing mechanisms for advertising Super Abbu received an Honorable Mention award at CHI 2020 [20].

Around this time, development-oriented voice-based telephone services started gaining widespread use throughout the developing world. These deployments have uncovered new challenges and requirements unique to these services that include large-scale impact evaluation, long-term user-retention, and Internet-scale content management.

2. Rapid Impact-evaluation of Voice-based Information Services

A major challenge of large-scale information dissemination is impact measurement and quantification of changes in beliefs and behaviors. Design of successful information campaigns requires rapid discovery of existing knowledge gaps in the community about known topics-of-interest, dissemination of specific information to address these gaps, and a follow-up measurement of knowledge retention. I use voice-based quizzes, embedded in information services, to measure the baseline knowledge of the target communities about predefined topics-of-interest, to disseminate information via the announcement of correct responses of quiz questions and measurement of knowledge retention by asking rephrased versions of these questions at various intervals-of-interest. As an example, I created Sawaal [21] that allows its open community of users to post and attempt multiple-choice questions and to vote and comment on them. Sawaal is designed to spread virally as users challenge friends via shared quizzes and compete for high scores. Administrator-posted questions allow discovering knowledge gaps, spreading correct information and measuring knowledge-retention via rephrased, repeated questions. Community-contributed quiz content and an ability to play against friends for high scores, leads to inclusion and ownership, active collaborative learning and a spirit of competition among the users. Sawaal spreads organically among the target audience, receives an enthusiastic response, and successfully retains a significant fraction of the users for several weeks.

3. Voice-based Social Connectivity for Under-served Populations

User-retention and large-scale content moderation have emerged as significant challenges of voice-based services. Entertainment-driven services like Polly face high churn as the novelty of entertainment wears off and users leave the service at a rapid rate. Community-driven services like Super Abbu entail a significant workload of content moderation to filter and approve contributed content. To address these challenges, I focused on engaging the target populations in sustained long-term interactions, involving them in content moderation, and providing them community support, a voice, and digital social identity. I achieved this via voice-based social networks accessible over simple phones. To this end, I created Baang, an IVR-based versatile and inclusive social platform with mechanisms to achieve greater spread and uptake as well as deeper and long-term engagement built into the service [22]. Baang allows audio content creation and sharing among its open community of users. Once advertised, it keeps attracting new users while old users also keep returning to post new material and access, enjoy and assess posted content. Due to its viral and organic nature, Baang automatically overcomes user recruitment and content diversity challenges. Baang creates a vibrant community of users from diverse socio-economic and linguistic backgrounds including visually impaired, low-educated, unemployed, young men from all over Pakistan. Baang’s open community included people from remote areas and linguistic minorities. Social network features like voting, content sharing, and voice comments lead to community-driven content classification, viral and enthusiastic uptake of the service, high user engagement and retention, and true dialog among the users. Browsing and scoring mechanisms of Baang ensure majority-driven quality assurance but not at the risk of drowning the voice of minorities. Baang provides a window into the collective values of a community as they raise their voice against disability abuse, female harassment, foul language, hatred, terrorism and unite for their rights and in support of the oppressed.

Despite the fact that voice-based platforms provide under-served populations with information and social connectivity, we see a very low fraction of users identified as females on these and similar services around the world (Sangeet Swara (6% females) [23], Baang (10%), Sawaal (8%), CG Net Swara (12%) [24], and Ila Dhageyso (15%) [25]). With collaborators from the University of Washington, I have investigated the reasons for the low participation of women in social media voice forums by examining the use of Sangeet Swara in India and Baang in Pakistan by marginalized women and men. Our mixed-methods approach spanning content analysis of audio posts, quantitative analysis of interactions between users, and qualitative interviews with users indicate gender inequity due to deep-rooted patriarchal values. We found that women on these forums faced systemic discrimination and encountered abusive content, flirts, threats, and harassment [26]. These results reveal secondary barriers to social inclusion that are similar to the hurdles being faced by mainstream social media platforms.

4. Speech and Natural Language Processing

My research interests in Speech and NLP span core problems as well as localization of techniques and availability of corpora for under-resourced languages. I made available the first medium vocabulary spontaneous Automatic Speech Recognition (ASR) system for Urdu [27]. The publicly available corpora and linguistic resources developed as part of this research [28] form the backbone of several research projects and applications in this domain. The release of these resources was reported by Dawn News [29] and MIT Tech Review Pakistan [30]. I use my voice-based social networks for rapid collection of spontaneous speech data over the mobile phone channel [31]. These services reached 11,000 speakers and gathered 1,200 hours of speech data, spanning Urdu and nine regional languages. I have trained a high accuracy Urdu ASR using just 10 hours of this corpus. These forums automatically overcome common spontaneous speech data collection challenges like speaker recruitment, natural speech elicitation, content diversity, informed consent, sampling real-world ambient noise, and reach (for geographically remote communities). This technique is especially useful for gathering speech corpora for under-resourced languages hence enabling the development of ASR, keyword spotting, speaker ID, and noise classification systems (among others) for such languages. It also allows rapid, automatic preservation of spoken languages and oral aspects of culture. This technique can be extended to collect speech data for endangered languages, oral cultures, and linguistic minorities.

I have also been creating tools and making them available publicly to enable state-of-the-art language processing systems for Urdu. To enable Urdu ASR, I have made available an automatic pronunciation lexicon generation tool for Urdu [32] that uses LSTMs. This brings down the bar of linguistic and technical expertise for researchers working on such systems. NLP algorithms rely heavily on efficient word segmentation and Urdu exhibits unique challenges in this regard. I have recently released an automatic word segmentation system for Urdu [33] that uses Conditional Random Fields (CRFs). With Mr. Nasrullah Mehr, a famous calligrapher, I have released the first open type, character-based Urdu Nastaliq web font with size less than 110KB. Our work also gained media attention (MIT Tech Review Pakistan [34], Geo News [35], Jang News [36], Dawn News [37]). My students are also actively working on emotion classification, speaker identification, and keyword spotting (spoken term detection) systems for Urdu.

Publication Venues, Funding Sources, and Research Services

I publish in venues including CHI, InterSpeech, COLING, ICTD, and LREC. Over the last six years, the benefits of my research have directly reached more than 300,000 underserved people via projects in Pakistan (Polly, Job-Audio Portal, Hello-Rozgar, Baang, Sawaal, Super Abbu), India (Polly in collaboration with Babajob.com and Jharkhand Mobile Vaani) and West Africa (Polly-Message, Ebola-HealthLine). My research has been awarded highly competitive grants by prestigious donor organizations including Google Inc., UNICEF Innovations Fund, Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ), National Institutes of Health, USA (NIH), the National Academies of Sciences, Engineering, Medicine, Keck Futures Initiative (NAKFI), and Higher Education Commission (HEC) of Pakistan’s initiatives on NRPU and NCBC. I was an Associate Chair for CHI 2019, short papers PC chair for ICTD 2017 and have been a PC member for ICTD, and ACM COMPASS (formerly ACM DEV) and a reviewer for CHI and DIS.

Current Projects

Currently, my work is being funded by four awards that capture my research direction over the next three years.

1. Stories as Vehicles for Measuring and Influencing Behavior

In 2018, I received a USD 500,000 from the National Academies of Sciences, Engineering, Medicine, Keck Futures Initiative (NAKFI), in collaboration with the University of Michigan, Ann Arbor and University of Cape Town. We are focusing on designing and deploying automated mechanisms of persuasive behavior-changing communication. I am looking into the use of choose-your-own-adventure style audio stories as a means of measuring and influencing peoples’ attitudes. Users will assume the roles of story characters and make decisions on their behalf to unravel subsequent stages of the stories. Such role-play is expected to inculcate more engagement and ownership. Each story will be designed to convey specific development-related lessons (such as lessons related to maternal health, hygiene, and childcare). As the story reaches its culmination, reasons for the outcomes and their causal relationship with users’ decisions would be revealed to persuade them to reconsider their choices. Users will be offered to change their decisions made during the course of the story to find out how the final outcome gets impacted as a result. Finally, user behavior in response to similar decisions in other stories can reveal changes in their thinking. Stories have been used in the past as mechanisms of knowledge-spread however we do not find examples of such attempts with under-served populations using speech over simple phones. We have reported our findings from user studies in CHI 2020 [38].

2. Impact Evaluation of Super Abbu Hotline

In 2018, I won an award of USD 250,000 from the National Institutes of Health (NIH), USA, in collaboration with the University of California, Davis, and University of Michigan, Ann Arbor. This two-year award allows us to perform an impact evaluation of Super Abbu to investigate improvements in knowledge, attitudes, behaviors and birth outcomes using Randomized Controlled Trials (RCTs). We are currently conducting an ethnographic survey and recruiting expectant parents as participants. Each treatment arm of these participants will be exposed to a different version of Super Abbu. We plan to experiment with adding social connectivity and quiz-based automatic evaluation on top of the basic version of Super Abbu and its impact on usability, user retention, knowledge retention, and behavior change. Towards the end of the year-long intervention, an endline survey will reveal comparative changes.

3. Speech Forensics, Voice Biometrics, Proof-of-Life

My lab has been funded USD 600,000 by the Higher Education Commission of Pakistan as part of the National Center in Big Data and Cloud Computing initiative. We will develop Speech Recognition, Keyword Spotting in spontaneous speech and Speaker Identification for Urdu. These modules will eventually be used by law enforcement agencies for speech forensics to help with the prevention and investigation of crime.

4. Plagiarism Detection for Urdu

Currently, there is no tool available to detect plagiarism for digital text written in Urdu and other local languages of Pakistan. This is alarming as more than 30 Pakistani universities offer undergraduate to Ph.D. level programs in Urdu language departments, where all of their research and theses are written in Urdu. I have been funded by the HEC to develop a rich Urdu plagiarism corpus consisting of documents with various levels of academic plagiarism (e.g. near copy, light revision, heavy revision, non-plagiarized, paraphrased), and try to adapt state-of-the-art plagiarism detection techniques to develop a plagiarism detection tool for Urdu.

Long-term Research Agenda

My long-term research goals follow from the four questions that I mentioned in the beginning. The following proposed projects capture these research direction and plans.

1. Preservation of Endangered Languages using Voice-based Social Networks

About 2,500 languages have been declared endangered by UNESCO, and more than 50% of the world’s languages are expected to become extinct by the year 2100. In Pakistan, of the 71 local languages, 28 have been declared endangered (of the 471 endangered languages in the world, 369 are in Asia). When a language disappears, an entire culture goes extinct along with the wealth of knowledge and wisdom gathered over centuries. I am developing automated mechanisms to preserve endangered languages. Major challenges of language preservation and corpus collection include reaching remote populations, speaker recruitment, elicitation of natural speech, and choice of genres. My proposed mechanism promises to overcome all of these challenges employing voice-based social platforms to elicit folktales and stories in local languages from native speakers. These platforms will allow story-tellers and their audience, the community, to replicate their story-telling social practices in speech-over-phone modality. Users will record stories and folktales and listen to content recorded by others, hence contributing data towards the preservation of their native language. Soft incentives like social media popularity gauged through likes, and comments will motivate the community to contribute content and take part in its moderation and quality assurance. Peer-to-peer sharing will promote the viral spread of the service leading to automatic speaker recruitment.

2. Persuasive Information Dissemination using Psychological Therapeutic Techniques

Over the last ten years, my research has progressed from developing strategies for information dissemination to the measurement of impact on knowledge and behaviors. One of my future research directions is to employ psychological techniques, such as Cognitive Behavior Therapy (CBT), to create persuasive modes of automated behavior changing communication using dialog systems. I am already working on emotion detection in spontaneous telephone speech that could be a helpful component for this line of research. I plan to systematically develop persuasive communication techniques initially as an aid for on-ground intervention workers and eventually as standalone services. A CBT-based interview could be guided by a tree of options helping surveyors. Eventually, the human can be taken out of the procedure, transferring more control to the dialog system.

3. Tackling Secondary Barriers to Social Inclusion

The speech-over-phones line of research has reached a point where new challenges are being uncovered. These challenges include secondary barriers to social inclusion and creation of marginalities within marginalities. Detailed analysis of user behavior on voice-based social platforms is revealing attitudes like female harassment, misinformation, and disinformation. Some of these challenges are similar to the challenges faced by mainstream social media platforms such as WhatsApp, Twitter, and Facebook. However, most of the solutions developed for textual and visual social media are not directly applicable to voice-based social platforms. I wish to focus on the creation of a more inclusive social media for the world – individuals as well as collectives.

References

  1. K. Sprague, F. Grijpink, J. Manyika, L. Moodley, B. Chappuis, K. Pattabiraman, and J. Bughin, “Offline and falling behind: barriers to internet adoption. mckinsey & company,” Technical Report, Tech. Rep., 2014.
  2. W. J. Ong, Orality and literacy. Routledge, 2013.
  3. A. A. Raza, M. Pervaiz, C. Milo, S. Razaq, G. Alster, J. Sherwani, U. Saif, and R. Rosenfeld, “Viral entertainment as a vehicle for disseminating speech-based services to low-literate users,” in Proceedings of the Fifth International Conference on Information and Communication Technologies and Development. ACM, 2012, pp. 350–359.
  4. A. A. Raza, F. Ul Haq, Z. Tariq, M. Pervaiz, S. Razaq, U. Saif, and R. Rosenfeld, “Job opportunities through entertainment: Virally spread speech-based services for low-literate users,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2013, pp. 2803–2812.
  5. J. Palca, “How a drunken chipmunk voice helps send a public service message,” Jun 2015. [Online]. Available: http://¬www.npr.org/¬sections/-goatsandsoda/¬2015/¬06/¬01/¬410026531/¬how-a-drunken-chipmunk-voice-helps-send-a-public-service-message
  6. “New tab,” http://¬link.cs.cmu.edu/¬files/¬The_Link_CMU_Spring_2013.pdf, (Accessed on 03/10/2019).
  7. “Phone game helps illiterate pakistanis find employment - the tartan,” http://¬thetartan.org/¬2013/¬4/¬29/¬scitech/¬parrots, (Accessed on 03/10/2019).
  8. “www.technologyreview.com,” http://¬www.technologyreview.com/¬news/¬512156/¬viral-phone-game-helps-illiterate-pakistanis-find-job-listings/¬, (Accessed on 03/10/2019).
  9. “Technology: Phone-based job portal, social network introduced | the express tribune,” https://¬tribune.com.pk/¬story/¬1085665/¬technology-phone-based-job-portal-social-network-introduced/¬, (Accessed on 03/10/2019).
  10. http://¬articles.timesofindia.indiatimes.com/¬2013-04-19/¬job-trends/¬38672942_1_telephone-polly-management-sciences, (Accessed on 03/10/2019).
  11. “404 page | ieee signal processing society,” https://¬signalprocessingsociety.org/¬technical-committees/¬list/¬sl-tc/¬spl-nl/¬2013-08/-SpeechInterfacesforLow-LiterateUsers/¬, (Accessed on 03/10/2019).
  12. “https://www.sciencenewsline.com/articles/2013041815550005.html,” https://¬www.sciencenewsline.com/¬articles/-2013041815550005.html, (Accessed on 03/10/2019).
  13. “old.post-gazette.com,” http://¬old.post-gazette.com/¬stories/¬news/¬science/¬a-game-called-polly-is-designed-to-promote-literacy-686462, (Accessed on 03/10/2019).
  14. “Funny phone game helping illiterates in pak get job opportunities,” https://¬in.finance.yahoo.com/¬news/¬funny-phone-game-helping-illiterates-072623468.html?guccounter=1&guce_referrer=aHR0cDovL2FnaGFhbGlyYXphLmNvbS9wcmVzcy5odG1s&guce_referrer_sig=AQAAABvn-NCrvRizSJ5-XhqNywRhVMmHvEItBpyyctu5wRh69oGcuesjp_Wtr2OwZ6YKk1Qc97i7l8Tx_lHrPxOIJbuIiKcg21LeA_cBKbiKDQlm-7E6aw1RXjFjm87RV_bVlHd3jzEMOR-Ue4QGGls3jgSscKUt6q2ZuJcLDfD8efwx, (Accessed on 03/10/2019).
  15. A. A. Raza, R. Kulshreshtha, S. Gella, S. Blagsvedt, M. Chandrasekaran, B. Raj, and R. Rosenfeld, “Viral spread via entertainment and voice-messaging among telephone users in india,” in Proceedings of the Eighth International Conference on Information and Communication Technologies and Development. ACM, 2016, p. 1.
  16. N. Wolfe, J. Hong, A. A. Raza, B. Raj, and R. Rosenfeld, “Rapid development of public health education systems in low-literacy multilingual environments: Combating ebola through voice messaging,” in ISCA Special Interest Group on Speech and Language Technology in Education (SLaTE). INTERSPEECH, 2015.
  17. S. S.-O. Ahmad, M. Naseem, and A. A. Raza, “Maternal awareness for low-literate expecting parents via voice-based telephone services.” HCI, 2017.
  18. S. S.-O. Ahmad, M. B. Saleem, M. Ayub, T. Ahmad, S. Randhawa, Z. Tariq, M. Naseem, and A. A. Raza, “Usage and feedback from a 3-week launch of a maternal health line for men in pakistan.”
  19. https://¬www.bbc.com/¬urdu/¬science-47140529, (Accessed on 03/10/2019).
  20. M. Naseem, B. Saleem, N.-A. Dhabi, S. Ahmad, J. Chen, and A. A. Raza, “An empirical comparison of technologically mediated advertising in under-connected populations,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, 2020.
  21. A. A. Raza, Z. Tariq, S. Randhawa, B. Saleem, A. Athar, U. Saif, and R. Rosenfeld, “Voice-based quizzes for measuring knowledge retention in hard-to-reach populations,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 2019.
  22. A. A. Raza, B. Saleem, S. Randhawa, Z. Tariq, A. Athar, U. Saif, and R. Rosenfeld, “Baang: A viral speech-based social platform for under-connected populations,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 2018, p. 643.
  23. A. Vashistha, E. Cutrell, G. Borriello, and W. Thies, “Sangeet swara: A community-moderated voice forum in rural india,” in Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2015, pp. 417–426.
  24. P. Mudliar, J. Donner, and W. Thies, “Emergent practices around cgnet swara, voice forum for citizen journalism in rural india,” in Proceedings of the Fifth International Conference on Information and Communication Technologies and Development. ACM, 2012, pp. 159–168. 
  25. M. Gulaid and A. Vashistha, “Ila dhageyso: an interactive voice forum to foster transparent governance in somaliland,” in Proceedings of the Sixth International Conference on Information and Communications Technologies and Development: Notes-Volume 2. ACM, 2013, pp. 41–44.
  26. A. Vashistha, A. Garg, R. Anderson, and A. A. Raza, “Threats, abuses, flirting, and blackmail: Gender inequity in social media voice forums,” 2019. 
  27. A. A. Raza, S. Hussain, H. Sarfraz, I. Ullah, and Z. Sarfraz, “An asr system for spontaneous urdu speech,” the Proc. of Oriental COCOSDA, pp. 24–25, 2010. 
  28. A. A. Raza, S. Hussain, H. Sarfraz, I. Ullah, and Z.Sarfraz, “Design and development of phonetically rich urdu speech corpus,” in Speech Database and Assessments, 2009 Oriental COCOSDA International Conference on. IEEE, 2009, pp. 38–43. 
  29. “Pakistani researchers lay the foundation for building urdu speech recognition system - pakistan - dawn.com,” https://¬www.dawn.com/-news/¬1310242, (Accessed on 03/10/2019).
  30. “Pakistani researchers lay the foundation for building urdu speech recognition system,” http://¬www.technologyreview.pk/¬one-step-forward-towards-urdu-speech-recognition-apps/¬, (Accessed on 03/10/2019).
  31. A. A. Raza, A. Athar, S. Randhawa, Z. Tariq, M. B. Saleem, H.-B. Zia, U. Saif, and R. Rosenfeld, “Rapid collection of spontaneous speech corpora using telephonic community forums,” Proc. Interspeech 2018, pp. 1021–1025, 2018.
  32. H. B. Zia, A. A. Raza, and A. Athar, “Pronouncur: An urdu pronunciation lexicon generator,” 2018.
  33. H. Zia, A. A. Raza, and A. Athar, “Urdu word segmentation using conditional random fields (crfs),” Proceedings of the 27th International Conference on Computational Linguistics, 2010. 
  34. “Pakistani researchers develop a calligraphy-based urdu font for the internet,” http://¬www.technologyreview.pk/¬pakistani-researchers-develop-calligraphy-based-urdu-font-internet/¬, (Accessed on 03/10/2019). 
  35. https://¬urdu.geo.tv/¬latest/¬158722-internet-ky-ly-khatati-per-mabni-urdu-font-tayar, (Accessed on 03/10/2019). 
  36. “Calligraphy urdu font ready for internet users | misc-news - daily jang 08 february 2017,” https://¬jang.com.pk/¬news/¬264242-calligraphy-urdu-font-ready-for-internet-users, (Accessed on 03/10/2019).
  37. “Pakistani researchers develop a calligraphy-based urdu font for the internet - pakistan - dawn.com,” https://¬www.dawn.com/¬news/-1313737/¬pakistani-researchers-develop-a-calligraphy-based-urdu-font-for-the-internet, (Accessed on 03/10/2019). 
  38. M. Mustafa, A. Batool, B. Fatima, F. Nawaz, K. Toyama, and A. A. Raza, “Patriarchy, maternal health and spiritual healing: Designing maternal health interventions in pakistan,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, 2020.