16 citations · 27 across the 14 of their papers we have counts for
15 papers
Pak3H: Evaluating the Cost of Cultural Mismatch in LLM Alignment with a Human-Contextualized Urdu Benchmark
Abdullah Hashmat, Usman Naseem, Agha Ali Raza
Large language models (LLMs) demonstrate strong Helpfulness, Harmlessness, and Honesty (3H) alignment in English-centric settings, but these gains transfer poorly to low-resource l…
Empathy Applicability Modeling for General Health Queries
Shan Randhawa, Agha Ali Raza, Kentaro Toyama +2
LLMs are increasingly being integrated into clinical workflows, yet they often lack clinical empathy, an essential aspect of effective doctor-patient communication. Existing NLP fr…
PakBBQ: A Culturally Adapted Bias Benchmark for QA
Abdullah Hashmat, Muhammad Arham Mirza, Agha Ali Raza
With the widespread adoption of Large Language Models (LLMs) across various applications, it is empirical to ensure their fairness across all user communities. However, most LLMs a…
Language Model-Driven Data Pruning Enables Efficient Active Learning
Abdul Hameed Azeemi, Ihsan Ayyub Qazi, Agha Ali Raza
Active learning (AL) optimizes data labeling efficiency by selecting the most informative instances for annotation. A key component in this procedure is an acquisition function tha…
With a Grain of SALT: Are LLMs Fair Across Social Dimensions?
Samee Arif, Zohaib Khan, Maaidah Kaleem +3
This paper presents a systematic analysis of biases in open-source Large Language Models (LLMs), across gender, religion, and race. Our study evaluates bias in smaller-scale Llama…
Kahaani: A Multimodal Co-Creative Storytelling System
Samee Arif, Muhammad Saad Haroon, Aamina Jamal Khan +3
This paper introduces Kahaani, a multimodal, co-creative storytelling system that leverages Generative Artificial Intelligence, designed for children to address the challenge of su…
WER We Stand: Benchmarking Urdu ASR Models
Samee Arif, Sualeha Farid, Aamina Jamal Khan +3
This paper presents a comprehensive evaluation of Urdu Automatic Speech Recognition (ASR) models. We analyze the performance of three ASR model families: Whisper, MMS, and Seamless…
Beyond Uniform Query Distribution: Key-Driven Grouped Query Attention
Zohaib Khan, Muhammad Khaquan, Omer Tafveez +2
The Transformer architecture has revolutionized deep learning through its Self-Attention mechanism, which effectively captures contextual information. However, the memory footprint…
The Fellowship of the LLMs: Multi-Model Workflows for Synthetic Preference Optimization Dataset Generation
Samee Arif, Sualeha Farid, Abdul Hameed Azeemi +2
This paper presents a novel methodology for generating synthetic Preference Optimization (PO) datasets using multi-model workflows. We evaluate the effectiveness and potential of t…
Generalists vs. Specialists: Evaluating Large Language Models for Urdu
Samee Arif, Abdul Hameed Azeemi, Agha Ali Raza +1
In this paper, we compare general-purpose models, GPT-4-Turbo and Llama-3-8b, with special-purpose models--XLM-Roberta-large, mT5-large, and Llama-3-8b--that have been fine-tuned o…
UQA: Corpus for Urdu Question Answering
Samee Arif, Sualeha Farid, Awais Athar +1
This paper introduces UQA, a novel dataset for question answering and text comprehension in Urdu, a low-resource language with over 70 million native speakers. UQA is generated by…
To Label or Not to Label: Hybrid Active Learning for Neural Machine Translation
Abdul Hameed Azeemi, Ihsan Ayyub Qazi, Agha Ali Raza
Active learning (AL) techniques reduce labeling costs for training neural machine translation (NMT) models by selecting smaller representative subsets from unlabeled data for annot…
Representative Subset Selection for Efficient Fine-Tuning in Self-Supervised Speech Recognition
Abdul Hameed Azeemi, Ihsan Ayyub Qazi, Agha Ali Raza
Self-supervised speech recognition models require considerable labeled training data for learning high-fidelity representations for Automatic Speech Recognition (ASR) which is comp…
Urdu Word Segmentation using Conditional Random Fields (CRFs)
Haris Bin Zia, Agha Ali Raza, Awais Athar
State-of-the-art Natural Language Processing algorithms rely heavily on efficient word segmentation. Urdu is amongst languages for which word segmentation is a complex task as it e…
PronouncUR: An Urdu Pronunciation Lexicon Generator
Haris Bin Zia, Agha Ali Raza, Awais Athar
State-of-the-art speech recognition systems rely heavily on three basic components: an acoustic model, a pronunciation lexicon and a language model. To build these components, a re…