9 papers
Empathy Applicability Modeling for General Health Queries
Shan Randhawa, Agha Ali Raza, Kentaro Toyama +2
LLMs are increasingly being integrated into clinical workflows, yet they often lack clinical empathy, an essential aspect of effective doctor-patient communication. Existing NLP fr…
Kahaani: A Multimodal Co-Creative Storytelling System
Samee Arif, Muhammad Saad Haroon, Aamina Jamal Khan +3
This paper introduces Kahaani, a multimodal, co-creative storytelling system that leverages Generative Artificial Intelligence, designed for children to address the challenge of su…
PakBBQ: A Culturally Adapted Bias Benchmark for QA
Abdullah Hashmat, Muhammad Arham Mirza, Agha Ali Raza
With the widespread adoption of Large Language Models (LLMs) across various applications, it is empirical to ensure their fairness across all user communities. However, most LLMs a…
The Fellowship of the LLMs: Multi-Model Workflows for Synthetic Preference Optimization Dataset Generation
Samee Arif, Sualeha Farid, Abdul Hameed Azeemi +2
This paper presents a novel methodology for generating synthetic Preference Optimization (PO) datasets using multi-model workflows. We evaluate the effectiveness and potential of t…
WER We Stand: Benchmarking Urdu ASR Models
Samee Arif, Sualeha Farid, Aamina Jamal Khan +3
This paper presents a comprehensive evaluation of Urdu Automatic Speech Recognition (ASR) models. We analyze the performance of three ASR model families: Whisper, MMS, and Seamless…
With a Grain of SALT: Are LLMs Fair Across Social Dimensions?
Samee Arif, Zohaib Khan, Maaidah Kaleem +3
This paper presents a systematic analysis of biases in open-source Large Language Models (LLMs), across gender, religion, and race. Our study evaluates bias in smaller-scale Llama…