14 papers
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
Disentangling Recall and Reasoning in Transformer Models through Layer-wise Attention and Activation Analysis
Harshwardhan Fartale, Ashish Kattamuri, Rahul Raja +3
Transformer-based language models excel at both recall (retrieving memorized facts) and reasoning (performing multi-step inference), but whether these abilities rely on distinct in…
Bridging the Semantic Gap: Contrastive Rewards for Multilingual Text-to-SQL with GRPO
Ashish Kattamuri, Ishita Prasad, Meetu Malhotra +3
Current Text-to-SQL methods are evaluated and only focused on executable queries, overlooking the semantic alignment challenge -- both in terms of the semantic meaning of the query…
Equilibrium Dynamics and Mitigation of Gender Bias in Synthetically Generated Data
Ashish Kattamuri, Arpita Vats, Harshwardhan Fartale +3
Recursive prompting with large language models enables scalable synthetic dataset generation but introduces the risk of bias amplification. We investigate gender bias dynamics acro…
FUSE : A Ridge and Random Forest-Based Metric for Evaluating MT in Indigenous Languages
Rahul Raja, Arpita Vats
This paper presents the winning submission of the RaaVa team to the AmericasNLP 2025 Shared Task 3 on Automatic Evaluation Metrics for Machine Translation (MT) into Indigenous Lang…
Multilingual State Space Models for Structured Question Answering in Indic Languages
Arpita Vats, Rahul Raja, Mrinal Mathur +2
The diversity and complexity of Indic languages present unique challenges for natural language processing (NLP) tasks, particularly in the domain of question answering (QA).To addr…