10 papers
Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models
Divyaksh Shukla, Ashutosh Modi
Machine unlearning aims to remove the influence of specific training data from a model while preserving reliable behavior on the remaining data, making reliable prediction and unce…
Geometry of Decision Making in Language Models
Abhinav Joshi, Divyanshu Bhatt, Ashutosh Modi
Large Language Models (LLMs) show strong generalization across diverse tasks, yet the internal decision-making processes behind their predictions remain opaque. In this work, we st…
Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits
Areeb Ahmad, Abhinav Joshi, Ashutosh Modi
Transformer-based language models exhibit complex and distributed behavior, yet their internal computations remain poorly understood. Existing mechanistic interpretability methods…
Calibration Across Layers: Understanding Calibration Evolution in LLMs
Abhinav Joshi, Areeb Ahmad, Ashutosh Modi
Large Language Models (LLMs) have demonstrated inherent calibration capabilities, where predicted probabilities align well with correctness, despite prior findings that deep neural…
POSESTITCH-SLT: Linguistically Inspired Pose-Stitching for End-to-End Sign Language Translation
Abhinav Joshi, Vaibhav Sharma, Sanjeet Singh +1
Sign language translation remains a challenging task due to the scarcity of large-scale, sentence-aligned datasets. Prior arts have focused on various feature extraction and archit…
IL-PCSR: Legal Corpus for Prior Case and Statute Retrieval
Shounak Paul, Dhananjay Ghumare, Pawan Goyal +2
Identifying/retrieving relevant statutes and prior cases/precedents for a given legal situation are common tasks exercised by law practitioners. Researchers to date have addressed…
CoMuMDR: Code-mixed Multi-modal Multi-domain corpus for Discourse paRsing in conversations
Divyaksh Shukla, Ritesh Baviskar, Dwijesh Gohil +3
Discourse parsing is an important task useful for NLU applications such as summarization, machine comprehension, and emotion recognition. The current discourse parsing datasets bas…
Towards Quantifying Commonsense Reasoning with Mechanistic Insights
Abhinav Joshi, Areeb Ahmad, Divyaksh Shukla +1
Commonsense reasoning deals with the implicit knowledge that is well understood by humans and typically acquired via interactions with the world. In recent times, commonsense reaso…
COLD: Causal reasOning in cLosed Daily activities
Abhinav Joshi, Areeb Ahmad, Ashutosh Modi
Large Language Models (LLMs) have shown state-of-the-art performance in a variety of tasks, including arithmetic and reasoning; however, to gauge the intellectual capabilities of L…
Towards Robust Evaluation of Unlearning in LLMs via Data Transformations
Abhinav Joshi, Shaswati Saha, Divyaksh Shukla +4
Large Language Models (LLMs) have shown to be a great success in a wide range of applications ranging from regular NLP-based use cases to AI agents. LLMs have been trained on a vas…