5 papers
Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits
Areeb Ahmad, Abhinav Joshi, Ashutosh Modi
Transformer-based language models exhibit complex and distributed behavior, yet their internal computations remain poorly understood. Existing mechanistic interpretability methods…
Calibration Across Layers: Understanding Calibration Evolution in LLMs
Abhinav Joshi, Areeb Ahmad, Ashutosh Modi
Large Language Models (LLMs) have demonstrated inherent calibration capabilities, where predicted probabilities align well with correctness, despite prior findings that deep neural…
Towards Quantifying Commonsense Reasoning with Mechanistic Insights
Abhinav Joshi, Areeb Ahmad, Divyaksh Shukla +1
Commonsense reasoning deals with the implicit knowledge that is well understood by humans and typically acquired via interactions with the world. In recent times, commonsense reaso…
COLD: Causal reasOning in cLosed Daily activities
Abhinav Joshi, Areeb Ahmad, Ashutosh Modi
Large Language Models (LLMs) have shown state-of-the-art performance in a variety of tasks, including arithmetic and reasoning; however, to gauge the intellectual capabilities of L…
ScriptWorld: Text Based Environment For Learning Procedural Knowledge
Abhinav Joshi, Areeb Ahmad, Umang Pandey +1
Text-based games provide a framework for developing natural language understanding and commonsense knowledge about the world in reinforcement learning based agents. Existing text-b…