7 papers
Fisher8: Stabilizing Neural Heteroscedastic Regression via Output-Layer Fisher Geometry
Sumedh Vemuganti, Nickvash Kani
Training neural networks to jointly predict mean and uncertainty estimates from noisy observations can be unstable, prompting a series of independent stabilization efforts. We argu…
Parameter-Efficient Multi-Task Learning via Progressive Task-Specific Adaptation
Neeraj Gangwar, Anshuka Rangi, Rishabh Deshmukh +3
Parameter-efficient fine-tuning methods have emerged as a promising solution for adapting pre-trained models to various downstream tasks. While these methods perform well in single…
GiVA: Gradient-Informed Bases for Vector-Based Adaptation
Neeraj Gangwar, Rishabh Deshmukh, Michael Shavlovsky +4
As model sizes continue to grow, parameter-efficient fine-tuning has emerged as a powerful alternative to full fine-tuning. While LoRA is widely adopted among these methods, recent…
Integrating Arithmetic Learning Improves Mathematical Reasoning in Smaller Models
Neeraj Gangwar, Suma P Bhat, Nickvash Kani
While large models pre-trained on high-quality data exhibit excellent performance on mathematical reasoning (e.g., GSM8k, MultiArith), it remains challenging to specialize smaller…
Mathematical Derivation Graphs: A Relation Extraction Task in STEM Manuscripts
Vishesh Prasad, Brian Kim, Nickvash Kani
Recent advances in natural language processing (NLP), particularly with the emergence of large language models (LLMs), have significantly enhanced the field of textual analysis. Ho…
STEM-POM: Evaluating Language Models Math-Symbol Reasoning in Document Parsing
Jiaru Zou, Qing Wang, Pratyush Thakur +1
Advances in large language models (LLMs) have spurred research into enhancing their reasoning capabilities, particularly in math-rich STEM (Science, Technology, Engineering, and Ma…