6 papers
Understanding and Exploiting Anisotropy in Post-Training
Samyak Jha, Harshvardhan Saini, Yizhen Liao +2
LLM post-training combines supervised fine-tuning (SFT), a mode-covering forward-KL objective, with reinforcement learning (RL), a mode-seeking reverse-KL objective. Frequency-weig…
Masking Frequent Tokens Sharpens Direct Preference Optimization
Harshvardhan Saini, Samyak Jha, Yiming Tang +1
Direct Preference Optimization (DPO) aligns language models by optimizing over sequence-level sums of token-wise implicit reward differences. However, we identify a pervasive patho…
Capsule Lens: Locating and Tracking Concept Geometry in Model Representations
Yiming Tang, Harshvardhan Saini, Samyak Jha +3
Understanding how concepts are encoded in the internal representations of machine learning models is a central problem in mechanistic interpretability, essential both for the scien…
When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models
Harshvardhan Saini, Samyak Jha, Yiming Tang +1
Vision-Language Models (VLMs) increasingly power high-stakes applications, from medical imaging to autonomous systems, yet they routinely hallucinate, confidently describing conten…
CAPA: Contribution-Aware Pruning and FFN Approximation for Efficient Large Vision-Language Models
Samyak Jha, Junho Kim
Efficient inference in Large Vision-Language Models is constrained by the high cost of processing thousands of visual tokens, yet it remains unclear which tokens and computations c…
CRoPS: A Training-Free Hallucination Mitigation Framework for Vision-Language Models
Neeraj Anand, Samyak Jha, Udbhav Bamba +1
Despite the rapid success of Large Vision-Language Models (LVLMs), a persistent challenge is their tendency to generate hallucinated content, undermining reliability in real-world…