4 papers
Do LLMs Encode Functional Importance of Reasoning Tokens?
Janvijay Singh, Dilek Hakkani-Tür
Large language models solve complex tasks by generating long reasoning chains, achieving higher accuracy at the cost of increased computational cost and reduced ability to isolate…
On the Shelf Life of Fine-Tuned LLM-Judges: Future-Proofing, Backward-Compatibility, and Question Generalization
Janvijay Singh, Austin Xu, Yilun Zhou +3
The LLM-as-a-judge paradigm is widely used in both evaluating free-text model responses and reward modeling for model alignment and fine-tuning. Recently, fine-tuning judges with j…
Variation in Verification: Understanding Verification Dynamics in Large Language Models
Yefan Zhou, Austin Xu, Yilun Zhou +3
Recent advances have shown that scaling test-time computation enables large language models (LLMs) to solve increasingly complex problems across diverse domains. One effective para…
AURA: A Diagnostic Framework for Tracking User Satisfaction of Interactive Planning Agents
Takyoung Kim, Janvijay Singh, Shuhaib Mehri +6
The growing capabilities of large language models (LLMs) in instruction-following and context-understanding lead to the era of agents with numerous applications. Among these, task…