45 papers
The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models
Adeel Yousaf, Soumik Ghosh, James Beetham +2
Safety alignment of text-to-image (T2I) diffusion models aims to suppress harmful generations while preserving utility on benign prompts. Recent methods often appear to deliver hig…
The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding
Avinash Reddy, Thayne T. Walker, James S. Ide +1
Large language models (LLMs) are increasingly used to generate executable outputs, JSON objects, and API calls, where a single syntax error can make the output unusable. Constraine…
Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models
Prajakta Kini, Avinash Reddy, Souradip Chakraborty +4
Instruction-tuned LLMs are increasingly converted into reasoning models through post-training to improve multi-step task performance. This conversion is usually optimized for reaso…
The Hidden Bias of Process Reward Models:PRISM for Rewarding the Right Reasoning
Aakriti Agrawal, Souradip Chakraborty, Armin Saghafian +6
Process Reward Models (PRMs) improve credit assignment for reasoning by providing step-level feedback. However, we identify a hidden bias in PRMs caused by severe imbalance in step…
OmniToM: Benchmarking Theory of Mind in LLMs via Explicit Belief Modeling
Adam Bawatneh, Sagar Sapkota, Amrit Singh Bedi +2
Theory of Mind (ToM), the ability to infer others' knowledge, intentions, and emotions, is commonly evaluated in large language models (LLMs) using end-point question answering, wh…
RL with Learnable Textual Feedback: A Bilevel Approach
Utsav Singh, Sidhaarth Sredharan, Souradip Chakraborty +1
Reinforcement learning with verifiable rewards can improve LLM reasoning, but learning remains sample-inefficient when terminal rewards are sparse. This has motivated a growing lin…