11 papers
Selective Safety Steering via Value-Filtered Decoding
Bat-Sheva Einbinder, Hen Davidov, Yee Whye Teh +2
While large language models (LLMs) are trained to align with human values, their generations may still violate safety constraints. A growing line of work addresses this problem by…
Building Reliable Long-Form Generation via Hallucination Rejection Sampling
Lin Li, Georgia Channing, Suhaas M Bhat +2
Large language models (LLMs) have achieved remarkable progress in open-ended text generation, yet they remain prone to hallucinating incorrect or unsupported content, which undermi…
Training Transformers for KV Cache Compressibility
Yoav Gelberg, Yam Eitan, Michael Bronstein +2
Long-context language modeling is increasingly constrained by the Key-Value (KV) cache, whose memory and decode-time access costs scale linearly with the prefix length. This bottle…
Muon is Not That Special: Random or Inverted Spectra Work Just as Well
Zakhar Shumaylov, Nathaël Da Costa, Peter Zaika +6
The recent empirical success of the Muon optimizer has renewed interest in non-Euclidean optimization, typically justified by similarities with second-order methods, and linear min…
Uncertainty Quantification for LLM Function-Calling
Zihuiwen Ye, Lukas Aichberger, Michael Kirchhof +5
Large Language Models (LLMs) are increasingly deployed to autonomously solve real-world tasks. A key ingredient for this is the LLM Function-Calling paradigm, a widely used approac…
Simple Baselines are Competitive with Code Evolution
Yonatan Gideoni, Sebastian Risi, Yarin Gal
Code evolution is a family of techniques that rely on large language models to search through possible computer programs by evolving or mutating existing code. Many proposed code e…