5 papers
Test-time Recursive Thinking: Self-Improvement without External Feedback
Yufan Zhuang, Chandan Singh, Liyuan Liu +5
Modern Large Language Models (LLMs) have shown rapid improvements in reasoning capabilities, driven largely by reinforcement learning (RL) with verifiable rewards. Here, we ask whe…
Training Language Models to Generate Quality Code with Program Analysis Feedback
Feng Yao, Zilong Wang, Liyuan Liu +7
Code generation with large language models (LLMs), often termed vibe coding, is increasingly adopted in production but fails to ensure code quality, particularly in security (e.g.,…
Text Generation Beyond Discrete Token Sampling
Yufan Zhuang, Liyuan Liu, Chandan Singh +2
In standard autoregressive generation, an LLM predicts the next-token distribution, samples a discrete token, and then discards the distribution, passing only the sampled token as…
Interpretable Next-token Prediction via the Generalized Induction Head
Eunji Kim, Sriya Mantena, Weiwei Yang +3
While large transformer models excel in predictive performance, their lack of interpretability restricts their usefulness in high-stakes domains. To remedy this, we propose the Gen…
Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering
Qingru Zhang, Xiaodong Yu, Chandan Singh +6
Large language models (LLMs) have demonstrated remarkable performance across various real-world tasks. However, they often struggle to fully comprehend and effectively utilize thei…