15 citations · 31 across the 15 of their papers we have counts for
16 papers
Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluation
Jesse St. Amand, Callum Canavan, Sohaib Imran +5
Self-Generated Text Recognition (SGTR)--the ability of an LLM to identify its own outputs--poses risks to AI safeguards that rely on LLMs as evaluators or monitors: an LLM may reco…
What are Key Factors for Updates in RL for LLM Reasoning?
Peidong Wang, Demi Wang, Xufang Luo +5
Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as a promising framework for enhancing the reasoning ability of large language models. However, much of the existi…
Building Comparative Motivation Profiles with Instrumental Interventions
David Vella Zarb, Rustem Turtayev, Taywon Min +2
Safety evaluations often infer latent motivations from behavioral patterns, but the construct validity of these inferences is unclear. We study this problem in alignment faking, wh…
Self-Recognition Finetuning can Prevent and Reverse Emergent Misalignment
Arush Tagade, Shaoheng Zhou, Jiaxin Wen +1
Emergent misalignment (EM) has been linked to the activation of misaligned persona vectors and evil character traits, suggesting that EM operates through disruption of the model's…
Self-Improvement as Coherence Optimization: A Theoretical Account
Tianyi Qiu, Ahmed Hani Ismail, Zhonghao He +1
Can language models improve their accuracy without external supervision? Methods such as debate, bootstrap, and internal coherence maximization achieve this surprising feat, even m…
Gaming the Answer Matcher: Examining the Impact of Text Manipulation on Automated Judgment
Manas Khatore, Sumana Sridharan, Kevork Sulahian +2
Automated answer matching, which leverages LLMs to evaluate free-text responses by comparing them to a reference answer, shows substantial promise as a scalable and aligned alterna…
Emergent Persuasion: Will LLMs Persuade Without Being Prompted?
Vincent Chang, Thee Ho, Sunishchal Dev +4
With the wide-scale adoption of conversational AI systems, AI are now able to exert unprecedented influence on human opinion and beliefs. Recent work has shown that many Large Lang…
Mitigating Self-Preference by Authorship Obfuscation
Taslim Mahbub, Shi Feng
Language models (LMs) judges are widely used to evaluate the quality of LM outputs. Despite many advantages, LM judges display concerning biases that can impair their integrity in…
Parameter-Efficient Construction of the Rashomon Slice for Concept Bottleneck Models
Shihan Feng, Cheng Zhang, Michael Xi +3
In many machine learning problems, there may exist multiple models that achieve nearly identical predictive performance while relying on fundamentally different internal logic. How…
DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL
Rui Lu, Zhenyu Hou, Zihan Wang +6
Augmenting large language models (LLMs) with browsing tools substantially improves their potential as deep search agents to solve complex, real-world tasks. Yet, open LLMs still pe…
AI Debate Aids Assessment of Controversial Claims
Salman Rahman, Sheriff Issaka, Ashima Suvarna +11
As AI grows more powerful, it will increasingly shape how we understand the world. But with this influence comes the risk of amplifying misinformation and deepening social divides-…
Do LLM Evaluators Prefer Themselves for a Reason?
Wei-Lin Chen, Zhepei Wei, Xinyu Zhu +2
Large language models (LLMs) are increasingly used as automatic evaluators in applications such as benchmarking, reward modeling, and self-refinement. Prior work highlights a poten…
Language Models Learn to Mislead Humans via RLHF
Jiaxin Wen, Ruiqi Zhong, Akbir Khan +6
Language models (LMs) can produce errors that are hard to detect for humans, especially when the task is complex. RLHF, the most popular post-training method, may exacerbate this p…
Spontaneous Reward Hacking in Iterative Self-Refinement
Jane Pan, He He, Samuel R. Bowman +1
Language models are capable of iteratively improving their outputs based on natural language feedback, thus enabling in-context optimization of user preference. In place of human u…
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
Daking Rai, Yilun Zhou, Shi Feng +2
Mechanistic interpretability (MI) is an emerging sub-field of interpretability that seeks to understand a neural network model by reverse-engineering its internal computations. Rec…
LLM Evaluators Recognize and Favor Their Own Generations
Arjun Panickssery, Samuel R. Bowman, Shi Feng
Self-evaluation using large language models (LLMs) has proven valuable not only in benchmarking but also methods like reward modeling, constitutional AI, and self-refinement. But n…