1 citations · 3 across the 11 of their papers we have counts for
5 papers · 1 filter
Interpreting Model Reasoning through Behavioral Counterfactuals
Shawn Im, James Oldfield, Adel Bibi +1
Faithfully understanding the internal process that occurs as models form their chain of thought is an important problem with the rise in model capabilities. However, this remains c…
How Much Do Activation Monitors Benefit from Original Model Computations?
James Oldfield, Shawn Im, Jay J Culligan +3
One powerful way to safeguard AI systems is to monitor their internal computations--where signs of undesirable behavior might be more easily detectable than from models' inputs and…
Normalized Rewards for Preference Optimization
Shawn Im, Federico Danieli, Skyler Seto +2
Direct Alignment Algorithms (DAAs) such as DPO have become a common way to post-train and align LLMs with human preferences. However, DAAs have been observed to over-optimize their…
Cyclical Entropy Eruption: Entropy Dynamics in Agent Reinforcement Learning
Wendi Li, Shawn Im, Sharon Li
Agentic large language models are increasingly used to solve real-world tasks by reasoning over goals, invoking tools, and interacting with external environments. Reinforcement lea…
How Do Transformers Learn to Associate Tokens: Gradient Leading Terms Bring Mechanistic Interpretability
Shawn Im, Changdae Oh, Zhen Fang +1
Semantic associations such as the link between "bird" and "flew" are foundational for language modeling as they enable models to go beyond memorization and instead generalize and g…