1 citations · 3 across the 11 of their papers we have counts for
15 papers
Interpreting Model Reasoning through Behavioral Counterfactuals
Shawn Im, James Oldfield, Adel Bibi +1
Faithfully understanding the internal process that occurs as models form their chain of thought is an important problem with the rise in model capabilities. However, this remains c…
How Much Do Activation Monitors Benefit from Original Model Computations?
James Oldfield, Shawn Im, Jay J Culligan +3
One powerful way to safeguard AI systems is to monitor their internal computations--where signs of undesirable behavior might be more easily detectable than from models' inputs and…
Normalized Rewards for Preference Optimization
Shawn Im, Federico Danieli, Skyler Seto +2
Direct Alignment Algorithms (DAAs) such as DPO have become a common way to post-train and align LLMs with human preferences. However, DAAs have been observed to over-optimize their…
Cyclical Entropy Eruption: Entropy Dynamics in Agent Reinforcement Learning
Wendi Li, Shawn Im, Sharon Li
Agentic large language models are increasingly used to solve real-world tasks by reasoning over goals, invoking tools, and interacting with external environments. Reinforcement lea…
How Do Transformers Learn to Associate Tokens: Gradient Leading Terms Bring Mechanistic Interpretability
Shawn Im, Changdae Oh, Zhen Fang +1
Semantic associations such as the link between "bird" and "flew" are foundational for language modeling as they enable models to go beyond memorization and instead generalize and g…
How Well Can Preference Optimization Generalize Under Noisy Feedback?
Shawn Im, Sharon Li
As large language models (LLMs) advance their capabilities, aligning these models with human preferences has become crucial. Preference optimization, which trains models to disting…
Non-equilibrium Dynamics of Two-level Systems directly after Cryogenic Alternating Bias
V. Iaia, E. S. Joseph, S. Im +10
Two-level systems (TLSs) are tunneling states commonly found in amorphous materials that electrically couple to qubits, resonators, and vibrational modes in materials, leading to e…
Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoders
James Oldfield, Shawn Im, Sharon Li +3
Multilayer perceptrons (MLPs) are an integral part of large language models, yet their dense representations render them difficult to understand, edit, and steer. Recent methods le…
Visual Instruction Bottleneck Tuning
Changdae Oh, Jiatong Li, Shawn Im +1
Despite widespread adoption, multimodal large language models (MLLMs) suffer performance degradation when encountering unfamiliar queries under distribution shifts. Existing method…
A Unified Understanding and Evaluation of Steering Methods
Shawn Im, Sharon Li
Latent space steering methods provide a practical approach to controlling large language models by applying steering vectors to intermediate activations, guiding outputs toward des…
Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approach
Changdae Oh, Zhen Fang, Shawn Im +2
Multimodal large language models (MLLMs) have shown promising capabilities but struggle under distribution shifts, where evaluation data differ from instruction tuning distribution…
Challenges and Future Directions of Data-Centric AI Alignment
Min-Hsuan Yeh, Jeffrey Wang, Xuefeng Du +4
As AI systems become increasingly capable and influential, ensuring their alignment with human values, preferences, and goals has become a critical research focus. Current alignmen…
Can DPO Learn Diverse Human Values? A Theoretical Scaling Law
Shawn Im, Sharon Li
Large language models (LLMs) have demonstrated remarkable capabilities but often struggle to align with human preferences, leading to harmful or undesirable outputs. Preference lea…
Understanding the Learning Dynamics of Alignment with Human Feedback
Shawn Im, Yixuan Li
Aligning large language models (LLMs) with human intentions has become a critical task for safely deploying models in real-world systems. While existing alignment approaches have s…
Evaluating the Utility of Model Explanations for Model Development
Shawn Im, Jacob Andreas, Yilun Zhou
One of the motivations for explainable AI is to allow humans to make better and more informed decisions regarding the use and deployment of AI models. But careful evaluations are n…