6 papers
Understanding Annotator Safety Policy with Interpretability
Alex Oesterling, Donghao Ren, Yannick Assogba +4
Safety policies define what constitutes safe and unsafe AI outputs, guiding data annotation and model development. However, annotation disagreement is pervasive and can stem from m…
Semantic Regexes: Auto-Interpreting LLM Features with a Structured Language
Angie Boggust, Donghao Ren, Yannick Assogba +3
Automated interpretability aims to translate large language model (LLM) features into human understandable descriptions. However, natural language feature descriptions can be vague…
Constructive Circuit Amplification: Improving Math Reasoning in LLMs via Targeted Sub-Network Updates
Nikhil Prakash, Donghao Ren, Dominik Moritz +1
Prior studies investigating the internal workings of LLMs have uncovered sparse subnetworks, often referred to as circuits, that are responsible for performing specific tasks. Addi…
EncQA: Benchmarking Vision-Language Models on Visual Encodings for Charts
Kushin Mukherjee, Donghao Ren, Dominik Moritz +1
Multimodal vision-language models (VLMs) continue to achieve ever-improving scores on chart understanding benchmarks. Yet, we find that this progress does not fully capture the bre…
Evaluating Long Range Dependency Handling in Code Generation LLMs
Yannick Assogba, Donghao Ren
As language models support larger and larger context sizes, evaluating their ability to make effective use of that context becomes increasingly important. We analyze the ability of…
Exploring Empty Spaces: Human-in-the-Loop Data Augmentation
Catherine Yeh, Donghao Ren, Yannick Assogba +2
Data augmentation is crucial to make machine learning models more robust and safe. However, augmenting data can be challenging as it requires generating diverse data points to rigo…