4 papers
Addressing Over-Refusal in LLMs with Competing Rewards
Taeyoun Kim, Aviral Kumar
Safety training on language models often induces over-refusal: improved safety on harmful prompts at the cost of increased refusal on harmless ones. Though this trade-off can be mi…
Reasoning as an Adaptive Defense for Safety
Taeyoun Kim, Fahim Tajwar, Aditi Raghunathan +1
Reasoning methods that adaptively allocate test-time compute have advanced LLM performance on easy to verify domains such as math and code. In this work, we study how to utilize th…
Mitigating Bias in RAG: Controlling the Embedder
Taeyoun Kim, Jacob Springer, Aditi Raghunathan +1
In retrieval augmented generation (RAG) systems, each individual component -- the LLM, embedder, and corpus -- could introduce biases in the form of skews towards outputting certai…
Predicting the Performance of Foundation Models via Agreement-on-the-Line
Rahul Saxena, Taeyoun Kim, Aman Mehra +3
Estimating the out-of-distribution performance in regimes where labels are scarce is critical to safely deploy foundation models. Recently, it was shown that ensembles of neural ne…