53 citations · 189 across the 12 of their papers we have counts for
12 papers
AttuneBench: A Conversation-Based Benchmark for LLM Emotional Intelligence
Kate M. Lubrano, Faisal Sayed, Ankita Rathod +4
Emotional intelligence (EI), the ability to perceive, understand, and respond appropriately to others' emotional states, is central to human communication, and increasingly importa…
PitchBench: Measuring Pitch Hearing in Audio-Language Models
Milan Liessens Dujardin, Song-Ze Yu, Craver Corbyn Thomas-Smith +2
Audio-language models (ALMs) are increasingly used in real-world applications that require understanding music, from music tutoring and transcription to captioning, recommendation…
PostTrainBench: Can LLM Agents Automate LLM Post-Training?
Ben Rank, Hardik Bhatnagar, Ameya Prabhu +4
AI agents have become surprisingly proficient at software engineering over the past year, largely due to improvements in reasoning capabilities. This raises a deeper question: can…
Evaluating and Mitigating Discrimination in Language Model Decisions
Alex Tamkin, Amanda Askell, Liane Lovitt +6
As language models (LMs) advance, interest is growing in applying them to high-stakes societal decisions, such as determining financing or housing eligibility. However, their poten…
Specific versus General Principles for Constitutional AI
Sandipan Kundu, Yuntao Bai, Saurav Kadavath +33
Human feedback can prevent overtly harmful utterances in conversational models, but may not automatically mitigate subtle problematic behaviors such as a stated desire for self-pre…
Studying Large Language Model Generalization with Influence Functions
Roger Grosse, Juhan Bae, Cem Anil +14
When trying to gain better visibility into a machine learning model in order to understand and mitigate the associated risks, a potentially valuable source of evidence is: which tr…
Question Decomposition Improves the Faithfulness of Model-Generated Reasoning
Ansh Radhakrishnan, Karina Nguyen, Anna Chen +21
As large language models (LLMs) perform more difficult tasks, it becomes harder to verify the correctness and safety of their behavior. One approach to help with this issue is to p…
Measuring Faithfulness in Chain-of-Thought Reasoning
Tamera Lanham, Anna Chen, Ansh Radhakrishnan +27
Large language models (LLMs) perform better when they produce step-by-step, "Chain-of-Thought" (CoT) reasoning before answering a question, but it is unclear if the stated reasonin…
Vision Transformers for Mobile Applications: A Short Survey
Nahid Alam, Steven Kolawole, Simardeep Sethi +2
Vision Transformers (ViTs) have demonstrated state-of-the-art performance on many Computer Vision Tasks. Unfortunately, deploying these large-scale ViTs is resource-consuming and i…
FAIR-Ensemble: When Fairness Naturally Emerges From Deep Ensembling
Wei-Yin Ko, Daniel D'souza, Karina Nguyen +2
Ensembling multiple Deep Neural Networks (DNNs) is a simple and effective way to improve top-line metrics and to outperform a larger single model. In this work, we go beyond top-li…
The Capacity for Moral Self-Correction in Large Language Models
Deep Ganguli, Amanda Askell, Nicholas Schiefer +46
We test the hypothesis that language models trained with reinforcement learning from human feedback (RLHF) have the capability to "morally self-correct" -- to avoid producing harmf…
Discovering Language Model Behaviors with Model-Written Evaluations
Ethan Perez, Sam Ringer, Kamilė Lukošiūtė +60
As language models (LMs) scale, they develop many novel behaviors, good and bad, exacerbating the need to evaluate how they behave. Prior work creates evaluations with crowdwork (w…