40 citations · 127 across the 15 of their papers we have counts for
17 papers
Learning to Discover at Test Time
Mert Yuksekgonul, Daniel Koceja, Xinhao Li +8
How can we use AI to discover a new state of the art for a scientific problem? Prior work in test-time scaling, such as AlphaEvolve, performs search by prompting a frozen LLM. We p…
Sparse Reward Subsystem in Large Language Models
Guowei Xu, Mert Yuksekgonul, James Zou
Recent studies show that LLM hidden states encode reward-related information, such as answer correctness and model confidence. However, existing approaches typically fit black-box…
Inefficiencies of Meta Agents for Agent Design
Batu El, Mert Yuksekgonul, James Zou
Recent works began to automate the design of agentic systems using meta-agents that propose and iteratively refine new agent architectures. In this paper, we examine three key chal…
Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025
Nitya Thakkar, Mert Yuksekgonul, Jake Silberg +6
Peer review at AI conferences is stressed by rapidly rising submission volumes, leading to deteriorating review quality and increased author dissatisfaction. To address these issue…
SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning
Wanjia Zhao, Mert Yuksekgonul, Shirley Wu +1
Multi-agent AI systems powered by large language models (LLMs) are increasingly applied to solve complex tasks. However, these systems often rely on fragile, manually designed prom…
TextGrad: Automatic "Differentiation" via Text
Mert Yuksekgonul, Federico Bianchi, Joseph Boen +4
AI is undergoing a paradigm shift, with breakthroughs achieved by systems orchestrating multiple large language models (LLMs) and other complex components. As a result, developing…
ChatGPT Exhibits Gender and Racial Biases in Acute Coronary Syndrome Management
Angela Zhang, Mert Yuksekgonul, Joshua Guild +2
Recent breakthroughs in large language models (LLMs) have led to their rapid dissemination and widespread use. One early application has been to medicine, where LLMs have been inve…
KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval
Marah I Abdin, Suriya Gunasekar, Varun Chandrasekaran +5
We study the ability of state-of-the art models to answer constraint satisfaction queries for information retrieval (e.g., 'a list of ice cream shops in San Diego'). In the past, s…
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
Mert Yuksekgonul, Varun Chandrasekaran, Erik Jones +5
We investigate the internal behavior of Transformer-based Large Language Models (LLMs) when they generate factually incorrect text. We propose modeling factual queries as constrain…
Discover and Cure: Concept-aware Mitigation of Spurious Correlation
Shirley Wu, Mert Yuksekgonul, Linjun Zhang +1
Deep neural networks often rely on spurious correlations to make predictions, which hinders generalization beyond training environments. For instance, models that associate cats wi…
Beyond Confidence: Reliable Models Should Also Consider Atypicality
Mert Yuksekgonul, Linjun Zhang, James Zou +1
While most machine learning models can provide confidence in their predictions, confidence is insufficient to understand a prediction's reliability. For instance, the model may hav…
GPT detectors are biased against non-native English writers
Weixin Liang, Mert Yuksekgonul, Yining Mao +2
The rapid adoption of generative language models has brought about substantial advancements in digital communication, while simultaneously raising concerns regarding the potential…
SkinCon: A skin disease dataset densely annotated by domain experts for fine-grained model debugging and analysis
Roxana Daneshjou, Mert Yuksekgonul, Zhuo Ran Cai +2
For the deployment of artificial intelligence (AI) in high-risk settings, such as healthcare, methods that provide interpretability/explainability or allow fine-grained error analy…
Post-hoc Concept Bottleneck Models
Mert Yuksekgonul, Maggie Wang, James Zou
Concept Bottleneck Models (CBMs) map the inputs onto a set of interpretable concepts (``the bottleneck'') and use the concepts to make predictions. A concept bottleneck enhances in…
Meaningfully Debugging Model Mistakes using Conceptual Counterfactual Explanations
Abubakar Abid, Mert Yuksekgonul, James Zou
Understanding and explaining the mistakes made by trained models is critical to many machine learning objectives, such as improving robustness, addressing concept drift, and mitiga…
Learning Maximally Predictive Prototypes in Multiple Instance Learning
Mert Yuksekgonul, Ozgur Emre Sivrikaya, Mustafa Gokce Baydogan
In this work, we propose a simple model that provides permutation invariant maximally predictive prototype generator from a given dataset, which leads to interpretability of the so…
Pretraining boosts out-of-domain robustness for pose estimation
Alexander Mathis, Thomas Biasi, Steffen Schneider +4
Neural networks are highly effective tools for pose estimation. However, as in other computer vision tasks, robustness to out-of-domain data remains a challenge, especially for sma…