299 citations · 983 across the 34 of their papers we have counts for
35 papers
Beyond Pipelines: A Survey of the Paradigm Shift toward Model-Native Agentic AI
Jitao Sang, Jinlin Xiao, Jiarun Han +5
The rapid evolution of agentic AI marks a new phase in artificial intelligence, where Large Language Models (LLMs) no longer merely respond but act, reason, and adapt. This survey…
The Bias is in the Details: An Assessment of Cognitive Bias in LLMs
R. Alexander Knipper, Charles S. Knipper, Kaiqi Zhang +3
As Large Language Models (LLMs) are increasingly embedded in real-world decision-making processes, it becomes crucial to examine the extent to which they exhibit cognitive biases.…
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Gheorghe Comanici, Eric Bieber, Mike Schaekermann +3431
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our…
Seed-Free Synthetic Data Generation Framework for Instruction-Tuning LLMs: A Case Study in Thai
Parinthapat Pengpun, Can Udomcharoenchaikit, Weerayut Buaphet +1
We present a synthetic data approach for instruction-tuning large language models (LLMs) for low-resource languages in a data-efficient manner, specifically focusing on Thai. We id…
Mitigating Extrinsic Gender Bias for Bangla Classification Tasks
Sajib Kumar Saha Joy, Arman Hassan Mahy, Meherin Sultana +4
In this study, we investigate extrinsic gender bias in Bangla pretrained language models, a largely underexplored area in low-resource languages. To assess this bias, we construct…
Beyond Performance: Quantifying and Mitigating Label Bias in LLMs
Yuval Reif, Roy Schwartz
Large language models (LLMs) have shown remarkable adaptability to diverse tasks, by leveraging context prompts containing instructions, or minimal input-output examples. However,…
Holistic Safety and Responsibility Evaluations of Advanced AI Models
Laura Weidinger, Joslyn Barnhart, Jenny Brennan +16
Safety and responsibility evaluations of advanced AI models are a critical but developing field of research and practice. In the development of Google DeepMind's advanced AI models…
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Gemini Team, Petko Georgiev, Ving Ian Lei +1132
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over…
Generalized People Diversity: Learning a Human Perception-Aligned Diversity Representation for People Images
Hansa Srinivasan, Candice Schumann, Aradhana Sinha +5
Capturing the diversity of people in images is challenging: recent literature tends to focus on diversifying one or two attributes, requiring expensive attribute labels or building…
Beyond ChatBots: ExploreLLM for Structured Thoughts and Personalized Model Responses
Xiao Ma, Swaroop Mishra, Ariel Liu +6
Large language model (LLM) powered chatbots are primarily text-based today, and impose a large interactional cognitive load, especially for exploratory or sensemaking tasks such as…
Improving Few-shot Generalization of Safety Classifiers via Data Augmented Parameter-Efficient Fine-Tuning
Ananth Balashankar, Xiao Ma, Aradhana Sinha +4
As large language models (LLMs) are widely adopted, new safety issues and policies emerge, to which existing safety classifiers do not generalize well. If we have only observed a f…
Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Voting
Preethi Lahoti, Nicholas Blumm, Xiao Ma +8
A crucial challenge for generative large language models (LLMs) is diversity: when a user's prompt is under-specified, models may follow implicit assumptions while generating a res…
Controlled Decoding from Language Models
Sidharth Mudgal, Jong Lee, Harish Ganapathy +10
KL-regularized reinforcement learning (RL) is a popular alignment framework to control the language model responses towards high reward outcomes. We pose a tokenwise RL objective a…
Break it, Imitate it, Fix it: Robustness by Generating Human-Like Attacks
Aradhana Sinha, Ananth Balashankar, Ahmad Beirami +3
Real-world natural language processing systems need to be robust to human adversaries. Collecting examples of human adversaries for training is an effective but expensive solution.…
Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering
Han Zhou, Xingchen Wan, Lev Proleev +4
Prompting and in-context learning (ICL) have become efficient learning paradigms for large language models (LLMs). However, LLMs suffer from prompt brittleness and various bias fac…
Towards A Scalable Solution for Improving Multi-Group Fairness in Compositional Classification
James Atwood, Tina Tian, Ben Packer +5
Despite the rich literature on machine learning fairness, relatively little attention has been paid to remediating complex systems, where the final prediction is the combination of…
Let's Do a Thought Experiment: Using Counterfactuals to Improve Moral Reasoning
Xiao Ma, Swaroop Mishra, Ahmad Beirami +2
Language models still struggle on moral reasoning, despite their impressive performance in many other tasks. In particular, the Moral Scenarios task in MMLU (Multi-task Language Un…
Improving Classifier Robustness through Active Generation of Pairwise Counterfactuals
Ananth Balashankar, Xuezhi Wang, Yao Qin +5
Counterfactual Data Augmentation (CDA) is a commonly used technique for improving robustness in natural language classifiers. However, one fundamental challenge is how to discover…
A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures
Amanda Baughan, Allison Mercurio, Ariel Liu +3
Despite huge gains in performance in natural language understanding via large language models in recent years, voice assistants still often fail to meet user expectations. In this…
Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers
Jieyu Zhao, Xuezhi Wang, Yao Qin +2
Large pre-trained language models have shown remarkable performance over the past few years. These models, however, sometimes learn superficial features from the dataset and cannot…
A Human-ML Collaboration Framework for Improving Video Content Reviews
Meghana Deodhar, Xiao Ma, Yixin Cai +3
We deal with the problem of localized in-video taxonomic human annotation in the video content moderation domain, where the goal is to identify video segments that violate granular…
Simpson's Paradox in Recommender Fairness: Reconciling differences between per-user and aggregated evaluations
Flavien Prost, Ben Packer, Jilin Chen +9
There has been a flurry of research in recent years on notions of fairness in ranking and recommender systems, particularly on how to evaluate if a recommender allocates exposure e…
Flexible text generation for counterfactual fairness probing
Zee Fryer, Vera Axelrod, Ben Packer +3
A common approach for testing fairness issues in text-based classifiers is through the use of counterfactuals: does the classifier output change if a sensitive attribute in the inp…
Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning
Yuyan Wang, Xuezhi Wang, Alex Beutel +3
As multi-task models gain popularity in a wider range of machine learning applications, it is becoming increasingly important for practitioners to understand the fairness implicati…