7 papers
ProEval: Proactive Failure Discovery and Efficient Performance Estimation for Generative AI Evaluation
Yizheng Huang, Wenjun Zeng, Aditi Kumaresan +1
Evaluating generative AI models is increasingly resource-intensive due to slow inference, expensive raters, and a rapidly growing landscape of models and benchmarks. We propose Pro…
Can Vision-Language-Action Models Learn from Real-World Data Continually without Forgetting?
Jiarun Zhu, Yijun Hong, Xiaoquan Sun +7
Vision-Language-Action (VLA) models provide a promising foundation for general-purpose robotics, yet their real-world deployment demands the ability to continually acquire new skil…
Interpreting and Controlling Model Behavior via Constitutions for Atomic Concept Edits
Neha Kalibhat, Zi Wang, Prasoon Bajpai +4
We introduce a black-box interpretability framework that learns a verifiable constitution: a natural language summary of how changes to a prompt affect a model's specific behavior,…
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Gheorghe Comanici, Eric Bieber, Mike Schaekermann +3431
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our…
Proactive Agents for Multi-Turn Text-to-Image Generation Under Uncertainty
Meera Hahn, Wenjun Zeng, Nithish Kannen +4
User prompts for generative AI models are often underspecified, leading to a misalignment between the user intent and models' understanding. As a result, users commonly have to pai…
X-Guard: Multilingual Guard Agent for Content Moderation
Bibek Upadhayay, Vahid Behzadan, Ph. D
Large Language Models (LLMs) have rapidly become integral to numerous applications in critical domains where reliability is paramount. Despite significant advances in safety framew…