Publications (13)
How Low Can We Go: Trading Memory for Error in Low-Precision Training
Chengrun Yang, Ziyang Wu, Jerry Chee +2
Low-precision arithmetic trains deep learning models using less energy, less memory and less time. However, we pay a price for the savings: lower precision may yield larger round-o…
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Gheorghe Comanici, Eric Bieber, Mike Schaekermann +3431
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our…
Large Language Models as Optimizers
Chengrun Yang, Xuezhi Wang, Yifeng Lu +4
Optimization is ubiquitous. While derivative-based algorithms have been powerful tools for various problems, the absence of gradient imposes challenges on many real-world applicati…
Dr Genre: Reinforcement Learning from Decoupled LLM Feedback for Generic Text Rewriting
Yufei Li, John Nham, Ganesh Jawahar +7
Generic text rewriting is a prevalent large language model (LLM) application that covers diverse real-world tasks, such as style transfer, fact correction, and email editing. These…
Euclidean-Norm-Induced Schatten-p Quasi-Norm Regularization for Low-Rank Tensor Completion and Tensor Robust Principal Component Analysis
Jicong Fan, Lijun Ding, Chengrun Yang +2
The nuclear norm and Schatten- quasi-norm are popular rank proxies in low-rank matrix recovery. However, computing the nuclear norm or Schatten- quasi-norm of a tensor is har…
Spectral Frank-Wolfe Algorithm: Strict Complementarity and Linear Convergence
Lijun Ding, Yingjie Fei, Qiantong Xu +1
We develop a novel variant of the classical Frank-Wolfe algorithm, which we call spectral Frank-Wolfe, for convex optimization over a spectrahedron. The spectral Frank-Wolfe algori…
Long-form factuality in large language models
Jerry Wei, Chengrun Yang, Xinying Song +9
Large language models (LLMs) often generate content that contains factual errors when responding to fact-seeking prompts on open-ended topics. To benchmark a model's long-form fact…
OBOE: Collaborative Filtering for AutoML Model Selection
Chengrun Yang, Yuji Akimoto, Dae Won Kim +1
Algorithm selection and hyperparameter tuning remain two of the most challenging tasks in machine learning. Automated machine learning (AutoML) seeks to automate these tasks to ena…
Robust Non-Linear Matrix Factorization for Dictionary Learning, Denoising, and Clustering
Jicong Fan, Chengrun Yang, Madeleine Udell
Low dimensional nonlinear structure abounds in datasets across computer vision and machine learning. Kernelized matrix factorization techniques have recently been proposed to learn…
Efficient AutoML Pipeline Search with Matrix and Tensor Factorization
Chengrun Yang, Jicong Fan, Ziyang Wu +1
Data scientists seeking a good supervised learning model on a new dataset have many choices to make: they must preprocess the data, select features, possibly reduce the dimension,…
SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scaling
Jiefeng Chen, Jie Ren, Xinyun Chen +4
Recent advancements in Large Language Models (LLMs) have created new opportunities to enhance performance on complex reasoning tasks by leveraging test-time computation. However, e…
TabNAS: Rejection Sampling for Neural Architecture Search on Tabular Datasets
Chengrun Yang, Gabriel Bender, Hanxiao Liu +5
The best neural architecture for a given machine learning problem depends on many factors: not only the complexity and structure of the dataset, but also on resource constraints in…
TenIPS: Inverse Propensity Sampling for Tensor Completion
Chengrun Yang, Lijun Ding, Ziyang Wu +1
Tensors are widely used to represent multiway arrays of data. The recovery of missing entries in a tensor has been extensively studied, generally under the assumption that entries…