Publications (16)
Low-Rank Adaptation for Multilingual Summarization: An Empirical Study
Chenxi Whitehouse, Fantine Huot, Jasmijn Bastings +3
Although the advancements of pre-trained Large Language Models have significantly accelerated recent progress in NLP, their ever-increasing size poses significant challenges for co…
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Gheorghe Comanici, Eric Bieber, Mike Schaekermann +3431
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our…
Gemini: A Family of Highly Capable Multimodal Models
Gemini Team, Rohan Anil, Sebastian Borgeaud +1340
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consist…
Unsupervised POS Induction with Word Embeddings
Chu-Cheng Lin, Waleed Ammar, Chris Dyer +1
Unsupervised word embeddings have been shown to be valuable as features in supervised learning problems; however, their role in unsupervised problems has been less thoroughly explo…
SEVerA: Verified Synthesis of Self-Evolving Agents
Debangshu Banerjee, Changming Xu, Eugene Ie +4
Recent advances have shown the effectiveness of self-evolving LLM agents on tasks such as program repair and scientific discovery. In this paradigm, a planner LLM synthesizes an ag…
Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection
Yun Zhu, Jia-Chen Gu, Caitlin Sikora +8
Large language models (LLMs) augmented with retrieval exhibit robust performance and extensive versatility by incorporating external contexts. However, the input length grows linea…
Type-Compliant Adaptation Cascades: Adapting Programmatic LM Workflows to Data
Chu-Cheng Lin, Daiyi Peng, Yifeng Lu +2
Reliably composing Large Language Models (LLMs) for complex, multi-step workflows remains a significant challenge. The dominant paradigm -- optimizing discrete prompts in a pipelin…
Limitations of Autoregressive Models and Their Alternatives
Chu-Cheng Lin, Aaron Jaech, Xin Li +2
Standard autoregressive language models perform only polynomial-time computation to compute the probability of the next symbol. While this is attractive, it means they cannot model…
Inducing Generalization across Languages and Tasks using Featurized Low-Rank Mixtures
Chu-Cheng Lin, Xinyi Wang, Jonathan H. Clark +4
Adapting pretrained large language models (LLMs) to various downstream tasks in tens or hundreds of human languages is computationally expensive. Parameter-efficient fine-tuning (P…
SiRA: Sparse Mixture of Low Rank Adaptation
Yun Zhu, Nevan Wichers, Chu-Cheng Lin +8
Parameter Efficient Tuning has been an prominent approach to adapt the Large Language Model to downstream tasks. Most previous works considers adding the dense trainable parameters…
UT5: Pretraining Non autoregressive T5 with unrolled denoising
Mahmoud G. Salem, Jiayu Ye, Chu-Cheng Lin +1
Recent advances in Transformer-based Large Language Models have made great strides in natural language generation. However, to decode K tokens, an autoregressive model needs K sequ…
Actionable Email Intent Modeling with Reparametrized RNNs
Chu-Cheng Lin, Dongyeop Kang, Michael Gamon +3
Emails in the workplace are often intentional calls to action for its recipients. We propose to annotate these emails for what action its recipient will take. We argue that our app…
How Fast Should a Model Commit to Supervision? Training Reasoning Models on the Tsallis Loss Continuum
Chu-Cheng Lin, Eugene Ie
SFT-then-RLVR is widely used for post-training reasoning models, but why this specific ordering, and why RLVR-only stalls at cold start, have lacked a unifying theoretical account.…
Recurrent-Neural-Network for Language Detection on Twitter Code-Switching Corpus
Joseph Chee Chang, Chu-Cheng Lin
Mixed language data is one of the difficult yet less explored domains of natural language processing. Most research in fields like machine translation or sentiment analysis assume…
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Gemini Team, Petko Georgiev, Ving Ian Lei +1132
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over…
Neural Particle Smoothing for Sampling from Conditional Sequence Models
Chu-Cheng Lin, Jason Eisner
We introduce neural particle smoothing, a sequential Monte Carlo method for sampling annotations of an input string from a given probability model. In contrast to conventional part…