982 citations · 1.6k across the 12 of their papers we have counts for
12 papers
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Gemini Team, Petko Georgiev, Ving Ian Lei +1132
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over…
LiPO: Listwise Preference Optimization through Learning-to-Rank
Tianqi Liu, Zhen Qin, Junru Wu +9
Aligning language models (LMs) with curated human feedback is critical to control their behaviors in real-world applications. Several recent policy optimization methods, such as DP…
Statistical Rejection Sampling Improves Preference Optimization
Tianqi Liu, Yao Zhao, Rishabh Joshi +4
Improving the alignment of language models with human preferences remains an active research challenge. Previous approaches have primarily utilized Reinforcement Learning from Huma…
SLiC-HF: Sequence Likelihood Calibration with Human Feedback
Yao Zhao, Rishabh Joshi, Tianqi Liu +3
Learning from human feedback has been shown to be effective at aligning language models with human preferences. Past work has often relied on Reinforcement Learning from Human Feed…
Improving the Robustness of Summarization Models by Detecting and Removing Input Noise
Kundan Krishna, Yao Zhao, Jie Ren +4
The evaluation of abstractive summarization models typically uses test data that is identically distributed as training data. In real-world practice, documents to be summarized may…
Calibrating Sequence likelihood Improves Conditional Language Generation
Yao Zhao, Misha Khalman, Rishabh Joshi +3
Conditional language models are predominantly trained with maximum likelihood estimation (MLE), giving probability mass to sparsely observed target sequences. While MLE trained mod…
Out-of-Distribution Detection and Selective Generation for Conditional Language Models
Jie Ren, Jiaming Luo, Yao Zhao +4
Machine learning algorithms typically assume independent and identically distributed samples in training and at test time. Much work has shown that high-performing ML classifiers c…
WebRED: Effective Pretraining And Finetuning For Relation Extraction On The Web
Robert Ormandi, Mohammad Saleh, Erin Winter +1
Relation extraction is used to populate knowledge bases that are important to many applications. Prior datasets used to train relation extraction models either suffer from noisy la…
SEAL: Segment-wise Extractive-Abstractive Long-form Text Summarization
Yao Zhao, Mohammad Saleh, Peter J. Liu
Most prior work in the sequence-to-sequence paradigm focused on datasets with input sequence lengths in the hundreds of tokens due to the computational constraints of common RNN an…
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
Jingqing Zhang, Yao Zhao, Mohammad Saleh +1
Recent work pre-training Transformers with self-supervised objectives on large text corpora has shown great success when fine-tuned on downstream NLP tasks including text summariza…
Assessing The Factual Accuracy of Generated Text
Ben Goodrich, Vinay Rao, Mohammad Saleh +1
We propose a model-based metric to estimate the factual accuracy of generated text that is complementary to typical scoring schemes like ROUGE (Recall-Oriented Understudy for Gisti…
Generating Wikipedia by Summarizing Long Sequences
Peter J. Liu, Mohammad Saleh, Etienne Pot +4
We show that generating English Wikipedia articles can be approached as a multi- document summarization of source documents. We use extractive summarization to coarsely identify sa…