297 citations · 585 across the 14 of their papers we have counts for
17 papers
Gemma 2: Improving Open Language Models at a Practical Size
Gemma Team, Morgane Riviere, Shreya Pathak +195
In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In th…
From RAG to RICHES: Retrieval Interlaced with Sequence Generation
Palak Jain, Livio Baldini Soares, Tom Kwiatkowski
We present RICHES, a novel approach that interleaves retrieval with sequence generation tasks. RICHES offers an alternative to conventional RAG systems by eliminating the need for…
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Gemini Team, Petko Georgiev, Ving Ian Lei +1132
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over…
1-PAGER: One Pass Answer Generation and Evidence Retrieval
Palak Jain, Livio Baldini Soares, Tom Kwiatkowski
We present 1-Pager the first system that answers a question and retrieves evidence using a single Transformer-based model and decoding process. 1-Pager incrementally partitions the…
Calibrating Likelihoods towards Consistency in Summarization Models
Polina Zablotskaia, Misha Khalman, Rishabh Joshi +4
Despite the recent advances in abstractive text summarization, current summarization models still suffer from generating factually inconsistent summaries, reducing their utility fo…
NAIL: Lexical Retrieval Indices with Efficient Non-Autoregressive Decoders
Livio Baldini Soares, Daniel Gillick, Jeremy R. Cole +1
Neural document rerankers are extremely effective in terms of accuracy. However, the best models require dedicated hardware for serving, which is costly and often not feasible. To…
Evaluating and Modeling Attribution for Cross-Lingual Question Answering
Benjamin Muller, John Wieting, Jonathan H. Clark +6
Trustworthy answer content is abundant in many high-resource languages and is instantly accessible through question answering systems, yet this content can be hard to access for th…
Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
Bernd Bohnet, Vinh Q. Tran, Pat Verga +19
Large language models (LLMs) have shown impressive results while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in info…
Scaling Up Models and Data with and
Adam Roberts, Hyung Won Chung, Anselm Levskaya +40
Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves. Scaling can…
Evaluating Explanations: How much do explanations from the teacher aid students?
Danish Pruthi, Rachit Bansal, Bhuwan Dhingra +5
While many methods purport to explain predictions by highlighting salient features, what aims these explanations serve and how they ought to be evaluated often go unstated. In this…
QED: A Framework and Dataset for Explanations in Question Answering
Matthew Lamm, Jennimaria Palomaki, Chris Alberti +4
A question answering system that in addition to providing an answer provides an explanation of the reasoning that leads to that answer has potential advantages in terms of debuggab…
Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge
Pat Verga, Haitian Sun, Livio Baldini Soares +1
Massive language models are the core of modern NLP modeling and have been shown to encode impressive amounts of commonsense and factual information. However, that knowledge exists…
Empirical Evaluation of Pretraining Strategies for Supervised Entity Linking
Thibault Févry, Nicholas FitzGerald, Livio Baldini Soares +1
In this work, we present an entity linking model which combines a Transformer architecture with large scale pretraining from Wikipedia links. Our model achieves the state-of-the-ar…
New Protocols and Negative Results for Textual Entailment Data Collection
Samuel R. Bowman, Jennimaria Palomaki, Livio Baldini Soares +1
Natural language inference (NLI) data has proven useful in benchmarking and, especially, as pretraining data for tasks requiring language understanding. However, the crowdsourcing…
Entities as Experts: Sparse Memory Access with Entity Supervision
Thibault Févry, Livio Baldini Soares, Nicholas FitzGerald +2
We focus on the problem of capturing declarative knowledge about entities in the learned parameters of a language model. We introduce a new model - Entities as Experts (EAE) - that…
Learning Cross-Context Entity Representations from Text
Jeffrey Ling, Nicholas FitzGerald, Zifei Shan +4
Language modeling tasks, in which words, or word-pieces, are predicted on the basis of a local context, have been very effective for learning word embeddings and context dependent…
Matching the Blanks: Distributional Similarity for Relation Learning
Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling +1
General purpose relation extractors, which can model arbitrary relations, are a core aspiration in information extraction. Efforts have been made to build general purpose extractor…