118 citations · 300 across the 13 of their papers we have counts for
13 papers
Olmo 3
Team Olmo, :, Allyson Ettinger +66
We introduce Olmo 3, a family of state-of-the-art, fully-open language models at the 7B and 32B parameter scales. Olmo 3 model construction targets long-context reasoning, function…
Inverse Scaling: When Bigger Isn't Better
Ian R. McKenzie, Alexander Lyzhov, Michael Pieler +24
Work on scaling laws has found that large language models (LMs) show predictable improvements to overall loss with increased scale (model size, training data, and compute). Here, w…
How Language Model Hallucinations Can Snowball
Muru Zhang, Ofir Press, William Merrill +2
A major risk of using language models in practical applications is their tendency to hallucinate incorrect statements. Hallucinations are often attributed to knowledge gaps in LMs,…
We're Afraid Language Models Aren't Modeling Ambiguity
Alisa Liu, Zhaofeng Wu, Julian Michael +6
Ambiguity is an intrinsic feature of natural language. Managing ambiguity is a key part of human language understanding, allowing us to anticipate misunderstanding as communicators…
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra +4
Large "instruction-tuned" language models (i.e., finetuned to respond to instructions) have demonstrated a remarkable ability to generalize zero-shot to new tasks. Nevertheless, th…
Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Experts
Skyler Hallinan, Alisa Liu, Yejin Choi +1
Text detoxification has the potential to mitigate the harms of toxicity by rephrasing text to remove offensive meaning, but subtle toxicity remains challenging to tackle. We introd…
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation
Alisa Liu, Swabha Swayamdipta, Noah A. Smith +1
A recurring challenge of crowdsourcing NLP datasets at scale is that human writers often rely on repetitive patterns when crafting examples, leading to a lack of linguistic diversi…
Generated Knowledge Prompting for Commonsense Reasoning
Jiacheng Liu, Alisa Liu, Ximing Lu +5
It remains an open question whether incorporating external knowledge benefits commonsense reasoning while maintaining the flexibility of pretrained sequence models. To investigate…
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts
Alisa Liu, Maarten Sap, Ximing Lu +4
Despite recent advances in natural language generation, it remains challenging to control attributes of generated text. We propose DExperts: Decoding-time Experts, a decoding-time…
Incorporating Music Knowledge in Continual Dataset Augmentation for Music Generation
Alisa Liu, Alexander Fang, Gaëtan Hadjeres +2
Deep learning has rapidly become the state-of-the-art approach for music generation. However, training a deep model typically requires a large training set, which is often not avai…
Bach or Mock? A Grading Function for Chorales in the Style of J.S. Bach
Alexander Fang, Alisa Liu, Prem Seetharaman +1
Deep generative systems that learn probabilistic models from a corpus of existing music do not explicitly encode knowledge of a musical style, compared to traditional rule-based sy…
Model selection for deep audio source separation via clustering analysis
Alisa Liu, Prem Seetharaman, Bryan Pardo
Audio source separation is the process of separating a mixture (e.g. a pop band recording) into isolated sounds from individual sources (e.g. just the lead vocals). Deep learning m…
Multi-sense Definition Modeling using Word Sense Decompositions
Ruimin Zhu, Thanapon Noraset, Alisa Liu +2
Word embeddings capture syntactic and semantic information about words. Definition modeling aims to make the semantic content in each embedding explicit, by outputting a natural la…