9 papers
Co-LMLM: Continuous-Query Limited Memory Language Models
Yair Feldman, Linxi Zhao, Nathan Godey +5
Limited memory language models (LMLMs) externalize factual knowledge during pretraining to a knowledge base (KB), rather than memorizing it in their weights. During generation, the…
Self-Augmenting Retrieval for Diffusion Language Models
Paul Jünger, Justin Lovelace, Linxi Zhao +2
Discrete diffusion language models generate text by iteratively denoising an entire response in parallel. At each step, they predict tentative tokens for every masked position, com…
Music Transcription with (Almost) No Supervision
Saebyeol Shin, Chao Wan, Zhenzhen Liu +4
Competitive music transcription models require large amounts of paired audio-score data, which is scarce due to collection costs, alignment difficulty, and copyright restrictions.…
Prescriptive Scaling Laws for Data Constrained Training
Justin Lovelace, Christian Belardi, Srivatsa Kundurthy +2
Training compute is increasingly outpacing the availability of high-quality data. This shifts the central challenge from optimal compute allocation to extracting maximum value from…
Adaptive Moments are Surprisingly Effective for Plug-and-Play Diffusion Sampling
Christian Belardi, Justin Lovelace, Kilian Q. Weinberger +1
Guided diffusion sampling relies on approximating often intractable likelihood scores, which introduces significant noise into the sampling dynamics. We propose using adaptive mome…
Electron affinity difference distributions guide the discovery of the superconductor PtPbBi
Omri Lesser, Yanjun Liu, Natalie Maus +11
Predicting the superconducting transition temperature () from crystal structure and composition remains a central challenge in condensed-matter physics, reflecting the absence…