18 citations · 27 across the 15 of their papers we have counts for
16 papers
How I learned to stop worrying and love StopGrads: Stationarity, Convergence, and a case study on Flow Map Learning
Max W. Shen, Mark Goldstein, Zichu Wang +2
Stopgrads are widely used in training machine learning models, but stopgrads can alter the gradient, stationary points and convergence guarantees of the original objective, which c…
Flow Map Learning via Nongradient Vector Flow
Mark Goldstein, Anshuk Uppal, Raghav Singhal +2
Diffusion and flow-based models benefit from simple regression losses, but inference incurs significant overhead because sampling requires integration. Consistency models address t…
What LLMs explain is not what they believe: Evaluating explanation sufficiency under models' own input beliefs
Nhi Nguyen, Shauli Ravfogel, Rajesh Ranganath
Large language models (LLMs) are increasingly deployed in high-stakes domains, where free-text explanations such as chain-of-thought and post-hoc rationales are used to justify mod…
Estimating Tail Risks in Language Model Output Distributions
Rico Angell, Raghav Singhal, Zachary Horvitz +4
Language models are increasingly capable and are being rapidly deployed on a population-level scale. As a result, the safety of these models is increasingly high-stakes. Fortunatel…
To Use or not to Use Muon: How Simplicity Bias in Optimizers Matters
Sara Dragutinović, Yedi Zhang, Rajesh Ranganath
While Adam has long been the ubiquitous default optimizer for deep neural networks, Muon has recently seen rapid adoption due to its superior training speed. Although much of the l…
Controllably Efficient Language Models
Jatin Prakash, Aahlad Puli, Rajesh Ranganath
The substantial inference costs of attention in transformers motivated the development of efficient sequence mixers: namely sparse and sliding window attention, convolutions and li…
KL-Regularized Reinforcement Learning is Designed to Mode Collapse
Anthony GX-Chen, Jatin Prakash, Jeff Guo +2
It is commonly believed that optimizing the reverse KL divergence results in "mode seeking", while optimizing forward KL results in "mass covering", with the latter being preferred…
Three Forms of Stochastic Injection for Improved Distribution-to-Distribution Generative Modeling
Shiye Su, Yuhui Zhang, Linqi Zhou +2
Modeling transformations between arbitrary data distributions is a fundamental scientific challenge, arising in applications like drug discovery and evolutionary simulation. While…
Rethinking Reasoning with MDLMs: Early Exits, Post-hoc Reasoning, and Beyond
Zachary Horvitz, Raghav Singhal, Hao Zou +4
The reasoning paradigm, where language models reason before answering, has enabled breakthroughs on tasks such as mathematical problem-solving. While current tooling for reasoning…
A General Framework for Inference-time Scaling and Steering of Diffusion Models
Raghav Singhal, Zachary Horvitz, Ryan Teehan +4
Diffusion models produce impressive results in modalities ranging from images and video to protein design and text. However, generating samples with user-specified properties remai…
Explanations that reveal all through the definition of encoding
Aahlad Puli, Nhi Nguyen, Rajesh Ranganath
Feature attributions attempt to highlight what inputs drive predictive power. Good attributions or explanations are thus those that produce inputs that retain this predictive power…
Contrasting with Symile: Simple Model-Agnostic Representation Learning for Unlimited Modalities
Adriel Saporta, Aahlad Puli, Mark Goldstein +1
Contrastive learning methods, such as CLIP, leverage naturally paired data-for example, images and their corresponding text captions-to learn general representations that transfer…
What's the score? Automated Denoising Score Matching for Nonlinear Diffusions
Raghav Singhal, Mark Goldstein, Rajesh Ranganath
Reversing a diffusion process by learning its score forms the heart of diffusion-based generative modeling and for estimating properties of scientific systems. The diffusion proces…
Adaptive Sampling of k-Space in Magnetic Resonance for Rapid Pathology Prediction
Chen-Yu Yen, Raghav Singhal, Umang Sharma +3
Magnetic Resonance (MR) imaging, despite its proven diagnostic utility, remains an inaccessible imaging modality for disease surveillance at the population level. A major factor re…
On the Challenges and Opportunities in Generative AI
Laura Manduchi, Clara Meister, Kushagra Pandey +23
The field of deep generative modeling has grown rapidly in the last few years. With the availability of massive amounts of training data coupled with advances in scalable unsupervi…
Robust Anomaly Detection for Particle Physics Using Multi-Background Representation Learning
Abhijith Gandrakota, Lily Zhang, Aahlad Puli +4
Anomaly, or out-of-distribution, detection is a promising tool for aiding discoveries of new particles or processes in particle physics. In this work, we identify and address two o…