345 citations · 487 across the 9 of their papers we have counts for
10 papers
Reading the Room: Foundations, Design, and Challenges of Normative Competence in LLMs
Andrea Wynn, Harsh Satija, Nathan Baek +5
Human communities are governed by normative systems: shared standards that produce \textit{norms} dictating acceptable behavior, enforced through community sanctioning. Aligning in…
Self-Compacting Language Model Agents
Tianjian Li, Jingyu Zhang, William Jurayj +5
Long agent traces composed of chains of thought and tool calls accumulate stale content that anchor subsequent generations, and eventually outgrow the context window. Existing scaf…
The Myth of Expert Specialization in MoEs: Why Routing Reflects Geometry, Not Necessarily Domain Expertise
Xi Wang, Soufiane Hayou, Eric Nalisnick
Mixture of Experts (MoEs) are now ubiquitous in large language models, yet the mechanisms behind their "expert specialization" remain poorly understood. We show that, since MoE rou…
A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models
Theo X. Olausson, Metod Jazbec, Xi Wang +4
Much work has been done on designing fast and accurate sampling for diffusion language models (dLLMs). However, these efforts have largely focused on the tradeoff between speed and…
Scalable Generative Modeling of Weighted Graphs
Richard Williams, Eric Nalisnick, Andrew Holbrook
Weighted graphs are ubiquitous throughout biology, chemistry, and the social sciences, motivating the development of generative models for abstract weighted graph data using deep n…
Are vision language models robust to uncertain inputs?
Xi Wang, Eric Nalisnick
Robustness against uncertain and ambiguous inputs is a critical challenge for deep learning models. While recent advancements in large scale vision language models (VLMs, e.g. GPT4…
Normalizing Flows for Probabilistic Modeling and Inference
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende +2
Normalizing flows provide a general mechanism for defining expressive probability distributions, only requiring the specification of a (usually simple) base distribution and a seri…
Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh +1
Recent work has shown that deep generative models can assign higher likelihood to out-of-distribution data sets than to their training data (Nalisnick et al., 2019; Choi et al., 20…
Hybrid Models with Deep and Invertible Features
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh +2
We propose a neural hybrid model consisting of a linear model defined on a set of features computed by a deep, invertible transformation (i.e. a normalizing flow). An attractive pr…
Do Deep Generative Models Know What They Don't Know?
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh +2
A neural network deployed in the wild may be asked to make predictions for inputs that were drawn from a different distribution than that of the training data. A plethora of work h…