8 papers
Weighted Ensemble Sampling for Rare-Event Estimation in Language Models
Zixuan Liu, Gustavo Didier, Fangzheng Wu +2
Rare but severe failures of large language models can become practically significant at deployment scale. Estimating their probabilities with direct Monte Carlo sampling is highly…
Where to Look Is Not How to Fix: Pre-Denoising Diagnostics and Modality-Dependent Control in Diffusion Composition
Fangzheng Wu, Brian Summa
Understanding compositional failures in text-to-image diffusion requires identifying both where stress is detectable and how intervention changes the output. We study these questio…
Risk-Conditioned Fine-Tuning of Large Language Models
Zixuan Liu, Fangzheng Wu, Brian Summa +2
Large Language Models (LLMs) are increasingly deployed in settings where rare but severe harmful generations can have significant consequences. Existing Risk-Averse RLHF addresses…
Adaptive Multilevel Twisted Sequential Monte Carlo for Rare Events Estimation in Language Models
Zixuan Liu, Fangzheng Wu, Brian Summa +1
Rare unsafe behaviors in large language models can remain practically significant even when their probability is extremely small, particularly at deployment scales involving millio…
Robust General Utility for Reinforcement Learning
Zixuan Liu, Fangzheng Wu, Brian Summa +1
Reinforcement learning (RL) with general utility extends classic RL by optimizing an arbitrary utility functional of the policy-induced occupancy measure, thereby enabling a broade…
Attention Sinks in Diffusion Transformers: A Causal Analysis
Fangzheng Wu, Brian Summa
Attention sinks -- tokens that receive disproportionate attention mass -- are assumed to be functionally important in autoregressive language models, but their role in diffusion tr…
Model-Centric Diagnostics: A Framework for Internal State Readouts
Fangzheng Wu, Brian Summa
We present a model-centric diagnostic framework that treats training state as a latent variable and unifies a family of internal readouts -- head-gradient norms, confidence, entrop…
Rapid and Precise Topological Comparison with Merge Tree Neural Networks
Yu Qin, Brittany Terese Fasy, Carola Wenk +1
Merge trees are a valuable tool in the scientific visualization of scalar fields; however, current methods for merge tree comparisons are computationally expensive, primarily due t…