8 papers
The Relative Instability of Model Comparison with Cross-validation
Alexandre Bayle, Lucas Janson, Lester Mackey
Cross-validation (CV) is known to provide asymptotically exact tests and confidence intervals for model improvement but only when the model comparison is relatively stable. Surpris…
WildCat: Near-Linear Attention in Theory and Practice
Tobias Schröder, Lester Mackey
We introduce WildCat, a high-accuracy, low-cost approach to compressing the attention mechanism in neural networks. While attention is a staple of modern network architectures, it…
Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner
Cai Zhou, Chenxiao Yang, Yi Hu +7
Diffusion language models, especially masked discrete diffusion models, have achieved great success recently. While there are some theoretical and primary empirical results showing…
Estimating Treatment Effects with Independent Component Analysis
Patrik Reizinger, Lester Mackey, Wieland Brendel +1
Independent Component Analysis (ICA) uses a measure of non-Gaussianity to identify latent sources from data and estimate their mixing coefficients (Shimizu et al., 2006). Meanwhile…
Low-Rank Thinning
Annabelle Michael Carrell, Albert Gong, Abhishek Shetty +2
The goal in thinning is to summarize a dataset using a small set of representative points. Remarkably, sub-Gaussian thinning algorithms like Kernel Halving and Compress can match t…
Informed Correctors for Discrete Diffusion Models
Yixiu Zhao, Jiaxin Shi, Feng Chen +3
Discrete diffusion has emerged as a powerful framework for generative modeling in discrete domains, yet efficiently sampling from these models remains challenging. Existing samplin…