5 papers
WildCat: Near-Linear Attention in Theory and Practice
Tobias Schröder, Lester Mackey
We introduce WildCat, a high-accuracy, low-cost approach to compressing the attention mechanism in neural networks. While attention is a staple of modern network architectures, it…
Estimating Treatment Effects with Independent Component Analysis
Patrik Reizinger, Lester Mackey, Wieland Brendel +1
Independent Component Analysis (ICA) uses a measure of non-Gaussianity to identify latent sources from data and estimate their mixing coefficients (Shimizu et al., 2006). Meanwhile…
Low-Rank Thinning
Annabelle Michael Carrell, Albert Gong, Abhishek Shetty +2
The goal in thinning is to summarize a dataset using a small set of representative points. Remarkably, sub-Gaussian thinning algorithms like Kernel Halving and Compress can match t…
SureMap: Simultaneous Mean Estimation for Single-Task and Multi-Task Disaggregated Evaluation
Mikhail Khodak, Lester Mackey, Alexandra Chouldechova +1
Disaggregated evaluation -- estimation of performance of a machine learning model on different subpopulations -- is a core task when assessing performance and group-fairness of AI…
Bounding Hellinger Distance with Stein's Method
Morgane Austern, Lester Mackey
This work introduces a new, explicit bound on the Hellinger distance between a continuous random variable and a Gaussian with matching mean and variance. As example applications, w…