81 citations · 540 across the 31 of their papers we have counts for
31 papers
Queer In AI: A Case Study in Community-Led Participatory AI
Organizers Of QueerInAI, :, Anaelia Ovalle +48
We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community…
Exphormer: Sparse Transformers for Graphs
Hamed Shirzad, Ameya Velingker, Balaji Venkatachalam +2
Graph transformers have emerged as a promising architecture for a variety of graph learning and representation tasks. Despite their successes, though, it remains challenging to sca…
Differentially Private Neural Tangent Kernels for Privacy-Preserving Data Generation
Yilin Yang, Kamil Adamczewski, Danica J. Sutherland +2
Maximum mean discrepancy (MMD) is a particularly useful distance metric for differentially private data generation: when used with finite-dimensional features it allows us to summa…
How to prepare your task head for finetuning
Yi Ren, Shangmin Guo, Wonho Bae +1
In deep learning, transferring information from a pretrained network to a downstream task by finetuning has many benefits. The choice of task head plays an important role in fine-t…
Efficient Conditionally Invariant Representation Learning
Roman Pogodin, Namrata Deka, Yazhe Li +3
We introduce the Conditional Independence Regression CovariancE (CIRCE), a measure of conditional independence for multivariate continuous-valued variables. CIRCE applies as a regu…
MMD-B-Fair: Learning Fair Representations with Statistical Testing
Namrata Deka, Danica J. Sutherland
We introduce a method, MMD-B-Fair, to learn fair representations of data via kernel two-sample testing. We find neural features of our data where a maximum mean discrepancy (MMD) t…
A Non-Asymptotic Moreau Envelope Theory for High-Dimensional Generalized Linear Models
Lijia Zhou, Frederic Koehler, Pragya Sur +2
We prove a new generalization bound that shows for any class of linear predictors in Gaussian space, the Rademacher complexity of the class and the training error under any continu…
Object Discovery via Contrastive Learning for Weakly Supervised Object Detection
Jinhwan Seo, Wonho Bae, Danica J. Sutherland +2
Weakly Supervised Object Detection (WSOD) is a task that detects objects in an image using a model trained only on image-level annotations. Current state-of-the-art models benefit…
Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels
Mohamad Amin Mohamadi, Wonho Bae, Danica J. Sutherland
We propose a new method for approximating active learning acquisition strategies that are based on retraining with hypothetically-labeled candidate data points. Although this is us…
Evaluating Graph Generative Models with Contrastively Learned Features
Hamed Shirzad, Kaveh Hassani, Danica J. Sutherland
A wide range of models have been proposed for Graph Generative Models, necessitating effective methods to evaluate their quality. So far, most techniques use either traditional met…
A Fast, Well-Founded Approximation to the Empirical Neural Tangent Kernel
Mohamad Amin Mohamadi, Wonho Bae, Danica J. Sutherland
Empirical neural tangent kernels (eNTKs) can provide a good understanding of a given network's representation: they are often far less expensive to compute and applicable more broa…
One Weird Trick to Improve Your Semi-Weakly Supervised Semantic Segmentation Model
Wonho Bae, Junhyug Noh, Milad Jalali Asadabadi +1
Semi-weakly supervised semantic segmentation (SWSSS) aims to train a model to identify objects in images based on a small number of images with pixel-level labels, and many more im…
Pre-trained Perceptual Features Improve Differentially Private Image Generation
Fredrik Harder, Milad Jalali Asadabadi, Danica J. Sutherland +1
Training even moderately-sized generative models with differentially-private stochastic gradient descent (DP-SGD) is difficult: the required level of noise for reasonable levels of…
Better Supervisory Signals by Observing Learning Paths
Yi Ren, Shangmin Guo, Danica J. Sutherland
Better-supervised models might have better performance. In this paper, we first clarify what makes for good supervision for a classification problem, and then explain two existing…
Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression
Lijia Zhou, Frederic Koehler, Danica J. Sutherland +1
We study a localized notion of uniform convergence known as an "optimistic rate" (Panchenko 2002; Srebro et al. 2010) for linear regression with Gaussian data. Our refined analysis…
Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds, and Benign Overfitting
Frederic Koehler, Lijia Zhou, Danica J. Sutherland +1
We consider interpolation learning in high-dimensional linear regression with Gaussian data, and prove a generic uniform convergence guarantee on the generalization error of interp…
Self-Supervised Learning with Kernel Dependence Maximization
Yazhe Li, Roman Pogodin, Danica J. Sutherland +1
We approach self-supervised learning of image representations from a statistical dependence perspective, proposing Self-Supervised Learning with the Hilbert-Schmidt Independence Cr…
Meta Two-Sample Testing: Learning Kernels for Testing with Limited Data
Feng Liu, Wenkai Xu, Jie Lu +1
Modern kernel-based two-sample tests have shown great success in distinguishing complex, high-dimensional distributions with appropriate learned kernels. Previous work has demonstr…
Does Invariant Risk Minimization Capture Invariance?
Pritish Kamath, Akilesh Tangella, Danica J. Sutherland +1
We show that the Invariant Risk Minimization (IRM) formulation of Arjovsky et al. (2019) can fail to capture "natural" invariances, at least when used in its practical "linear" for…
On Uniform Convergence and Low-Norm Interpolation Learning
Lijia Zhou, Danica J. Sutherland, Nathan Srebro
We consider an underdetermined noisy linear regression model where the minimum-norm interpolating predictor is known to be consistent, and ask: can uniform convergence in a norm ba…
Learning Deep Kernels for Non-Parametric Two-Sample Tests
Feng Liu, Wenkai Xu, Jie Lu +3
We propose a class of kernel-based two-sample tests, which aim to determine whether two sets of samples are drawn from the same distribution. Our tests are constructed from kernels…
The Role of Machine Learning in the Next Decade of Cosmology
Michelle Ntampaka, Camille Avestruz, Steven Boada +27
In recent years, machine learning (ML) methods have remarkably improved how cosmologists can interpret data. The next decade will bring new opportunities for data-driven cosmologic…
Learning deep kernels for exponential family densities
Li Wenliang, Danica J. Sutherland, Heiko Strathmann +1
The kernel exponential family is a rich class of distributions, which can be fit efficiently and with statistical guarantees by score matching. Being required to choose a priori a…
On gradient regularizers for MMD GANs
Michael Arbel, Danica J. Sutherland, Mikołaj Bińkowski +1
We propose a principled method for gradient-based regularization of the critic of GAN-like models trained by adversarially optimizing the kernel of a Maximum Mean Discrepancy (MMD)…