116 citations · 1.1k across the 96 of their papers we have counts for
39 papers · 1 filter
Graph Spectral Embedding using the Geodesic Betweeness Centrality
Shay Deutsch, Stefano Soatto
We introduce the Graph Sylvester Embedding (GSE), an unsupervised graph representation of local similarity, connectivity, and global structure. GSE uses the solution of the Sylvest…
Gacs-Korner Common Information Variational Autoencoder
Michael Kleinman, Alessandro Achille, Stefano Soatto +1
We propose a notion of common information that allows one to quantify and separate the information that is shared between two random variables from the information that is unique t…
Towards Differential Relational Privacy and its use in Question Answering
Simone Bombari, Alessandro Achille, Zijian Wang +6
Memorization of the relation between entities in a dataset can lead to privacy issues when using a trained model for question answering. We introduce Relational Memorization (RM) t…
Contrastive Neighborhood Alignment
Pengkai Zhu, Zhaowei Cai, Yuanjun Xiong +4
We present Contrastive Neighborhood Alignment (CNA), a manifold learning approach to maintain the topology of learned features whereby data points that are mapped to nearby represe…
DIVA: Dataset Derivative of a Learning Task
Yonatan Dukler, Alessandro Achille, Giovanni Paolini +3
We present a method to compute the derivative of a learning task with respect to a dataset. A learning task is a function from a training set to the validation error, which can be…
Uniform Sampling over Episode Difficulty
Sébastien M. R. Arnold, Guneet S. Dhillon, Avinash Ravichandran +1
Episodic training is a core ingredient of few-shot learning to train models on tasks with limited labelled data. Despite its success, episodic training remains largely understudied…
Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning
Zhaowei Cai, Avinash Ravichandran, Subhransu Maji +3
We present a plug-in replacement for batch normalization (BN) called exponential moving average normalization (EMAN), which improves the performance of existing student-teacher bas…
Estimating informativeness of samples with Smooth Unique Information
Hrayr Harutyunyan, Alessandro Achille, Giovanni Paolini +4
We define a notion of information that an individual sample provides to the training of a neural network, and we specialize it to measure both how much a sample informs the final w…
Structured Prediction as Translation between Augmented Natural Languages
Giovanni Paolini, Ben Athiwaratkun, Jason Krone +6
We propose a new framework, Translation between Augmented Natural Languages (TANL), to solve many structured prediction language tasks including joint entity and relation extractio…
LQF: Linear Quadratic Fine-Tuning
Alessandro Achille, Aditya Golatkar, Avinash Ravichandran +2
Classifiers that are linear in their parameters, and trained by optimizing a convex loss function, have predictable behavior with respect to changes in the training data, initial c…
Mixed-Privacy Forgetting in Deep Networks
Aditya Golatkar, Alessandro Achille, Avinash Ravichandran +2
We show that the influence of a subset of the training samples can be removed -- or "forgotten" -- from the weights of a network trained on large-scale image classification tasks,…
Spectral Embedding of Graph Networks
Shay Deutsch, Stefano Soatto
We introduce an unsupervised graph embedding that trades off local node similarity and connectivity, and global structure. The embedding is based on a generalized graph Laplacian,…
Predicting Training Time Without Training
Luca Zancato, Alessandro Achille, Avinash Ravichandran +2
We tackle the problem of predicting the number of optimization steps that a pre-trained deep network needs to converge to a given value of the loss function. To do so, we leverage…
Stochastic batch size for adaptive regularization in deep network optimization
Kensuke Nakamura, Stefano Soatto, Byung-Woo Hong
We propose a first-order stochastic optimization algorithm incorporating adaptive regularization applicable to machine learning problems in deep learning framework. The adaptive re…
Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations
Aditya Golatkar, Alessandro Achille, Stefano Soatto
We describe a procedure for removing dependency on a cohort of training data from a trained deep network that improves upon and generalizes previous methods to different readout fu…
Incremental Meta-Learning via Indirect Discriminant Alignment
Qing Liu, Orchid Majumder, Alessandro Achille +3
Majority of the modern meta-learning methods for few-shot classification tasks operate in two phases: a meta-training phase where the meta-learner learns a generic representation b…
Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks
Aditya Golatkar, Alessandro Achille, Stefano Soatto
We explore the problem of selectively forgetting a particular subset of the data used for training a deep neural network. While the effects of the data to be forgotten can be hidde…
A geometric interpretation of stochastic gradient descent using diffusion metrics
R. Fioresi, P. Chaudhari, S. Soatto
Stochastic gradient descent (SGD) is a key ingredient in the training of deep neural networks and yet its geometrical significance appears elusive. We study a deterministic model i…
Meta-Q-Learning
Rasool Fakoor, Pratik Chaudhari, Stefano Soatto +1
This paper introduces Meta-Q-Learning (MQL), a new off-policy algorithm for meta-Reinforcement Learning (meta-RL). MQL builds upon three simple ideas. First, we show that Q-learnin…
A Baseline for Few-Shot Image Classification
Guneet S. Dhillon, Pratik Chaudhari, Avinash Ravichandran +1
Fine-tuning a deep network trained with the standard cross-entropy loss is a strong baseline for few-shot learning. When fine-tuned transductively, this outperforms the current sta…
Toward Understanding Catastrophic Forgetting in Continual Learning
Cuong V. Nguyen, Alessandro Achille, Michael Lam +3
We study the relationship between catastrophic forgetting and properties of task sequences. In particular, given a sequence of tasks, we would like to understand which properties o…
Time Matters in Regularizing Deep Networks: Weight Decay and Data Augmentation Affect Early Learning Dynamics, Matter Little Near Convergence
Aditya Golatkar, Alessandro Achille, Stefano Soatto
Regularization is typically understood as improving generalization by altering the landscape of local extrema to which the model eventually converges. Deep neural networks (DNNs),…
Unsupervised Domain Adaptation via Regularized Conditional Alignment
Safa Cicek, Stefano Soatto
We propose a method for unsupervised domain adaptation that trains a shared embedding to align the joint distributions of inputs (domain) and outputs (classes), making any classifi…
Where is the Information in a Deep Neural Network?
Alessandro Achille, Giovanni Paolini, Stefano Soatto
Whatever information a deep neural network has gleaned from training data is encoded in its weights. How this information affects the response of the network to future data remains…