57 citations · 301 across the 27 of their papers we have counts for
14 papers · 1 filter
Learning Rich Nearest Neighbor Representations from Self-supervised Ensembles
Bram Wallace, Devansh Arpit, Huan Wang +1
Pretraining convolutional neural networks via self-supervision, and applying them in transfer learning, is an incredibly fast-growing field that is rapidly and iteratively improvin…
Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalization
Devansh Arpit, Huan Wang, Yingbo Zhou +1
In Domain Generalization (DG) settings, models trained independently on a given set of training domains have notoriously chaotic performance on distribution shifted test domains, a…
Merlion: A Machine Learning Library for Time Series
Aadyot Bhatnagar, Paul Kassianik, Chenghao Liu +20
We introduce Merlion, an open-source machine learning library for time series. It features a unified interface for many commonly used models and datasets for anomaly detection and…
Understanding the Under-Coverage Bias in Uncertainty Estimation
Yu Bai, Song Mei, Huan Wang +1
Estimating the data uncertainty in regression tasks is often done by learning a quantile function or a prediction interval of the true label conditioned on the input. It is frequen…
Rethinking Adam: A Twofold Exponential Moving Average Approach
Yizhou Wang, Yue Kang, Can Qin +4
Adaptive gradient methods, e.g. \textsc{Adam}, have achieved tremendous success in machine learning. Scaling the learning rate element-wisely by a certain form of second moment est…
Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning
Tengyang Xie, Nan Jiang, Huan Wang +2
Recent theoretical work studies sample-efficient reinforcement learning (RL) extensively in two settings: learning interactively in the environment (online RL), or learning from an…
Dynamical Isometry: The Missing Ingredient for Neural Network Pruning
Huan Wang, Can Qin, Yue Bai +1
Several recent works [40, 24] observed an interesting phenomenon in neural network pruning: A larger finetuning learning rate can improve the final performance significantly. Unfor…
Local Calibration: Metrics and Recalibration
Rachel Luo, Aadyot Bhatnagar, Yu Bai +7
Probabilistic classifiers output confidence scores along with their predictions, and these confidence scores should be calibrated, i.e., they should reflect the reliability of the…
Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization
Stanislaw Jastrzebski, Devansh Arpit, Oliver Astrand +6
The early phase of training a deep neural network has a dramatic effect on the local curvature of the loss function. For instance, using a small learning rate does not guarantee st…
Towards Understanding Hierarchical Learning: Benefits of Neural Representations
Minshuo Chen, Yu Bai, Jason D. Lee +4
Deep neural networks can empirically perform efficient hierarchical learning, in which the layers learn useful representations of the data. However, how they make use of the interm…
Taylorized Training: Towards Better Approximation of Neural Network Training at Finite Width
Yu Bai, Ben Krause, Huan Wang +2
We propose \emph{Taylorized training} as an initiative towards better understanding neural network training at finite width. Taylorized training involves training the -th order…
On the Generalization Gap in Reparameterizable Reinforcement Learning
Huan Wang, Stephan Zheng, Caiming Xiong +1
Understanding generalization in reinforcement learning (RL) is a significant challenge, as many common assumptions of traditional supervised learning theory do not apply. We focus…
Identifying Generalization Properties in Neural Networks
Huan Wang, Nitish Shirish Keskar, Caiming Xiong +1
While it has not yet been proven, empirical evidence suggests that model generalization is related to local properties of the optima which can be described via the Hessian. We conn…
Structured Probabilistic Pruning for Convolutional Neural Network Acceleration
Huan Wang, Qiming Zhang, Yuehai Wang +1
In this paper, we propose a novel progressive parameter pruning method for Convolutional Neural Network acceleration, named Structured Probabilistic Pruning (SPP), which effectivel…