619 citations · 6.4k across the 178 of their papers we have counts for
17 papers · 1 filter
CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection
Hanshu Yan, Jingfeng Zhang, Gang Niu +3
We investigate the adversarial robustness of CNNs from the perspective of channel-wise activations. By comparing \textit{non-robust} (normally trained) and \textit{robustified} (ad…
Improving Generalization in Reinforcement Learning with Mixture Regularization
Kaixin Wang, Bingyi Kang, Jie Shao +1
Deep reinforcement learning (RL) agents trained in a limited set of environments tend to suffer overfitting and fail to generalize to unseen testing environments. To improve their…
Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning
Pan Zhou, Jiashi Feng, Chao Ma +3
It is not clear yet why ADAM-alike adaptive gradient algorithms suffer from worse generalization performance than SGD despite their faster training speed. This work aims to provide…
On Robustness of Neural Ordinary Differential Equations
Hanshu Yan, Jiawei Du, Vincent Y. F. Tan +1
Neural ordinary differential equations (ODEs) have been attracting increasing attention in various research domains recently. There have been some works studying optimization issue…
Hierarchic Neighbors Embedding
Shenglan Liu, Yang Yu, Yang Liu +3
Manifold learning now plays a very important role in machine learning and many relevant applications. Although its superior performance in dealing with nonlinear data distribution,…
Hierarchical Meta Learning
Yingtian Zou, Jiashi Feng
Meta learning is a promising solution to few-shot learning problems. However, existing meta learning methods are restricted to the scenarios where training and application tasks sh…
Understanding Generalization and Optimization Performance of Deep CNNs
Pan Zhou, Jiashi Feng
This work aims to provide understandings on the remarkable success of deep convolutional neural networks (CNNs) by theoretically analyzing their generalization performance and esta…
Nonconvex Sparse Spectral Clustering by Alternating Direction Method of Multipliers and Its Convergence Analysis
Canyi Lu, Jiashi Feng, Zhouchen Lin +1
Spectral Clustering (SC) is a widely used data clustering method which first learns a low-dimensional embedding of data by computing the eigenvectors of the normalized Laplacia…
Stochastic Primal-Dual Proximal ExtraGradient Descent for Compositely Regularized Optimization
Tianyi Lin, Linbo Qiao, Teng Zhang +2
We consider a wide range of regularized stochastic minimization problems with two regularization terms, one of which is composed with a linear function. This optimization model abs…
On Fundamental Limits of Robust Learning
Jiashi Feng
We consider the problems of robust PAC learning from distributed and streaming data, which may contain malicious errors and outliers, and analyze their fundamental complexity quest…
Outlier Robust Online Learning
Jiashi Feng, Huan Xu, Shie Mannor
We consider the problem of learning from noisy data in practical settings where the size of data is too large to store on a single machine. More challenging, the data coming from t…
Hyperparameter Transfer Learning through Surrogate Alignment for Efficient Deep Neural Network Training
Ilija Ilievski, Jiashi Feng
Recently, several optimization methods have been successfully applied to the hyperparameter optimization of deep neural networks (DNNs). The methods work by modeling the joint dist…
Deep Q-Networks for Accelerating the Training of Deep Neural Networks
Jie Fu
In this paper, we propose a principled deep reinforcement learning (RL) approach that is able to accelerate the convergence rate of general deep neural networks (DNNs). With our ap…
Ensemble Robustness and Generalization of Stochastic Deep Learning Algorithms
Tom Zahavy, Bingyi Kang, Alex Sivak +3
The question why deep learning algorithms generalize so well has attracted increasing research interest. However, most of the well-established approaches, such as hypothesis capaci…
DrMAD: Distilling Reverse-Mode Automatic Differentiation for Optimizing Hyperparameters of Deep Neural Networks
Jie Fu, Hongyin Luo, Jiashi Feng +2
The performance of deep neural networks is well-known to be sensitive to the setting of their hyperparameters. Recent advances in reverse-mode automatic differentiation allow for o…
Learning with -Graph: -Induced Sparse Subspace Clustering
Yingzhen Yang, Jiashi Feng, Jianchao Yang +1
Sparse subspace clustering methods, such as Sparse Subspace Clustering (SSC) \cite{ElhamifarV13} and -graph \cite{YanW09,ChengYYFH10}, are effective in partitioning the d…
Robust PCA in High-dimension: A Deterministic Approach
Jiashi Feng, Huan Xu, Shuicheng Yan
We consider principal component analysis for contaminated data-set in the high dimensional regime, where the dimensionality of each observation is comparable or even more than the…