3.7k citations · 5k across the 45 of their papers we have counts for
46 papers
Doubly Stochastic Models: Learning with Unbiased Label Noises and Inference Stability
Haoyi Xiong, Xuhong Li, Boyang Yu +3
Random label noises (or observational noises) widely exist in practical machine learning settings. While previous studies primarily focus on the affects of label noises to the perf…
MonoFlow: Rethinking Divergence GANs via the Perspective of Wasserstein Gradient Flows
Mingxuan Yi, Zhanxing Zhu, Song Liu
The conventional understanding of adversarial training in generative adversarial networks (GANs) is that the discriminator is trained to estimate a divergence, and the generator le…
Fine-grained differentiable physics: a yarn-level model for fabrics
Deshan Gong, Zhanxing Zhu, Andrew J. Bulpitt +1
Differentiable physics modeling combines physics models with gradient-based learning to provide model explicability and data efficiency. It has been used to learn dynamics, solve i…
Proceedings of ICML 2021 Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI
Quanshi Zhang, Tian Han, Lixin Fan +5
This is the Proceedings of ICML 2021 Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI. Deep neural networks (DNNs) have undoubtedly brought grea…
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization
Zeke Xie, Li Yuan, Zhanxing Zhu +1
It is well-known that stochastic gradient noise (SGN) acts as implicit regularization for deep learning and is essentially important for both optimization and generalization of dee…
Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting
Mengzhang Li, Zhanxing Zhu
Spatial-temporal data forecasting of traffic flow is a challenging task because of complicated spatial dependencies and dynamical trends of temporal pattern between different roads…
Amata: An Annealing Mechanism for Adversarial Training Acceleration
Nanyang Ye, Qianxiao Li, Xiao-Yun Zhou +1
Despite the empirical success in various domains, it has been revealed that deep neural networks are vulnerable to maliciously perturbed input data that much degrade their performa…
Knowledge Distillation in Wide Neural Networks: Risk Bound, Data Efficiency and Imperfect Teacher
Guangda Ji, Zhanxing Zhu
Knowledge distillation is a strategy of training a student network with guide of the soft output from a teacher network. It has been a successful method of model compression and kn…
Automatic Data Augmentation for 3D Medical Image Segmentation
Ju Xu, Mengzhang Li, Zhanxing Zhu
Data augmentation is an effective and universal technique for improving generalization performance of deep neural networks. It could enrich diversity of training samples that is es…
Neural Approximate Sufficient Statistics for Implicit Models
Yanzhi Chen, Dinghuai Zhang, Michael Gutmann +2
We consider the fundamental problem of how to automatically construct summary statistics for implicit generative models where the evaluation of the likelihood function is intractab…
Informative Dropout for Robust Representation Learning: A Shape-bias Perspective
Baifeng Shi, Dinghuai Zhang, Qi Dai +3
Convolutional Neural Networks (CNNs) are known to rely more on local texture rather than global shape when making decisions. Recent work also indicates a close relationship between…
Global Robustness Verification Networks
Weidi Sun, Yuteng Lu, Xiyue Zhang +2
The wide deployment of deep neural networks, though achieving great success in many domains, has severe safety and reliability concerns. Existing adversarial attack generation and…
Spherical Motion Dynamics: Learning Dynamics of Neural Network with Normalization, Weight Decay, and SGD
Ruosi Wan, Zhanxing Zhu, Xiangyu Zhang +1
In this work, we comprehensively reveal the learning dynamics of neural network with normalization, weight decay (WD), and SGD (with momentum), named as Spherical Motion Dynamics (…
On Leveraging Unlabeled Data for Concurrent Positive-Unlabeled Classification and Robust Generation
Bing Yu, Ke Sun, He Wang +2
The scarcity of class-labeled data is a ubiquitous bottleneck in many machine learning problems. While abundant unlabeled data typically exist and provide a potential solution, it…
Black-Box Certification with Randomized Smoothing: A Functional Optimization Based Framework
Dinghuai Zhang, Mao Ye, Chengyue Gong +2
Randomized classifiers have been shown to provide a promising approach for achieving certified robustness against adversarial attacks in deep learning. However, most existing metho…
Towards Making Deep Transfer Learning Never Hurt
Ruosi Wan, Haoyi Xiong, Xingjian Li +2
Transfer learning have been frequently used to improve deep neural network training through incorporating weights of pre-trained networks as the starting-point of optimization for…
Patch-level Neighborhood Interpolation: A General and Effective Graph-based Regularization Strategy
Ke Sun, Bing Yu, Zhouchen Lin +1
Regularization plays a crucial role in machine learning models, especially for deep neural networks. The existing regularization techniques mainly rely on the i.i.d. assumption and…
Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling
He Wang, Edmond S. L. Ho, Hubert P. H. Shum +1
Data-driven modeling of human motions is ubiquitous in computer graphics and computer vision applications, such as synthesizing realistic motions or recognizing actions. Recent res…
AdaGCN: Adaboosting Graph Convolutional Networks into Deep Models
Ke Sun, Zhanxing Zhu, Zhouchen Lin
The design of deep graph models still remains to be investigated and the crucial part is how to explore and exploit the knowledge from different hops of neighbors in an efficient w…
On the Noisy Gradient Descent that Generalizes as SGD
Jingfeng Wu, Wenqing Hu, Haoyi Xiong +3
The gradient noise of SGD is considered to play a central role in the observed strong generalization abilities of deep learning. While past studies confirm that the magnitude and t…
On the Learning Dynamics of Two-layer Nonlinear Convolutional Neural Networks
Bing Yu, Junzhao Zhang, Zhanxing Zhu
Convolutional neural networks (CNNs) have achieved remarkable performance in various fields, particularly in the domain of computer vision. However, why this architecture works wel…
Interpreting Adversarially Trained Convolutional Neural Networks
Tianyuan Zhang, Zhanxing Zhu
We attempt to interpret how adversarially trained convolutional neural networks (AT-CNNs) recognize objects. We design systematic approaches to interpret AT-CNNs in both qualitativ…
Bayesian Optimized Continual Learning with Attention Mechanism
Ju Xu, Jin Ma, Zhanxing Zhu
Though neural networks have achieved much progress in various applications, it is still highly challenging for them to learn from a continuous stream of tasks without forgetting. C…
Efficient Neural Architecture Search via Proximal Iterations
Quanming Yao, Ju Xu, Wei-Wei Tu +1
Neural architecture search (NAS) recently attracts much research attention because of its ability to identify better architectures than handcrafted ones. However, many NAS methods,…