20 citations · 134 across the 33 of their papers we have counts for
24 papers · 1 filter
Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspective
Yifei Wang, Liangchen Li, Jiansheng Yang +2
Adversarial Training (AT) has become arguably the state-of-the-art algorithm for extracting robust features. However, researchers recently notice that AT suffers from severe robust…
Robust Long-Tailed Learning via Label-Aware Bounded CVaR
Hong Zhu, Runpeng Yu, Xing Tang +3
Data in the real-world classification problems are always imbalanced or long-tailed, wherein the majority classes have the most of the samples that dominate the model training. In…
Rethinking Weak Supervision in Helping Contrastive Learning
Jingyi Cui, Weiran Huang, Yifei Wang +1
Contrastive learning has shown outstanding performances in both supervised and unsupervised learning, and has recently been introduced to solve weakly supervised learning problems…
ContraNorm: A Contrastive Learning Perspective on Oversmoothing and Beyond
Xiaojun Guo, Yifei Wang, Tianqi Du +1
Oversmoothing is a common phenomenon in a wide range of Graph Neural Networks (GNNs) and Transformers, where performance worsens as the number of layers increases. Instead of chara…
CFA: Class-wise Calibrated Fair Adversarial Training
Zeming Wei, Yifei Wang, Yiwen Guo +1
Adversarial training has been widely acknowledged as the most effective method to improve the adversarial robustness against adversarial examples for Deep Neural Networks (DNNs). S…
A Message Passing Perspective on Learning Dynamics of Contrastive Learning
Yifei Wang, Qi Zhang, Tianqi Du +3
In recent years, contrastive learning achieves impressive results on self-supervised visual representation learning, but there still lacks a rigorous understanding of its learning…
Towards a Unified Theoretical Understanding of Non-contrastive Learning via Rank Differential Mechanism
Zhijian Zhuo, Yifei Wang, Jinwen Ma +1
Recently, a variety of methods under the name of non-contrastive learning (like BYOL, SimSiam, SwAV, DINO) show that when equipped with some asymmetric architectural designs, align…
Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learning
Rundong Luo, Yifei Wang, Yisen Wang
Recent works have shown that self-supervised learning can achieve remarkable robustness when integrated with adversarial training (AT). However, the robustness gap between supervis…
USER: Unsupervised Structural Entropy-based Robust Graph Neural Network
Yifei Wang, Yupan Wang, Zeyu Zhang +3
Unsupervised/self-supervised graph neural networks (GNN) are vulnerable to inherent randomness in the input graph data which greatly affects the performance of the model in downstr…
On the Connection between Invariant Learning and Adversarial Training for Out-of-Distribution Generalization
Shiji Xin, Yifei Wang, Jingtong Su +1
Despite impressive success in many tasks, deep learning models are shown to rely on spurious features, which will catastrophically fail when generalized to out-of-distribution (OOD…
Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priors
Qixun Wang, Yifei Wang, Hong Zhu +1
Deep models often fail to generalize well in test domains when the data distribution differs from that in the training domain. Among numerous approaches to address this Out-of-Dist…
How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders
Qi Zhang, Yifei Wang, Yisen Wang
Masked Autoencoders (MAE) based on a reconstruction task have risen to be a promising paradigm for self-supervised learning (SSL) and achieve state-of-the-art performance across di…
Overparameterized ReLU Neural Networks Learn the Simplest Models: Neural Isometry and Exact Recovery
Yifei Wang, Yixuan Hua, Emmanuel Candés +1
The practice of deep learning has shown that neural networks generalize remarkably well even with an extreme number of learned parameters. This appears to contradict traditional st…
Optimization-Induced Graph Implicit Nonlinear Diffusion
Qi Chen, Yifei Wang, Yisen Wang +2
Due to the over-smoothing issue, most existing graph neural networks can only capture limited dependencies with their inherently finite aggregation layers. To overcome this limitat…
Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimization
Yifei Wang, Peng Chen, Mert Pilanci +1
The computation of Wasserstein gradient direction is essential for posterior sampling problems and scientific computing. The approximation of the Wasserstein gradient with finite s…
A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training
Yifei Wang, Yisen Wang, Jiansheng Yang +1
Adversarial Training (AT) is known as an effective approach to enhance the robustness of deep neural networks. Recently researchers notice that robust models with AT have good gene…
Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap
Yifei Wang, Qi Zhang, Yisen Wang +2
Recently, contrastive learning has risen to be a promising approach for large-scale self-supervised learning. However, theoretical understanding of how it works is still unclear. I…
Fooling Adversarial Training with Inducing Noise
Zhirui Wang, Yifei Wang, Yisen Wang
Adversarial training is widely believed to be a reliable approach to improve model robustness against adversarial attack. However, in this paper, we show that when trained on one t…
Residual Relaxation for Multi-view Representation Learning
Yifei Wang, Zhengyang Geng, Feng Jiang +4
Multi-view methods learn representations by aligning multiple views of the same image and their performance largely depends on the choice of data augmentation. In this paper, we no…
The Convex Geometry of Backpropagation: Neural Network Gradient Flows Converge to Extreme Points of the Dual Convex Program
Yifei Wang, Mert Pilanci
We study non-convex subgradient flows for training two-layer ReLU neural networks from a convex geometry and duality perspective. We characterize the implicit bias of unregularized…
Parallel Deep Neural Networks Have Zero Duality Gap
Yifei Wang, Tolga Ergen, Mert Pilanci
Training deep neural networks is a challenging non-convex optimization problem. Recent work has proven that the strong duality holds (which means zero duality gap) for regularized…
Projected Wasserstein gradient descent for high-dimensional Bayesian inference
Yifei Wang, Peng Chen, Wuchen Li
We propose a projected Wasserstein gradient descent method (pWGD) for high-dimensional Bayesian inference problems. The underlying density function of a particle system of WGD is a…
Dissecting the Diffusion Process in Linear Graph Convolutional Networks
Yifei Wang, Yisen Wang, Jiansheng Yang +1
Graph Convolutional Networks (GCNs) have attracted more and more attentions in recent years. A typical GCN layer consists of a linear feature propagation step and a nonlinear trans…
The Hidden Convex Optimization Landscape of Two-Layer ReLU Neural Networks: an Exact Characterization of the Optimal Solutions
Yifei Wang, Jonathan Lacotte, Mert Pilanci
We prove that finding all globally optimal two-layer ReLU neural networks can be performed by solving a convex optimization program with cone constraints. Our analysis is novel, ch…