38 citations · 82 across the 16 of their papers we have counts for
8 papers · 1 filter
On the Pros and Cons of Momentum Encoder in Self-Supervised Visual Representation Learning
Trung Pham, Chaoning Zhang, Axi Niu +2
Exponential Moving Average (EMA or momentum) is widely used in modern self-supervised learning (SSL) approaches, such as MoCo, for enhancing performance. We demonstrate that such m…
A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond
Chaoning Zhang, Chenshuang Zhang, Junha Song +3
Masked autoencoders are scalable vision learners, as the title of MAE \cite{he2022masked}, which suggests that self-supervised learning (SSL) in vision might undertake a similar tr…
Decoupled Adversarial Contrastive Learning for Self-supervised Adversarial Robustness
Chaoning Zhang, Kang Zhang, Chenshuang Zhang +4
Adversarial training (AT) for robust representation learning and self-supervised learning (SSL) for unsupervised representation learning are two active research fields. Integrating…
Understanding and Improving Group Normalization
Agus Gunawan, Xu Yin, Kang Zhang
Various normalization layers have been proposed to help the training of neural networks. Group Normalization (GN) is one of the effective and attractive studies that achieved signi…
Investigating Top- White-Box and Transferable Black-box Attack
Chaoning Zhang, Philipp Benz, Adil Karjauv +3
Existing works have identified the limitation of top- attack success rate (ASR) as a metric to evaluate the attack strength but exclusively investigated it in the white-box sett…
Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo
Chaoning Zhang, Kang Zhang, Trung X. Pham +4
Contrastive learning (CL) is widely known to require many negative samples, 65536 in MoCo for instance, for which the performance of a dictionary-free framework is often inferior b…
How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learning
Chaoning Zhang, Kang Zhang, Chenshuang Zhang +3
To avoid collapse in self-supervised learning (SSL), a contrastive loss is widely used but often requires a large number of negative samples. Without negative samples yet achieving…
Fast Adversarial Training with Noise Augmentation: A Unified Perspective on RandStart and GradAlign
Axi Niu, Kang Zhang, Chaoning Zhang +4
PGD-based and FGSM-based are two popular adversarial training (AT) approaches for obtaining adversarially robust models. Compared with PGD-based AT, FGSM-based one is significantly…