25 citations · 49 across the 6 of their papers we have counts for
6 papers
Revisiting Disentanglement in Downstream Tasks: A Study on Its Necessity for Abstract Visual Reasoning
Ruiqian Nai, Zixin Wen, Ji Li +2
In representation learning, a disentangled representation is highly desirable as it encodes generative factors of data in a separable and compact pattern. Researchers have advocate…
What Matters In The Structured Pruning of Generative Language Models?
Michael Santacroce, Zixin Wen, Yelong Shen +1
Auto-regressive large language models such as GPT-3 require enormous computational resources to use. Traditionally, structured pruning methods are employed to reduce resource usage…
The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning
Zixin Wen, Yuanzhi Li
Recently the surprising discovery of the Bootstrap Your Own Latent (BYOL) method by Grill et al. shows the negative term in contrastive loss can be removed if we add the so-called…
Improving Multi-Modal Learning with Uni-Modal Teachers
Chenzhuang Du, Tingle Li, Yichen Liu +4
Learning multi-modal representations is an essential step towards real-world robotic applications, and various multi-modal fusion models have been developed for this purpose. Howev…
Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning
Zixin Wen, Yuanzhi Li
How can neural networks trained by contrastive learning extract features from the unlabeled data? Why does contrastive learning usually need much stronger data augmentations than s…
Convergence of End-to-End Training in Deep Unsupervised Contrastive Learning
Zixin Wen
Unsupervised contrastive learning has gained increasing attention in the latest research and has proven to be a powerful method for learning representations from unlabeled data. Ho…