1 citations · 1 across the 12 of their papers we have counts for
12 papers
A Theoretical Framework for Masked Pretraining (MPT)
Qi Zhang, Runyu Zhou, Yifei Wang +1
Recently, Masked Pretraining (MPT) based on reconstruction pretraining tasks has risen to a promising self-supervised learning paradigm across various domains and achieves remarkab…
Diagnosing Visual Ignorance in Vision-Language Models
Runyu Zhou, Qi Zhang, Qixun Wang +1
Vision-language models (VLMs) achieve high accuracy on many visual question-answering benchmarks, yet it remains unclear whether this accuracy reflects reliable use of the visual d…
UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?
Zimo Wen, Boxiu Li, Wanbo Zhang +11
Unified multimodal models have recently demonstrated strong generative capabilities, yet whether and when generation improves understanding remains unclear. Existing benchmarks lac…
SAE as a Crystal Ball: Interpretable Features Predict Cross-domain Transferability of LLMs without Training
Qi Zhang, Yifei Wang, Xiaohan Wang +4
In recent years, pre-trained large language models have achieved remarkable success across diverse tasks. Besides the pivotal role of self-supervised pre-training, their effectiven…
An Augmentation Overlap Theory of Contrastive Learning
Qi Zhang, Yifei Wang, Yisen Wang
Recently, self-supervised contrastive learning has achieved great success on various tasks. However, its underlying working mechanism is yet unclear. In this paper, we first provid…
SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning
Xiaojun Guo, Runyu Zhou, Yifei Wang +8
Vision-language models (VLMs) have shown remarkable abilities by integrating large language models with visual inputs. However, they often fail to utilize visual evidence adequatel…
On the Limits of Sparse Autoencoders: A Theoretical Framework and Reweighted Remedy
Jingyi Cui, Qi Zhang, Yifei Wang +1
Sparse autoencoders (SAEs) have recently emerged as a powerful tool for interpreting the features learned by large language models (LLMs). By reconstructing features with sparsely…
Projection Head is Secretly an Information Bottleneck
Zhuo Ouyang, Kaiwen Hu, Qi Zhang +2
Recently, contrastive learning has risen to be a promising paradigm for extracting meaningful data representations. Among various special designs, adding a projection head on top o…
Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness
Qi Zhang, Yifei Wang, Jingyi Cui +4
Deep learning models often suffer from a lack of interpretability due to polysemanticity, where individual neurons are activated by multiple unrelated semantics, resulting in uncle…
Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining
Qi Zhang, Tianqi Du, Haotian Huang +2
In recent years, the rise of generative self-supervised learning (SSL) paradigms has exhibited impressive performance across visual, language, and multi-modal domains. While the va…
Non-negative Contrastive Learning
Yifei Wang, Qi Zhang, Yaoyu Guo +1
Deep representations have shown promising performance when transferred to downstream tasks in a black-box manner. Yet, their inherent lack of interpretability remains a significant…
Identifiable Contrastive Learning with Automatic Feature Importance Discovery
Qi Zhang, Yifei Wang, Yisen Wang
Existing contrastive learning methods rely on pairwise sample contrast to learn data representations, but the learned features often lack clear interpretability f…