111 citations · 275 across the 9 of their papers we have counts for
10 papers
Oasis: One Image is All You Need for Multimodal Instruction Data Synthesis
Letian Zhang, Quan Cui, Bingchen Zhao +1
The success of multi-modal large language models (MLLMs) has been largely attributed to the large-scale training data. However, the training data of many MLLMs is unavailable due t…
DOT: A Distillation-Oriented Trainer
Borui Zhao, Quan Cui, Renjie Song +1
Knowledge distillation transfers knowledge from a large model to a small one via task and distillation losses. In this paper, we observe a trade-off between task and distillation l…
Vision Learners Meet Web Image-Text Pairs
Bingchen Zhao, Quan Cui, Hao Wu +3
Many self-supervised learning methods are pre-trained on the well-curated ImageNet-1K dataset. In this work, given the excellent scalability of web data, we consider self-supervise…
Decoupled Knowledge Distillation
Borui Zhao, Quan Cui, Renjie Song +2
State-of-the-art distillation methods are mainly based on distilling deep features from intermediate layers, while the significance of logit distillation is greatly overlooked. To…
Discriminability-Transferability Trade-Off: An Information-Theoretic Perspective
Quan Cui, Bingchen Zhao, Zhao-Min Chen +5
This work simultaneously considers the discriminability and transferability properties of deep representations in the typical supervised learning task, i.e., image classification.…
Contrastive Vision-Language Pre-training with Limited Resources
Quan Cui, Boyan Zhou, Yu Guo +4
Pioneering dual-encoder pre-training works (e.g., CLIP and ALIGN) have revealed the potential of aligning multi-modal representations with contrastive learning. However, these work…