EVA-CLIP: Improved Training Techniques for CLIP at Scale

Abstract

Contrastive language-image pre-training, CLIP for short, has gained increasing attention for its potential in various scenarios. In this paper, we propose EVA-CLIP, a series of models that significantly improve the efficiency and effectiveness of CLIP training. Our approach incorporates new techniques for representation learning, optimization, and augmentation, enabling EVA-CLIP to achieve superior performance compared to previous CLIP models with the same number of parameters but significantly smaller training costs. Notably, our largest 5.0B-parameter EVA-02-CLIP-E/14+ with only 9 billion seen samples achieves 82.0 zero-shot top-1 accuracy on ImageNet-1K val. A smaller EVA-02-CLIP-L/14+ with only 430 million parameters and 6 billion seen samples achieves 80.4 zero-shot top-1 accuracy on ImageNet-1K val. To facilitate open access and open research, we release the complete suite of EVA-CLIP to the community at https://github.com/baaivision/EVA/tree/master/EVA-CLIP.

arXiv commentTo Rei and the moon. Code & Models: https://github.com/baaivision/EVA/tree/master/EVA-CLIP

Cited by in corpus (9)

• A Survey on Multimodal Large Language Models
• EVA-02: A Visual Representation for Neon Genesis
• CLIP in Medical Imaging: A Survey
• UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobility
• AstroCLIP: A Cross-Modal Foundation Model for Galaxies
• MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity
• GABInsight: Exploring Gender-Activity Binding Bias in Vision-Language Models
• Temporal Image Caption Retrieval Competition -- Description and Results
• A Matter of Time: Revealing the Structure of Time in Vision-Language Models