8 papers
Statistical Consistency and Generalization of Contrastive Representation Learning
Yuanfan Li, Xiyuan Wei, Tianbao Yang +1
Contrastive representation learning (CRL) underpins many modern foundation models. Despite recent theoretical progress, existing analyses suffer from several key limitations: (i) t…
A Geometry-Aware Efficient Algorithm for Compositional Entropic Risk Minimization
Xiyuan Wei, Linli Zhou, Bokun Wang +2
This paper studies optimization for a family of problems termed , in which each data's loss is formulated as a Log-Expectation-Ex…
Breaking the Limits of Open-Weight CLIP: An Optimization Framework for Self-supervised Fine-tuning of CLIP
Anant Mehta, Xiyuan Wei, Xingyu Chen +1
CLIP has become a cornerstone of multimodal representation learning, yet improving its performance typically requires a prohibitively costly process of training from scratch on bil…
NeuCLIP: Efficient Large-Scale CLIP Training with Neural Normalizer Optimization
Xiyuan Wei, Chih-Jen Lin, Tianbao Yang
Accurately estimating the normalization term (also known as the partition function) in the contrastive loss is a central challenge for training Contrastive Language-Image Pre-train…
AdFair-CLIP: Adversarial Fair Contrastive Language-Image Pre-training for Chest X-rays
Chenlang Yi, Zizhan Xiong, Qi Qi +5
Contrastive Language-Image Pre-training (CLIP) models have demonstrated superior performance across various visual tasks including medical image classification. However, fairness c…
Model Steering: Learning with a Reference Model Improves Generalization Bounds and Scaling Laws
Xiyuan Wei, Ming Lin, Fanjiang Ye +4
This paper formalizes an emerging learning paradigm that uses a trained model as a reference to guide and enhance the training of a target model through strategic data selection or…
FastCLIP: A Suite of Optimization Techniques to Accelerate CLIP Training with Limited Resources
Xiyuan Wei, Fanjiang Ye, Ori Yonay +4
Existing studies of training state-of-the-art Contrastive Language-Image Pretraining (CLIP) models on large-scale data involve hundreds of or even thousands of GPUs due to the requ…
Stability and Generalization of Stochastic Compositional Gradient Descent Algorithms
Ming Yang, Xiyuan Wei, Tianbao Yang +1
Many machine learning tasks can be formulated as a stochastic compositional optimization (SCO) problem such as reinforcement learning, AUC maximization, and meta-learning, where th…