10 citations · 19 across the 12 of their papers we have counts for
10 papers · 1 filter
Beyond What Seems Necessary: Hidden Gains from Scaling Training-Time Reasoning Length under Outcome Supervision
Yihao Xue, Allan Zhang, Jianhao Huang +2
Training LLMs to think and reason for longer has become a key ingredient in building state-of-the-art models that can solve complex problems previously out of reach. Recent efforts…
Representations Shape Weak-to-Strong Generalization: Theoretical Insights and Empirical Predictions
Yihao Xue, Jiping Li, Baharan Mirzasoleiman
Weak-to-Strong Generalization (W2SG), where a weak model supervises a stronger one, serves as an important analogy for understanding how humans might guide superhuman intelligence…
Investigating the Benefits of Projection Head for Representation Learning
Yihao Xue, Eric Gan, Jiayi Ni +2
An effective technique for obtaining high-quality representations is adding a projection head on top of the encoder during training, then discarding it and using the pre-projection…
Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shift
Yihao Xue, Siddharth Joshi, Dang Nguyen +1
Recently, multimodal contrastive learning (MMCL) approaches, such as CLIP, have achieved a remarkable success in learning representations that are robust against distribution shift…
Challenges and Opportunities in Improving Worst-Group Generalization in Presence of Spurious Features
Siddharth Joshi, Yu Yang, Yihao Xue +2
Deep neural networks often exploit *spurious* features that are present in the majority of examples within a class during training. This leads to *poor worst-group test accuracy*,…
Which Features are Learnt by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppression
Yihao Xue, Siddharth Joshi, Eric Gan +2
Contrastive learning (CL) has emerged as a powerful technique for representation learning, with or without label supervision. However, supervised CL is prone to collapsing represen…
Few-shot Adaptation to Distribution Shifts By Mixing Source and Target Embeddings
Yihao Xue, Ali Payani, Yu Yang +1
Pretrained machine learning models need to be adapted to distribution shifts when deployed in new target environments. When obtaining labeled data from the target distribution is e…
Investigating the Impact of Model Width and Density on Generalization in Presence of Label Noise
Yihao Xue, Kyle Whitecross, Baharan Mirzasoleiman
Increasing the size of overparameterized neural networks has been a key in achieving state-of-the-art performance. This is captured by the double descent phenomenon, where the test…
Investigating Why Contrastive Learning Benefits Robustness Against Label Noise
Yihao Xue, Kyle Whitecross, Baharan Mirzasoleiman
Self-supervised Contrastive Learning (CL) has been recently shown to be very effective in preventing deep networks from overfitting noisy labels. Despite its empirical success, the…
Toward Understanding the Influence of Individual Clients in Federated Learning
Yihao Xue, Chaoyue Niu, Zhenzhe Zheng +4
Federated learning allows mobile clients to jointly train a global model without sending their private data to a central server. Extensive works have studied the performance guaran…