11 citations · 31 across the 16 of their papers we have counts for
5 papers · 1 filter
LogitTrace: Detecting Benchmark Contamination via Layerwise Logit Trajectories
Zirui He, Haiyan Zhao, Yingcong Li +2
Large language models (LLMs) are commonly evaluated on challenging benchmarks such as AIME and Math500, where benchmark contamination can make memorized solutions appear as genuine…
BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoning
Xuechen Zhang, Zijian Huang, Yingcong Li +3
Small language models (SLMs) struggle to learn complex reasoning behaviors, especially when high-quality traces are scarce or difficult to learn from. The standard training approac…
When and How Unlabeled Data Provably Improve In-Context Learning
Yingcong Li, Xiangyu Chang, Muti Kara +3
Recent research shows that in-context learning (ICL) can be effective even when demonstrations have missing or incorrect labels. To shed light on this capability, we examine a cano…
Gating is Weighting: Understanding Gated Linear Attention through In-context Learning
Yingcong Li, Davoud Ataee Tarzanagh, Ankit Singh Rawat +2
Linear attention methods offer a compelling alternative to softmax attention due to their efficiency in recurrent decoding. Recent research has focused on enhancing standard linear…
Provable Benefits of Task-Specific Prompts for In-context Learning
Xiangyu Chang, Yingcong Li, Muti Kara +2
The in-context learning capabilities of modern language models have motivated a deeper mathematical understanding of sequence models. A line of recent work has shown that linear at…