4 citations · 11 across the 16 of their papers we have counts for
7 papers · 1 filter
When does Chain-of-Thought Help: A Markovian Perspective
Zihan Wang, Yijun Dong, Qi Lei
Chain-of-Thought (CoT) prompting is a widely used inference-time technique for improving reasoning, yet its gains are uneven across tasks. We analyze when and why CoT helps by mode…
A Task-Centric Theory for Iterative Self-Improvement with Easy-to-Hard Curricula
Chenruo Liu, Yijun Dong, Yiqiu Shen +1
Iterative self-improvement fine-tunes an autoregressive large language model (LLM) on reward-verified outputs generated by the LLM itself. In contrast to the empirical success of s…
Does Weak-to-strong Generalization Happen under Spurious Correlations?
Chenruo Liu, Yijun Dong, Qi Lei
We initiate a unified theoretical and algorithmic study of a key problem in weak-to-strong (W2S) generalization: when fine-tuning a strong pre-trained student with pseudolabels fro…
Discrepancies are Virtue: Weak-to-Strong Generalization through Lens of Intrinsic Dimension
Yijun Dong, Yicheng Li, Yunai Li +2
Weak-to-strong (W2S) generalization is a type of finetuning (FT) where a strong (large) student model is trained on pseudo-labels generated by a weak teacher. Surprisingly, W2S FT…
Sketchy Moment Matching: Toward Fast and Provable Data Selection for Finetuning
Yijun Dong, Hoang Phan, Xiang Pan +1
We revisit data selection in a modern context of finetuning from a fundamental perspective. Extending the classical wisdom of variance minimization in low dimensions to high-dimens…
Randomized Dimension Reduction with Statistical Guarantees
Yijun Dong
Large models and enormous data are essential driving forces of the unprecedented successes achieved by modern algorithms, especially in scientific computing and machine learning. N…
Sample Efficiency of Data Augmentation Consistency Regularization
Shuo Yang, Yijun Dong, Rachel Ward +3
Data augmentation is popular in the training of large neural networks; currently, however, there is no clear theoretical comparison between different algorithmic choices on how to…