2 citations · 3 across the 21 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.CL2024
How to Synthesize Text Data without Model Collapse?
Xuekai Zhu, Daixuan Cheng, Hengli Li +7
Model collapse in synthetic data indicates that iterative training on self-generated data leads to a gradual decline in performance. With the proliferation of AI models, synthetic…
cs.CL2024★ 2 cited
Critical Data Size of Language Models from a Grokking Perspective
Xuekai Zhu, Yao Fu, Bowen Zhou +1
We explore the critical data size in language models, a threshold that marks a fundamental shift from quick memorization to slow generalization. We formalize the phase transition u…