5 citations · 5 across the 11 of their papers we have counts for
5 papers · 1 filter
Recursive Language Models Generalize Out of Domain
Chenxiao Yang, Zhiyuan Li, David McAllester +1
We study when limiting what a language model can see improves learning. We compare standard CoT, the more general learner that reads the full trace, with recursive language models,…
Tight Sample Complexity of Transformers
Chenxiao Yang, Nathan Srebro, Zhiyuan Li
We tightly characterize the VC dimension of depth- Transformers with a total of parameters, mapping an input sequence of length to a single output, establishing an upper…
Hierarchical Domain Generalization
Chenxiao Yang, Zhiyuan Li, Shai Ben-David +1
We study hierarchical domain generalization as a problem of extrapolation from finite observed regions to an entire instance space, replacing i.i.d. sampling with arbitrary domain…
Training Transformers as a Universal Computer
Ruize Xu, Chenxiao Yang, Yanhong Li +1
We demonstrate that a small transformer can learn to execute programs in MicroPy, a simplified yet computationally universal programming language. Given procedure definitions toget…
Recursive Models for Long-Horizon Reasoning
Chenxiao Yang, Nathan Srebro, Zhiyuan Li
Modern language models reason within bounded context, an inherent constraint that poses a fundamental barrier to long-horizon reasoning. We identify recursion as a core principle f…