4 papers
Continually self-improving AI
Zitong Yang
Modern language model-based AI systems are remarkably powerful, yet their capabilities remain fundamentally capped by their human creators in three key ways. First, although a mode…
Towards Execution-Grounded Automated AI Research
Chenglei Si, Zitong Yang, Yejin Choi +3
Automated AI research holds great potential to accelerate scientific discovery. However, current LLMs often generate plausible-looking but ineffective ideas. Execution grounding ma…
Synthetic bootstrapped pretraining
Zitong Yang, Aonan Zhang, Hong Liu +4
We introduce Synthetic Bootstrapped Pretraining (SBP), a language model (LM) pretraining procedure that first learns a model of relations between documents from the pretraining dat…
s1: Simple test-time scaling
Niklas Muennighoff, Zitong Yang, Weijia Shi +7
Test-time scaling is a promising new approach to language modeling that uses extra test-time compute to improve performance. Recently, OpenAI's o1 model showed this capability but…