6 papers
Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedback
Hiroki Furuta, Heiga Zen, Dale Schuurmans +4
Large text-to-video models hold immense potential for a wide range of downstream applications. However, they struggle to accurately depict dynamic object interactions, often result…
MLE-Smith: Scaling MLE Tasks with Automated Multi-Agent Pipeline
Rushi Qiang, Yuchen Zhuang, Anikait Singh +4
While Language Models (LMs) have made significant progress in automating machine learning engineering (MLE), the acquisition of high-quality MLE training data is significantly cons…
WorldGym: World Model as An Environment for Policy Evaluation
Julian Quevedo, Ansh Kumar Sharma, Yixiang Sun +3
Evaluating robot control policies is difficult: real-world testing is costly, and handcrafted simulators require manual effort to improve in realism and generality. We propose a wo…
Reinforcement Learning for Machine Learning Engineering Agents
Sherry Yang, Joy He-Yueya, Percy Liang
Existing agents for solving tasks such as ML engineering rely on prompting powerful language models. As a result, these agents do not improve with more experience. In this paper, w…
MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering
Rushi Qiang, Yuchen Zhuang, Yinghao Li +8
We introduce MLE-Dojo, a Gym-style framework for systematically reinforcement learning, evaluating, and improving autonomous large language model (LLM) agents in iterative machine…
VideoAgent: Self-Improving Video Generation
Achint Soni, Sreyas Venkataraman, Abhranil Chandra +4
Video generation has been used to generate visual plans for controlling robotic systems. Given an image observation and a language instruction, previous work has generated video pl…