116 citations · 272 across the 19 of their papers we have counts for
20 papers
Long-Horizon Language Model Reinforcement Learning via Progressive Point Matching
Preston Fu, Kevin Frans, Oleh Rybkin +2
Current paradigms for training language models via reinforcement learning rely heavily on sparse outcome rewards. However, as we pursue tasks that require longer and more complicat…
Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning
Zhiyuan Zhou, Andy Peng, Charles Xu +4
Expressive continuous control policies, such as diffusion and flow models, form the backbone of recent advances in scaling imitation learning for simulated and real robot control.…
Composer 2 Technical Report
Cursor Research, :, Aaron Chan +53
Composer 2 is a specialized model designed for agentic software engineering. The model demonstrates strong long-term planning and coding intelligence while maintaining the ability…
What Really Matters in Matrix-Whitening Optimizers?
Kevin Frans, Pieter Abbeel, Sergey Levine
A range of recent optimizers have emerged that approximate the same "matrix-whitening" transformation in various ways. In this work, we systematically deconstruct such optimizers,…
A Stable Whitening Optimizer for Efficient Neural Network Training
Kevin Frans, Sergey Levine, Pieter Abbeel
In this work, we take an experimentally grounded look at neural network optimization. Building on the Shampoo family of algorithms, we identify and alleviate three key issues, resu…
Horizon Reduction Makes RL Scalable
Seohong Park, Kevin Frans, Deepinder Mann +3
In this work, we study the scalability of offline reinforcement learning (RL) algorithms. In principle, a truly scalable offline RL algorithm should be able to solve any given prob…
Diffusion Guidance Is a Controllable Policy Improvement Operator
Kevin Frans, Seohong Park, Pieter Abbeel +1
At the core of reinforcement learning is the idea of learning beyond the performance in the data. However, scaling such systems has proven notoriously tricky. In contrast, techniqu…
OGBench: Benchmarking Offline Goal-Conditioned RL
Seohong Park, Kevin Frans, Benjamin Eysenbach +1
Offline goal-conditioned reinforcement learning (GCRL) is a major problem in reinforcement learning (RL) because it provides a simple, unsupervised, and domain-agnostic way to acqu…
Prioritized Generative Replay
Renhao Wang, Kevin Frans, Pieter Abbeel +2
Sample-efficient online reinforcement learning often uses replay buffers to store experience for reuse when updating the value function. However, uniform replay is inefficient, sin…
Leveraging Skills from Unlabeled Prior Data for Efficient Online Exploration
Max Wilcoxson, Qiyang Li, Kevin Frans +1
Unsupervised pretraining has been transformative in many supervised domains. However, applying such ideas to reinforcement learning (RL) presents a unique challenge in that fine-tu…
One Step Diffusion via Shortcut Models
Kevin Frans, Danijar Hafner, Sergey Levine +1
Diffusion models and flow-matching models have enabled generating diverse and realistic images by learning to transfer noise to data. However, sampling from these models involves i…
Is Value Learning Really the Main Bottleneck in Offline RL?
Seohong Park, Kevin Frans, Sergey Levine +1
While imitation learning requires access to high-quality data, offline reinforcement learning (RL) should, in principle, perform similarly or better with substantially lower data q…
Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodings
Kevin Frans, Seohong Park, Pieter Abbeel +1
Can we pre-train a generalist agent from a large amount of unlabeled offline trajectories such that it can be immediately adapted to any new downstream tasks in a zero-shot manner?…
Powderworld: A Platform for Understanding Generalization via Rich Task Distributions
Kevin Frans, Phillip Isola
One of the grand challenges of reinforcement learning is the ability to generalize to new tasks. However, general agents require a set of rich, diverse tasks to train on. Designing…
CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders
Kevin Frans, L. B. Soros, Olaf Witkowski
This work presents CLIPDraw, an algorithm that synthesizes novel drawings based on natural language input. CLIPDraw does not require any training; rather a pre-trained CLIP languag…
Selecting for Selection: Learning To Balance Adaptive and Diversifying Pressures in Evolutionary Search
Kevin Frans, L. B. Soros, Olaf Witkowski
Inspired by natural evolution, evolutionary search algorithms have proven remarkably capable due to their dual abilities to radiantly explore through diverse populations and to con…
Population-Based Evolution Optimizes a Meta-Learning Objective
Kevin Frans, Olaf Witkowski
Meta-learning models, or models that learn to learn, have been a long-desired target for their ability to quickly solve new tasks. Traditional meta-learning methods can require exp…
Unsupervised Image to Sequence Translation with Canvas-Drawer Networks
Kevin Frans, Chin-Yi Cheng
Encoding images as a series of high-level constructs, such as brush strokes or discrete shapes, can often be key to both human and machine understanding. In many cases, however, da…
Meta Learning Shared Hierarchies
Kevin Frans, Jonathan Ho, Xi Chen +2
We develop a metalearning approach for learning hierarchically structured policies, improving sample efficiency on unseen tasks through the use of shared primitives---policies that…
Outline Colorization through Tandem Adversarial Networks
Kevin Frans
When creating digital art, coloring and shading are often time consuming tasks that follow the same general patterns. A solution to automatically colorize raw line art would have m…