12 papers
Milo, a Fully Autonomous Indoor/Outdoor Robotic Guide Dog
Florian Golemo, Joanna Wolski, Joel Ruben Antony Moniz +1
Many Blind and Low-Vision (BLV) people rely on guide dogs for moment-to-moment navigation, such as staying on path and avoiding obstacles and pedestrians. However, guide dogs are e…
Scaling Self-Play for End-to-End Driving
Luke Rowe, Roger Girgis, Rodrigue de Schaetzen +6
End-to-end autonomous driving models are typically trained on offline human-demonstration datasets that provide limited state coverage and often no closed-loop feedback, making the…
Constrained Group Relative Policy Optimization
Roger Girgis, Rodrigue de Schaetzen, Luke Rowe +3
Group Relative Policy Optimization (GRPO) remains the dominant critic-free approach for fine-tuning LLMs and VLMs, but its compatibility with constrained policy optimization (e.g.…
Neural Coherence : Find higher performance to out-of-distribution tasks from few samples
Simon Guiroy, Mats Richter, Sarath Chandar +1
To create state-of-the-art models for many downstream tasks, it has become common practice to fine-tune a pre-trained large vision model. However, it remains an open question of ho…
Spatio-Temporal Conditional Diffusion Models for Forecasting Future Multiple Sclerosis Lesion Masks Conditioned on Treatments
Gian Mario Favero, Ge Ya Luo, Nima Fathi +5
Image-based personalized medicine has the potential to transform healthcare, particularly for diseases that exhibit heterogeneous progression such as Multiple Sclerosis (MS). In th…
Poutine: Vision-Language-Trajectory Pre-Training and Reinforcement Learning Post-Training Enable Robust End-to-End Autonomous Driving
Luke Rowe, Rodrigue de Schaetzen, Roger Girgis +2
Maintaining good driving behavior in out-of-distribution scenarios remains a critical challenge in autonomous driving. A promising direction is to leverage the generalist knowledge…
Ctrl-Crash: Controllable Diffusion for Realistic Car Crashes
Anthony Gosselin, Ge Ya Luo, Luis Lara +5
Video diffusion techniques have advanced significantly in recent years; however, they struggle to generate realistic imagery of car crashes due to the scarcity of accident events i…
Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environments
Luke Rowe, Roger Girgis, Anthony Gosselin +3
We introduce Scenario Dreamer, a fully data-driven generative simulator for autonomous vehicle planning that generates both the initial traffic scene - comprising a lane graph and…
Beyond FVD: Enhanced Evaluation Metrics for Video Generation Quality
Ge Ya Luo, Gian Mario Favero, Zhi Hao Luo +2
The Fréchet Video Distance (FVD) is a widely adopted metric for evaluating video generation distribution quality. However, its effectiveness relies on critical assumptions. Our ana…
Ctrl-V: Higher Fidelity Video Generation with Bounding-Box Controlled Object Motion
Ge Ya Luo, Zhi Hao Luo, Anthony Gosselin +2
Controllable video generation has attracted significant attention, largely due to advances in video diffusion models. In domains such as autonomous driving, it is essential to deve…
CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning
Luke Rowe, Roger Girgis, Anthony Gosselin +5
Evaluating autonomous vehicle stacks (AVs) in simulation typically involves replaying driving logs from real-world recorded traffic. However, agents replayed from offline data are…
Reinforcement Learning for Blind Stair Climbing with Legged and Wheeled-Legged Robots
Simon Chamorro, Victor Klemm, Miguel de la Iglesia Valls +2
In recent years, legged and wheeled-legged robots have gained prominence for tasks in environments predominantly created for humans across various domains. One significant challeng…