A Champion-level Vision-based Reinforcement Learning Agent for Competitive Racing in Gran Turismo 7

Abstract

Deep reinforcement learning has achieved superhuman racing performance in high-fidelity simulators like Gran Turismo 7 (GT7). It typically utilizes global features that require instrumentation external to a car, such as precise localization of agents and opponents, limiting real-world applicability. To address this limitation, we introduce a vision-based autonomous racing agent that relies solely on ego-centric camera views and onboard sensor data, eliminating the need for precise localization during inference. This agent employs an asymmetric actor-critic framework: the actor uses a recurrent neural network with the sensor data local to the car to retain track layouts and opponent positions, while the critic accesses the global features during training. Evaluated in GT7, our agent consistently outperforms GT7's built-drivers. To our knowledge, this work presents the first vision-based autonomous racing agent to demonstrate champion-level performance in competitive racing scenarios.

Accepted for Publication at the IEEE Robotics and Automation Letters (RA-L) 2025

References in corpus (8)

• Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning
• Controlling an Autonomous Vehicle with Deep Reinforcement Learning
• Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning
• Motion Planning and Control for Multi Vehicle Autonomous Racing at High Speeds
• Minimum Race-Time Planning-Strategy for an Autonomous Electric Racecar
• Formula RL: Deep Reinforcement Learning for Autonomous Racing using Telemetry Data
• Learning Model Predictive Control with Error Dynamics Regression for Autonomous Racing
• A Super-human Vision-based Reinforcement Learning Agent for Autonomous Racing in Gran Turismo