19 citations · 88 across the 27 of their papers we have counts for
29 papers
Slow Brain, Fast Planner: Latency-Resilient VLM-Augmented Urban Navigation
Zhenghao "Mark'' Peng, Honglin He, Quanyi Li +2
Learning-based planners for sidewalk navigation can generate diverse candidate trajectories in real time, yet their scoring functions often fail to select the best trajectory in ch…
Predictive Preference Learning from Human Interventions
Haoyuan Cai, Zhenghao Peng, Bolei Zhou
Learning from human involvement aims to incorporate the human subject to monitor and correct agent behavior errors. Although most interactive imitation learning methods focus on co…
Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism
Haoyuan Cai, Zhenghao Peng, Bolei Zhou
Interactive Imitation Learning (IIL) allows agents to acquire desired behaviors through human interventions, but current methods impose high cognitive demands on human supervisors.…
SceneStreamer: Continuous Scenario Generation as Next Token Group Prediction
Zhenghao Peng, Yuxin Liu, Bolei Zhou
Realistic and interactive traffic simulation is essential for training and evaluating autonomous driving systems. However, most existing data-driven simulation methods rely on stat…
Adv-BMT: Bidirectional Motion Transformer for Safety-Critical Traffic Scenario Generation
Yuxin Liu, Zhenghao Peng, Xuanhao Cui +1
Scenario-based testing is essential for validating the performance of autonomous driving (AD) systems. However, such testing is limited by the scarcity of long-tailed, safety-criti…
Data-Efficient Learning from Human Interventions for Mobile Robots
Zhenghao Peng, Zhizheng Liu, Bolei Zhou
Mobile robots are essential in applications such as autonomous delivery and hospitality services. Applying learning-based methods to address mobile robot tasks has gained popularit…
Learning from Active Human Involvement through Proxy Value Propagation
Zhenghao Peng, Wenjie Mo, Chenda Duan +2
Learning from active human involvement enables the human subject to actively intervene and demonstrate to the AI agent during training. The interaction and corrective feedback from…
Embodied Scene Understanding for Vision Language Models via MetaVQA
Weizhen Wang, Chenda Duan, Zhenghao Peng +2
Vision Language Models (VLMs) demonstrate significant potential as embodied AI agents for various mobility applications. However, a standardized, closed-loop benchmark for evaluati…
Vid2Sim: Realistic and Interactive Simulation from Video for Urban Navigation
Ziyang Xie, Zhizheng Liu, Zhenghao Peng +2
Sim-to-real gap has long posed a significant challenge for robot learning in simulation, preventing the deployment of learned models in the real world. Previous work has primarily…
Shared Autonomy with IDA: Interventional Diffusion Assistance
Brandon J. McMahan, Zhenghao Peng, Bolei Zhou +1
The rapid development of artificial intelligence (AI) has unearthed the potential to assist humans in controlling advanced technologies. Shared autonomy (SA) facilitates control by…
SimGen: Simulator-conditioned Driving Scene Generation
Yunsong Zhou, Michael Simon, Zhenghao Peng +4
Controllable synthetic data generation can substantially lower the annotation cost of training data. Prior works use diffusion models to generate driving images conditioned on the…
SwinTextSpotter v2: Towards Better Synergy for Scene Text Spotting
Mingxin Huang, Dezhi Peng, Hongliang Li +6
End-to-end scene text spotting, which aims to read the text in natural images, has garnered significant attention in recent years. However, recent state-of-the-art methods usually…
CAT: Closed-loop Adversarial Training for Safe End-to-End Driving
Linrui Zhang, Zhenghao Peng, Quanyi Li +1
Driving safety is a top priority for autonomous vehicles. Orthogonal to prior work handling accident-prone traffic events by algorithm designs at the policy level, we investigate a…
ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modeling
Quanyi Li, Zhenghao Peng, Lan Feng +4
Large-scale driving datasets such as Waymo Open Dataset and nuScenes substantially accelerate autonomous driving research, especially for perception tasks such as 3D detection and…
Guarded Policy Optimization with Imperfect Online Demonstrations
Zhenghai Xue, Zhenghao Peng, Quanyi Li +2
The Teacher-Student Framework (TSF) is a reinforcement learning setting where a teacher agent guards the training of a student agent by intervening and providing online demonstrati…
TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarios
Lan Feng, Quanyi Li, Zhenghao Peng +2
Diverse and realistic traffic scenarios are crucial for evaluating the AI safety of autonomous driving systems in simulation. This work introduces a data-driven method called Traff…
Human-AI Shared Control via Policy Dissection
Quanyi Li, Zhenghao Peng, Haibin Wu +2
Human-AI shared control allows human to interact and collaborate with AI to accomplish control tasks in complex environments. Previous Reinforcement Learning (RL) methods attempt t…
Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretraining
Qihang Zhang, Zhenghao Peng, Bolei Zhou
Deep visuomotor policy learning, which aims to map raw visual observation to action, achieves promising results in control tasks such as robotic manipulation and autonomous driving…
SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition
Mingxin Huang, Yuliang Liu, Zhenghao Peng +6
End-to-end scene text spotting has attracted great attention in recent years due to the success of excavating the intrinsic synergy of the scene text detection and recognition. How…
Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization
Quanyi Li, Zhenghao Peng, Bolei Zhou
Human intervention is an effective way to inject human knowledge into the training loop of reinforcement learning, which can bring fast learning and ensured training safety. Given…
Safe Driving via Expert Guided Policy Optimization
Zhenghao Peng, Quanyi Li, Chunxiao Liu +1
When learning common skills like driving, beginners usually have domain experts standing by to ensure the safety of the learning process. We formulate such learning scheme under th…
Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimization
Zhenghao Peng, Quanyi Li, Ka Ming Hui +2
Self-Driven Particles (SDP) describe a category of multi-agent systems common in everyday life, such as flocking birds and traffic flows. In a SDP system, each agent pursues its ow…
MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning
Quanyi Li, Zhenghao Peng, Lan Feng +3
Driving safely requires multiple capabilities from human and intelligent agents, such as the generalizability to unseen environments, the safety awareness of the surrounding traffi…
Safe Exploration by Solving Early Terminated MDP
Hao Sun, Ziping Xu, Meng Fang +4
Safe exploration is crucial for the real-world application of reinforcement learning (RL). Previous works consider the safe exploration problem as Constrained Markov Decision Proce…