7 papers
ReGen: Generative Robot Simulation via Inverse Design
Phat Nguyen, Tsun-Hsuan Wang, Zhang-Wei Hong +5
Simulation plays a key role in scaling robot learning and validating policies, but constructing simulations remains a labor-intensive process. This paper introduces ReGen, a genera…
Surgical Foundation Model Leveraging Compression and Entropy Maximization for Image-Guided Surgical Assistance
Lianhao Yin, Ozanan Meireles, Guy Rosman +1
Real-time video understanding is critical to guide procedures in minimally invasive surgery (MIS). However, supervised learning approaches require large, annotated datasets that ar…
Generating Out-Of-Distribution Scenarios Using Language Models
Erfan Aasi, Phat Nguyen, Shiva Sreeram +3
The deployment of autonomous vehicles controlled by machine learning techniques requires extensive testing in diverse real-world environments, robust handling of edge cases and out…
Learning autonomous driving from aerial imagery
Varun Murali, Guy Rosman, Sertac Karaman +1
In this work, we consider the problem of learning end to end perception to control for ground vehicles solely from aerial imagery. Photogrammetric simulators allow the synthesis of…
Text-to-Drive: Diverse Driving Behavior Synthesis via Large Language Models
Phat Nguyen, Tsun-Hsuan Wang, Zhang-Wei Hong +2
Generating varied scenarios through simulation is crucial for training and evaluating safety-critical systems, such as autonomous vehicles. Yet, the task of modeling the trajectori…
Probing Multimodal LLMs as World Models for Driving
Shiva Sreeram, Tsun-Hsuan Wang, Alaa Maalouf +3
We provide a sober look at the application of Multimodal Large Language Models (MLLMs) in autonomous driving, challenging common assumptions about their ability to interpret dynami…
Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgery
Lianhao Yin, Yutong Ban, Jennifer Eckhoff +3
Understanding and anticipating intraoperative events and actions is critical for intraoperative assistance and decision-making during minimally invasive surgery. Automated predicti…