7 papers
Betting for Sim-to-Real Performance Certificates
Yujia Chen, Bowen Weng
Consider a typical test of a robot system: one observes a sequence of outcomes concerning some aspect of interest (crash or no crash, tracking error, time to completion), and repor…
Sim-to-Real Betting on the E-Process: Bringing "simulators" to anytime-valid confidence sequences
Yujia Chen, Bowen Weng
This note describes an integration of the sim-to-real performance estimate with betting (from Chen et al.) and the safe anytime-valid inference (from Ramdas et al.). Using the scal…
Betting for Sim-to-Real Performance Evaluation
Zaid Mahboob, Yujia Chen, Bowen Weng
This paper studies the problem of robot performance evaluation, focusing on how to obtain accurate and efficient estimates of real-world behavior under severe constraints on physic…
Aligning Microscopic Vehicle and Macroscopic Traffic Statistics: Reconstructing Driving Behavior from Partial Data
Zhihao Zhang, Keith Redmill, Chengyang Peng +1
A driving algorithm that aligns with good human driving practices, or at the very least collaborates effectively with human drivers, is crucial for developing safe and efficient au…
Rethink Repeatable Measures of Robot Performance with Statistical Query
Bowen Weng, Linda Capito, Guillermo A. Castillo +1
For a general standardized testing algorithm designed to evaluate a specific aspect of a robot's performance, several key expectations are commonly imposed. Beyond accuracy (i.e.,…
Post-Convergence Sim-to-Real Policy Transfer: A Principled Alternative to Cherry-Picking
Dylan Khor, Bowen Weng
Learning-based approaches, particularly reinforcement learning (RL), have become widely used for developing control policies for autonomous agents, such as locomotion policies for…