6 papers · 1 filter
Betting for Sim-to-Real Performance Certificates
Yujia Chen, Bowen Weng
Consider a typical test of a robot system: one observes a sequence of outcomes concerning some aspect of interest (crash or no crash, tracking error, time to completion), and repor…
Sim-to-Real Betting on the E-Process: Bringing "simulators" to anytime-valid confidence sequences
Yujia Chen, Bowen Weng
This note describes an integration of the sim-to-real performance estimate with betting (from Chen et al.) and the safe anytime-valid inference (from Ramdas et al.). Using the scal…
Betting for Sim-to-Real Performance Evaluation
Zaid Mahboob, Yujia Chen, Bowen Weng
This paper studies the problem of robot performance evaluation, focusing on how to obtain accurate and efficient estimates of real-world behavior under severe constraints on physic…
Rethink Repeatable Measures of Robot Performance with Statistical Query
Bowen Weng, Linda Capito, Guillermo A. Castillo +1
For a general standardized testing algorithm designed to evaluate a specific aspect of a robot's performance, several key expectations are commonly imposed. Beyond accuracy (i.e.,…
Post-Convergence Sim-to-Real Policy Transfer: A Principled Alternative to Cherry-Picking
Dylan Khor, Bowen Weng
Learning-based approaches, particularly reinforcement learning (RL), have become widely used for developing control policies for autonomous agents, such as locomotion policies for…
Repeatable and Reliable Efforts of Accelerated Risk Assessment in Robot Testing
Linda Capito, Guillermo A. Castillo, Bowen Weng
Risk assessment of a robot in controlled environments, such as laboratories and proving grounds, is a common means to assess, certify, validate, verify, and characterize the robots…