6 papers · 1 filter
AnyRank: Reliable Anytime Policy Ranking via Sequential Testing
Apurva Badithela, David Snyder, Tenny Yin +6
Due to the high cost of hardware experiments, robot policies are predominantly evaluated by comparing their empirical performance on a limited number of hardware trials. However, f…
Beyond Binary Success: Sample-Efficient and Statistically Rigorous Robot Policy Comparison
David Snyder, Apurva Badithela, Nikolai Matni +4
Generalist robot manipulation policies are becoming increasingly capable, but are limited in evaluation to a small number of hardware rollouts. This strong resource constraint in r…
Contextual Safety Reasoning and Grounding for Open-World Robots
Zachary Ravichandran, David Snyder, Alexander Robey +3
Robots are increasingly operating in open-world environments where safe behavior depends on context: the same hallway may require different navigation strategies when crowded versu…
Reliable and Scalable Robot Policy Evaluation with Imperfect Simulators
Apurva Badithela, David Snyder, Lihan Zha +4
Rapid progress in imitation learning, foundation models, and large-scale datasets has led to robot manipulation policies that generalize to a wide-range of tasks and environments.…
Guiding Data Collection via Factored Scaling Curves
Lihan Zha, Apurva Badithela, Michael Zhang +7
Generalist imitation learning policies trained on large datasets show great promise for solving diverse manipulation tasks. However, to ensure generalization to different condition…
Is Your Imitation Learning Policy Better than Mine? Policy Comparison with Near-Optimal Stopping
David Snyder, Asher James Hancock, Apurva Badithela +6
Imitation learning has enabled robots to perform complex, long-horizon tasks in challenging dexterous manipulation settings. As new methods are developed, they must be rigorously e…