Learning to Manipulate Object Collections Using Grounded State Representations

Matthew Wilson, Tucker Hermans

We propose a method for sim-to-real robot learning which exploits simulator state information in a way that scales to many objects. We first train a pair of encoder networks to cap…