Grounding Complex Navigational Instructions Using Scene Graphs

Michiel de Jong, Satyapriya Krishna, Anuva Agarwal

Training a reinforcement learning agent to carry out natural language instructions is limited by the available supervision, i.e. knowing when the instruction has been carried out.…