5 citations · 20 across the 12 of their papers we have counts for
12 papers
Adaptive Coordination in Social Embodied Rearrangement
Andrew Szot, Unnat Jain, Dhruv Batra +3
We present the task of "Social Rearrangement", consisting of cooperative everyday tasks like setting up the dinner table, tidying a house or unpacking groceries in a simulated mult…
Pretrained Language Models as Visual Planners for Human Assistance
Dhruvesh Patel, Hamid Eghbalzadeh, Nitin Kamra +3
In our pursuit of advancing multi-modal AI assistants capable of guiding users to achieve complex multi-step goals, we propose the task of "Visual Planning for Assistance (VPA)". G…
Effective Baselines for Multiple Object Rearrangement Planning in Partially Observable Mapped Environments
Engin Tekin, Elaheh Barati, Nitin Kamra +1
Many real-world tasks, from house-cleaning to cooking, can be formulated as multi-object rearrangement problems -- where an agent needs to get specific objects into appropriate goa…
Action Dynamics Task Graphs for Learning Plannable Representations of Procedural Tasks
Weichao Mao, Ruta Desai, Michael Louis Iuzzolino +1
Given video demonstrations and paired narrations of an at-home procedural task such as changing a tire, we present an approach to extract the underlying task structure -- relevant…
Cross-Domain Transfer via Semantic Skill Imitation
Karl Pertsch, Ruta Desai, Vikash Kumar +4
We propose an approach for semantic imitation, which uses demonstrations from a source domain, e.g. human videos, to accelerate reinforcement learning (RL) in a different target do…
Learning a Visually Grounded Memory Assistant
Meera Hahn, Kevin Carlberg, Ruta Desai +1
We introduce a novel interface for large scale collection of human memory and assistance. Using the 3D Matterport simulator we create a realistic indoor environments in which we ha…
EgoEnv: Human-centric environment representations from egocentric video
Tushar Nagarajan, Santhosh Kumar Ramakrishnan, Ruta Desai +2
First-person video highlights a camera-wearer's activities in the context of their persistent environment. However, current video understanding approaches reason over visual featur…
Episodic Memory Question Answering
Samyak Datta, Sameer Dharur, Vincent Cartillier +4
Egocentric augmented reality devices such as wearable glasses passively capture visual data as a human wearer tours a home environment. We envision a scenario wherein the human com…
How You Move Your Head Tells What You Do: Self-supervised Video Representation Learning with Egocentric Cameras and IMU Sensors
Satoshi Tsutsui, Ruta Desai, Karl Ridgeway
Understanding users' activities from head-mounted cameras is a fundamental task for Augmented and Virtual Reality (AR/VR) applications. A typical approach is to train a classifier…
Optimal Assistance for Object-Rearrangement Tasks in Augmented Reality
Benjamin Newman, Kevin Carlberg, Ruta Desai
Augmented-reality (AR) glasses that will have access to onboard sensors and an ability to display relevant information to the user present an opportunity to provide user assistance…
Automatic Design of Task-specific Robotic Arms
Ruta Desai, Margarita Safonova, Katharina Muelling +1
We present an interactive, computational design system for creating custom robotic arms given high-level task descriptions and environmental constraints. Various task requirements…
Interactive Co-Design of Form and Function for Legged Robots using the Adjoint Method
Ruta Desai, Beichen Li, Ye Yuan +1
Our goal is to make robotics more accessible to casual users by reducing the domain knowledge required in designing and building robots. Towards this goal, we present an interactiv…