5 papers
Learning a Visually Grounded Memory Assistant
Meera Hahn, Kevin Carlberg, Ruta Desai +1
We introduce a novel interface for large scale collection of human memory and assistance. Using the 3D Matterport simulator we create a realistic indoor environments in which we ha…
Episodic Memory Question Answering
Samyak Datta, Sameer Dharur, Vincent Cartillier +4
Egocentric augmented reality devices such as wearable glasses passively capture visual data as a human wearer tours a home environment. We envision a scenario wherein the human com…
How You Move Your Head Tells What You Do: Self-supervised Video Representation Learning with Egocentric Cameras and IMU Sensors
Satoshi Tsutsui, Ruta Desai, Karl Ridgeway
Understanding users' activities from head-mounted cameras is a fundamental task for Augmented and Virtual Reality (AR/VR) applications. A typical approach is to train a classifier…
Optimal Assistance for Object-Rearrangement Tasks in Augmented Reality
Benjamin Newman, Kevin Carlberg, Ruta Desai
Augmented-reality (AR) glasses that will have access to onboard sensors and an ability to display relevant information to the user present an opportunity to provide user assistance…
Automatic Design of Task-specific Robotic Arms
Ruta Desai, Margarita Safonova, Katharina Muelling +1
We present an interactive, computational design system for creating custom robotic arms given high-level task descriptions and environmental constraints. Various task requirements…