2.4k citations · 8.4k across the 129 of their papers we have counts for
16 papers · 1 filter
Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies
Alexander Sax, Bradley Emi, Amir R. Zamir +3
How much does having visual priors about the world (e.g. the fact that the world is 3D) assist in learning to perform downstream motor tasks (e.g. delivering a package)? We study t…
Coupled Recurrent Network (CRN)
Lin Sun, Kui Jia, Yuejia Shen +3
Many semantic video analysis tasks can benefit from multiple, heterogenous signals. For example, in addition to the original RGB input sequences, sequences of optical flow are usua…
RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation
Ajay Mandlekar, Yuke Zhu, Animesh Garg +9
Imitation Learning has empowered recent advances in learning robotic manipulation tasks by addressing shortcomings of Reinforcement Learning such as exploration and reward specific…
Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks
Michelle A. Lee, Yuke Zhu, Krishnan Srinivasan +5
Contact-rich manipulation tasks in unstructured environments often require both haptic and visual feedback. However, it is non-trivial to manually design a robot controller that co…
Translating Navigation Instructions in Natural Language to a High-Level Plan for Behavioral Robot Navigation
Xiaoxue Zang, Ashwini Pokle, Marynel Vázquez +4
We propose an end-to-end deep learning model for translating free-form natural language instructions to a high-level plan for behavioral robot navigation. We use attention models t…
Gibson Env: Real-World Perception for Embodied Agents
Fei Xia, Amir Zamir, Zhi-Yang He +3
Developing visual perception models for active agents and sensorimotor control are cumbersome to be done in the physical world, as existing algorithms are too slow to efficiently l…
Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration
De-An Huang, Suraj Nair, Danfei Xu +5
Our goal is to generate a policy to complete an unseen task given just a single video demonstration of the task in a given domain. We hypothesize that to successfully generalize to…
Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision
Kuan Fang, Yuke Zhu, Animesh Garg +4
Tool manipulation is vital for facilitating robots to complete challenging task goals. It requires reasoning about the desired effect of the task and thus properly grasping and man…
VUNet: Dynamic Scene View Synthesis for Traversability Estimation using an RGB Camera
Noriaki Hirose, Amir Sadeghian, Fei Xia +2
We present VUNet, a novel view(VU) synthesis method for mobile robots in dynamic environments, and its application to the estimation of future traversability. Our method predicts f…
SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints
Amir Sadeghian, Vineet Kosaraju, Ali Sadeghian +3
This paper addresses the problem of path prediction for multiple interacting agents in a scene, which is a crucial step for many autonomous platforms such as self-driving cars and…
Deep Learning under Privileged Information Using Heteroscedastic Dropout
John Lambert, Ozan Sener, Silvio Savarese
Unlike machines, humans learn through rapid, abstract model-building. The role of a teacher is not simply to hammer home right or wrong answers, but rather to provide intuitive com…
Generalizing to Unseen Domains via Adversarial Data Augmentation
Riccardo Volpi, Hongseok Namkoong, Ozan Sener +3
We are concerned with learning models that generalize well to different \emph{unseen} domains. We consider a worst-case formulation over data distributions that are near the source…
Taskonomy: Disentangling Task Transfer Learning
Amir Zamir, Alexander Sax, William Shen +3
Do visual tasks have a relationship, or are they unrelated? For instance, could having surface normals simplify estimating the depth of an image? Intuition answers these questions…
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
Agrim Gupta, Justin Johnson, Li Fei-Fei +2
Understanding human motion behavior is critical for autonomous moving platforms (like self-driving cars and social robots) if they are to navigate human-centric environments. This…
Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings
Kevin Chen, Christopher B. Choy, Manolis Savva +3
We present a method for generating colored 3D shapes from natural language. To this end, we first learn joint embeddings of freeform text descriptions and colored 3D shapes. Our mo…
GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimation
Noriaki Hirose, Amir Sadeghian, Marynel Vázquez +2
We present semi-supervised deep learning approaches for traversability estimation from fisheye images. Our method, GONet, and the proposed extensions leverage Generative Adversaria…