103 citations · 144 across the 9 of their papers we have counts for
10 papers
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Embodiment Collaboration, Abby O'Neill, Abdul Rehman +291
Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, thi…
Can Language Models Learn to Listen?
Evonne Ng, Sanjay Subramanian, Dan Klein +3
We present a framework for generating appropriate facial responses from a listener in dyadic social interactions based on the speaker's words. Given an input transcription of the s…
Robot Learning with Sensorimotor Pre-training
Ilija Radosavovic, Baifeng Shi, Letian Fu +3
We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. Given a seq…
Fast Image-based Neural Relighting with Translucency-Reflection Modeling
Shizhan Zhu, Shunsuke Saito, Aljaz Bozic +3
Image-based lighting (IBL) is a widely used technique that renders objects using a high dynamic range image or environment map. However, aggregating the irradiance at the object's…
TOAST: Transfer Learning via Attention Steering
Baifeng Shi, Siyu Gai, Trevor Darrell +1
Transfer learning involves adapting a pre-trained model to novel downstream tasks. However, we observe that current transfer learning methods often fail to focus on task-relevant f…
Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Models
Sheng Shen, Le Hou, Yanqi Zhou +17
Sparse Mixture-of-Experts (MoE) is a neural architecture design that can be utilized to add learnable parameters to Large Language Models (LLMs) without increasing inference cost.…
Real-World Humanoid Locomotion with Reinforcement Learning
Ilija Radosavovic, Tete Xiao, Bike Zhang +3
Humanoid robots that can autonomously operate in diverse environments have the potential to help address labour shortages in factories, assist elderly at homes, and colonize new pl…
Towards Learning to Play Piano with Dexterous Hands and Touch
Huazhe Xu, Yuping Luo, Shaoxiong Wang +2
The virtuoso plays the piano with passion, poetry and extraordinary technical ability. As Liszt said (a virtuoso)must call up scent and blossom, and breathe the breath of life. The…
PyTouch: A Machine Learning Library for Touch Processing
Mike Lambeta, Huazhe Xu, Jingwei Xu +4
With the increased availability of rich tactile sensors, there is an equally proportional need for open-source and integrated software capable of efficiently and effectively proces…
Attentive Explanations: Justifying Decisions and Pointing to the Evidence (Extended Abstract)
Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata +4
Deep models are the defacto standard in visual decision problems due to their impressive performance on a wide array of visual tasks. On the other hand, their opaqueness has led to…