Publications (14)
Learning to Ground Multi-Agent Communication with Autoencoders
Toru Lin, Minyoung Huh, Chris Stauffer +2
Communication requires having a common language, a lingua franca, between agents. This language could emerge via a consensus process, but it may require many generations of trial a…
Learning Visuotactile Skills with Two Multifingered Hands
Toru Lin, Yu Zhang, Qiyang Li +4
Aiming to replicate human-like dexterity, perceptual experiences, and motion patterns, we explore learning from human demonstrations using a bimanual system with multifingered hand…
MonoDuo: Using One Robot Arm to Learn Bimanual Policies
Sandeep Bajamahal, Lawrence Yunliang Chen, Toru Lin +3
Bimanual coordination is essential for many real-world manipulation tasks, yet learning bimanual robot policies is limited by the scarcity of bimanual robots and datasets. Single-a…
Learning Dexterous Manipulation Skills from Imperfect Simulations
Elvis Hsieh, Wen-Han Hsieh, Yen-Jen Wang +4
Reinforcement learning and sim-to-real transfer have made significant progress in dexterous manipulation. However, progress remains limited by the difficulty of simulating complex…
Model Based Planning with Energy Based Models
Yilun Du, Toru Lin, Igor Mordatch
Model-based planning holds great promise for improving both sample efficiency and generalization in reinforcement learning (RL). We show that energy-based models (EBMs) are a promi…
Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learning
Zhengyi Luo, Chen Tessler, Toru Lin +8
Human behavior is fundamentally shaped by visual perception -- our ability to interact with the world depends on actively gathering relevant information and adapting our movements…