107 citations · 390 across the 26 of their papers we have counts for
30 papers
Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learning
David Yunis, Justin Jung, Falcon Dai +1
Exploration in sparse-reward reinforcement learning is difficult due to the requirement of long, coordinated sequences of actions in order to achieve any reward. Moreover, in conti…
NeRFuser: Large-Scale Scene Representation by NeRF Fusion
Jiading Fang, Shengjie Lin, Igor Vasiljevic +5
A practical benefit of implicit visual representations like Neural Radiance Fields (NeRFs) is their memory efficiency: large scenes can be efficiently stored and shared as small ne…
Eliciting User Preferences for Personalized Multi-Objective Decision Making through Comparative Feedback
Han Shao, Lee Cohen, Avrim Blum +3
In classic reinforcement learning (RL) and decision making problems, policies are evaluated with respect to a scalar reward function, and all optimal policies are the same with reg…
Depth Field Networks for Generalizable Multi-view Scene Representation
Vitor Guizilini, Igor Vasiljevic, Jiading Fang +4
Modern 3D computer vision leverages learning to boost geometric reasoning, mapping image data to classical structures such as cost volumes or epipolar constraints to improve matchi…
Grasp and Motion Planning for Dexterous Manipulation for the Real Robot Challenge
Takuma Yoneda, Charles Schaff, Takahiro Maeda +1
This report describes our winning submission to the Real Robot Challenge (https://real-robot-challenge.com/). The Real Robot Challenge is a three-phase dexterous manipulation compe…
Boosting Contrastive Self-Supervised Learning with False Negative Cancellation
Tri Huynh, Simon Kornblith, Matthew R. Walter +2
Self-supervised representation learning has made significant leaps fueled by progress in contrastive learning, which seeks to learn transformations that embed positive input pairs…
Pow-Wow: A Dataset and Study on Collaborative Communication in Pommerman
Takuma Yoneda, Matthew R. Walter, Jason Naradowsky
In multi-agent learning, agents must coordinate with each other in order to succeed. For humans, this coordination is typically accomplished through the use of language. In this wo…
Integrated Benchmarking and Design for Reproducible and Accessible Evaluation of Robotic Agents
Jacopo Tani, Andrea F. Daniele, Gianmarco Bernasconi +10
As robotics matures and increases in complexity, it is more necessary than ever that robot autonomy research be reproducible. Compared to other sciences, there are specific challen…
Concurrent Training Improves the Performance of Behavioral Cloning from Observation
Zachary W. Robertson, Matthew R. Walter
Learning from demonstration is widely used as an efficient way for robots to acquire new skills. However, it typically requires that demonstrations provide full access to the state…
Residual Policy Learning for Shared Autonomy
Charles Schaff, Matthew R. Walter
Shared autonomy provides an effective framework for human-robot collaboration that takes advantage of the complementary strengths of humans and robots to achieve common goals. Many…
Loop Estimator for Discounted Values in Markov Reward Processes
Falcon Z. Dai, Matthew R. Walter
At the working heart of policy iteration algorithms commonly used and studied in the discounted setting of reinforcement learning, the policy evaluation step estimates the value of…
Language-guided Semantic Mapping and Mobile Manipulation in Partially Observable Environments
Siddharth Patki, Ethan Fahnestock, Thomas M. Howard +1
Recent advances in data-driven models for grounded language understanding have enabled robots to interpret increasingly complex instructions. Two fundamental limitations of these m…
DIODE: A Dense Indoor and Outdoor DEpth Dataset
Igor Vasiljevic, Nick Kolkin, Shanyi Zhang +8
We introduce DIODE, a dataset that contains thousands of diverse high resolution color images with accurate, dense, long-range depth measurements. DIODE (Dense Indoor/Outdoor DEpth…
Maximum Expected Hitting Cost of a Markov Decision Process and Informativeness of Rewards
Falcon Z. Dai, Matthew R. Walter
We propose a new complexity measure for Markov decision processes (MDPs), the maximum expected hitting cost (MEHC). This measure tightens the closely related notion of diameter [JO…
Multigrid Neural Memory
Tri Huynh, Michael Maire, Matthew R. Walter
We introduce a novel approach to endowing neural networks with emergent, long-term, large-scale memory. Distinct from strategies that connect neural networks to external memory ban…
Inferring Compact Representations for Efficient Natural Language Understanding of Robot Instructions
Siddharth Patki, Andrea F. Daniele, Matthew R. Walter +1
The speed and accuracy with which robots are able to interpret natural language is fundamental to realizing effective human-robot interaction. A great deal of attention has been pa…
The AI Driving Olympics at NeurIPS 2018
Julian Zilly, Jacopo Tani, Breandan Considine +14
Despite recent breakthroughs, the ability of deep learning and reinforcement learning to outperform traditional approaches to control physically embodied robotic agents remains lar…
Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning
Charles Schaff, David Yunis, Ayan Chakrabarti +1
The physical design of a robot and the policy that controls its motion are inherently coupled, and should be determined according to the task and environment. In an increasing numb…
Generalized Grounding Graphs: A Probabilistic Framework for Understanding Grounded Commands
Thomas Kollar, Stefanie Tellex, Matthew Walter +8
Many task domains require robots to interpret and act upon natural language commands which are given by people and which refer to the robot's physical surroundings. Such interpreta…
Satellite Image-based Localization via Learned Embeddings
Dong-Ki Kim, Matthew R. Walter
We propose a vision-based method that localizes a ground vehicle using publicly available satellite imagery as the only prior knowledge of the environment. Our approach takes as in…
Jointly Optimizing Placement and Inference for Beacon-based Localization
Charles Schaff, David Yunis, Ayan Chakrabarti +1
The ability of robots to estimate their location is crucial for a wide variety of autonomous operations. In settings where GPS is unavailable, measurements of transmissions from fi…
Coherent Dialogue with Attention-based Language Models
Hongyuan Mei, Mohit Bansal, Matthew R. Walter
We model coherent conversation continuation via RNN-based dialogue models equipped with a dynamic attention mechanism. Our attention-RNN language model dynamically increases the sc…
Navigational Instruction Generation as Inverse Reinforcement Learning with Neural Machine Translation
Andrea F. Daniele, Mohit Bansal, Matthew R. Walter
Modern robotics applications that involve human-robot interaction require robots to be able to communicate with humans seamlessly and effectively. Natural language provides a flexi…
Learning Articulated Motion Models from Visual and Lingual Signals
Zhengyang Wu, Mohit Bansal, Matthew R. Walter
In order for robots to operate effectively in homes and workplaces, they must be able to manipulate the articulated objects common within environments built for and by humans. Prev…