229 citations · 329 across the 32 of their papers we have counts for
32 papers
An offline approach to fNIRS-guided reinforcement learning for robot behavior
Julia Santaniello, Madelaine Brower, Benson Jiang +4
Human-in-the-loop Reinforcement Learning has become a popular approach for training, finetuning, and aligning robot behavior with user preferences. Our paper explores the feasibili…
Novelty Adaptation Through Hybrid Large Language Model (LLM)-Symbolic Planning and LLM-guided Reinforcement Learning
Hong Lu, Pierrick Lorang, Timothy R. Duggan +2
In dynamic open-world environments, autonomous agents often encounter novelties that hinder their ability to find plans to achieve their goals. Specifically, traditional symbolic p…
Towards Reinforcement Learning from Neural Feedback: Mapping fNIRS Signals to Agent Performance
Julia Santaniello, Matthew Russell, Benson Jiang +3
Reinforcement Learning from Human Feedback (RLHF) is a methodology that aligns agent behavior with human preferences by integrating user feedback into the agent's training process.…
Mapping Neural Signals to Agent Performance, A Step Towards Reinforcement Learning from Neural Feedback
Julia Santaniello, Matthew Russell, Benson Jiang +3
Implicit Human-in-the-Loop Reinforcement Learning (HITL-RL) is a methodology that integrates passive human feedback into autonomous agent training while minimizing human workload.…
FLEX: A Framework for Learning Robot-Agnostic Force-based Skills Involving Sustained Contact Object Manipulation
Shijie Fang, Wenchang Gao, Shivam Goel +3
Learning to manipulate objects efficiently, particularly those involving sustained contact (e.g., pushing, sliding) and articulated parts (e.g., drawers, doors), presents significa…
Creative Problem Solving in Large Language and Vision Models -- What Would it Take?
Lakshmi Nair, Evana Gizzi, Jivko Sinapov
We advocate for a strong integration of Computational Creativity (CC) with research in large language and vision models (LLVMs) to address a key limitation of these models, i.e., c…
Logical Specifications-guided Dynamic Task Sampling for Reinforcement Learning Agents
Yash Shukla, Tanushree Burman, Abhishek Kulkarni +3
Reinforcement Learning (RL) has made significant strides in enabling artificial agents to learn diverse behaviors. However, learning an effective policy often requires a large numb…
Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community
Casey Kennington, Malihe Alikhani, Heather Pon-Barry +20
The ability to interact with machines using natural human language is becoming not just commonplace, but expected. The next step is not just text interfaces, but speech interfaces…
NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds
Shivam Goel, Yichen Wei, Panagiotis Lymperopoulos +3
As AI agents leave the lab and venture into the real world as autonomous vehicles, delivery robots, and cooking robots, it is increasingly necessary to design and comprehensively e…
LgTS: Dynamic Task Sampling using LLM-generated sub-goals for Reinforcement Learning Agents
Yash Shukla, Wenchang Gao, Vasanth Sarathy +3
Recent advancements in reasoning abilities of Large Language Models (LLM) has promoted their usage in problems that require high-level planning for robots and artificial agents. Ho…
A Framework for Few-Shot Policy Transfer through Observation Mapping and Behavior Cloning
Yash Shukla, Bharat Kesari, Shivam Goel +2
Despite recent progress in Reinforcement Learning for robotics applications, many tasks remain prohibitively difficult to solve because of the expensive interaction cost. Transfer…
MOSAIC: Learning Unified Multi-Sensory Object Property Representations for Robot Learning via Interactive Perception
Gyan Tatiya, Jonathan Francis, Ho-Hsiang Wu +2
A holistic understanding of object properties across diverse sensory modalities (e.g., visual, audio, and haptic) is essential for tasks ranging from object categorization to compl…
Automaton-Guided Curriculum Generation for Reinforcement Learning Agents
Yash Shukla, Abhishek Kulkarni, Robert Wright +2
Despite advances in Reinforcement Learning, many sequential decision making tasks remain prohibitively expensive and impractical to learn. Recently, approaches that automatically g…
Methods and Mechanisms for Interactive Novelty Handling in Adversarial Environments
Tung Thai, Ming Shen, Mayank Garg +10
Learning to detect, characterize and accommodate novelties is a challenge that agents operating in open-world domains need to address to be able to guarantee satisfactory task perf…
Cross-Tool and Cross-Behavior Perceptual Knowledge Transfer for Grounded Object Recognition
Gyan Tatiya, Jonathan Francis, Jivko Sinapov
Humans learn about objects via interaction and using multiple perceptions, such as vision, sound, and touch. While vision can provide information about an object's appearance, non-…
Knowledge-driven Scene Priors for Semantic Audio-Visual Embodied Navigation
Gyan Tatiya, Jonathan Francis, Luca Bondi +4
Generalisation to unseen contexts remains a challenge for embodied navigation agents. In the context of semantic audio-visual navigation (SAVi) tasks, the notion of generalisation…
Transferring Implicit Knowledge of Non-Visual Object Properties Across Heterogeneous Robot Morphologies
Gyan Tatiya, Jonathan Francis, Jivko Sinapov
Humans leverage multiple sensor modalities when interacting with objects and discovering their intrinsic properties. Using the visual modality alone is insufficient for deriving in…
RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environments
Shivam Goel, Yash Shukla, Vasanth Sarathy +2
We propose RAPid-Learn: Learning to Recover and Plan Again, a hybrid planning and learning method, to tackle the problem of adapting to sudden and unexpected changes in an agent's…
Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework
Evana Gizzi, Lakshmi Nair, Sonia Chernova +1
Creative Problem Solving (CPS) is a sub-area within Artificial Intelligence (AI) that focuses on methods for solving off-nominal, or anomalous problems in autonomous systems. Despi…
ACuTE: Automatic Curriculum Transfer from Simple to Complex Environments
Yash Shukla, Christopher Thierauf, Ramtin Hosseini +2
Despite recent advances in Reinforcement Learning (RL), many problems, especially real-world tasks, remain prohibitively expensive to learn. To address this issue, several lines of…
An Augmented Reality Platform for Introducing Reinforcement Learning to K-12 Students with Robots
Ziyi Zhang, Samuel Micah Akai-Nettey, Adonai Addo +2
Interactive reinforcement learning, where humans actively assist during an agent's learning process, has the promise to alleviate the sample complexity challenges of practical algo…
AI-HRI 2021 Proceedings
Reuth Mirsky, Megan Zimmerman, Muneed Ahmad +10
The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successful venue of discussion and collaboration since 2014. During that time, these symposi…
A Framework for Multisensory Foresight for Embodied Agents
Xiaohui Chen, Ramtin Hosseini, Karen Panetta +1
Predicting future sensory states is crucial for learning agents such as robots, drones, and autonomous vehicles. In this paper, we couple multiple sensory modalities with explorato…
Planning Multimodal Exploratory Actions for Online Robot Attribute Learning
Xiaohan Zhang, Jivko Sinapov, Shiqi Zhang
Robots frequently need to perceive object attributes, such as "red," "heavy," and "empty," using multimodal exploratory actions, such as "look," "lift," and "shake." Robot attribut…