3 citations · 8 across the 5 of their papers we have counts for
9 papers
Detecting and Accommodating Novel Types and Concepts in an Embodied Simulation Environment
Sadaf Ghaffari, Nikhil Krishnaswamy
In this paper, we present methods for two types of metacognitive tasks in an AI system: rapidly expanding a neural classification model to accommodate a new category of object, and…
Exploiting Embodied Simulation to Detect Novel Object Classes Through Interaction
Nikhil Krishnaswamy, Sadaf Ghaffari
In this paper we present a novel method for a naive agent to detect novel objects it encounters in an interaction. We train a reinforcement learning policy on a stacking task given…
Neurosymbolic AI for Situated Language Understanding
Nikhil Krishnaswamy, James Pustejovsky
In recent years, data-intensive AI, particularly the domain of natural language processing and understanding, has seen significant progress driven by the advent of large datasets a…
Situated Multimodal Control of a Mobile Robot: Navigation through a Virtual Environment
Katherine Krajovic, Nikhil Krishnaswamy, Nathaniel J. Dimick +2
We present a new interface for controlling a navigation robot in novel environments using coordinated gesture and language. We use a TurtleBot3 robot with a LIDAR and a camera, an…
A Formal Analysis of Multimodal Referring Strategies Under Common Ground
Nikhil Krishnaswamy, James Pustejovsky
In this paper, we present an analysis of computationally generated mixed-modality definite referring expressions using combinations of gesture and linguistic descriptions. In doing…
Multimodal Continuation-style Architectures for Human-Robot Interaction
Nikhil Krishnaswamy, James Pustejovsky
We present an architecture for integrating real-time, multimodal input into a computational agent's contextual model. Using a human-avatar interaction in a virtual world, we treat…