Large Language Models as Zero-Shot Human Models for Human-Robot Interaction

Abstract

Human models play a crucial role in human-robot interaction (HRI), enabling robots to consider the impact of their actions on people and plan their behavior accordingly. However, crafting good human models is challenging; capturing context-dependent human behavior requires significant prior knowledge and/or large amounts of interaction data, both of which are difficult to obtain. In this work, we explore the potential of large-language models (LLMs) -- which have consumed vast amounts of human-generated text data -- to act as zero-shot human models for HRI. Our experiments on three social datasets yield promising results; the LLMs are able to achieve performance comparable to purpose-built models. That said, we also discuss current limitations, such as sensitivity to prompts and spatial/numerical reasoning mishaps. Based on our findings, we demonstrate how LLM-based human models can be integrated into a social robot's planning process and applied in HRI scenarios. Specifically, we present one case study on a simulated trust-based table-clearing task and replicate past results that relied on custom models. Next, we conduct a new robot utensil-passing experiment (n = 65) where preliminary results show that planning with a LLM-based human model can achieve gains over a basic myopic plan. In summary, our results show that LLMs offer a promising (but incomplete) approach to human modeling for HRI.

8 pages

References in corpus (4)

• Training language models to follow instructions with human feedback
• Scaling Instruction-Finetuned Language Models
• Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks
• Translating Natural Language to Planning Goals with Large-Language Models

Cited by in corpus (11)

• LaMI: Large Language Models for Multi-Modal Human-Robot Interaction
• On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)
• The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcare
• A Survey on Dialogue Management in Human-Robot Interaction
• The Last JITAI? Exploring Large Language Models for Issuing Just-in-Time Adaptive Interventions: Fostering Physical Activity in a Conceptual Cardiac Rehabilitation Setting
• To Help or Not to Help: LLM-based Attentive Support for Human-Robot Group Interactions
• LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions
• Bootstrapping Cognitive Agents with a Large Language Model
• Comparing Apples to Oranges: LLM-powered Multimodal Intention Prediction in an Object Categorization Task
• Are Large Language Models Aligned with People's Social Intuitions for Human-Robot Interactions?
• Robot Behavior Personalization from Sparse User Feedback