Enhancing Supermarket Robot Interaction: A Multi-Level LLM Conversational Interface for Handling Diverse Customer Intents

Abstract

This paper presents the design and evaluation of a novel multi-level LLM interface for supermarket robots to assist customers. The proposed interface allows customers to convey their needs through both generic and specific queries. While state-of-the-art systems like OpenAI's GPTs are highly adaptable and easy to build and deploy, they still face challenges such as increased response times and limitations in strategic control of the underlying model for tailored use-case and cost optimization. Driven by the goal of developing faster and more efficient conversational agents, this paper advocates for using multiple smaller, specialized LLMs fine-tuned to handle different user queries based on their specificity and user intent. We compare this approach to a specialized GPT model powered by GPT-4 Turbo, using the Artificial Social Agent Questionnaire (ASAQ) and qualitative participant feedback in a counterbalanced within-subjects experiment. Our findings show that our multi-LLM chatbot architecture outperformed the benchmarked GPT model across all 13 measured criteria, with statistically significant improvements in four key areas: performance, user satisfaction, user-agent partnership, and self-image enhancement. The paper also presents a method for supermarket robot navigation by mapping the final chatbot response to correct shelf numbers, enabling the robot to sequentially navigate towards the respective products, after which lower-level robot perception, control, and planning can be used for automated object retrieval. We hope this work encourages more efforts into using multiple, specialized smaller models instead of relying on a single powerful, but more expensive and slower model.

References in corpus (11)

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• Robust Speech Recognition via Large-Scale Weak Supervision
• Retrieval-Augmented Generation for Large Language Models: A Survey
• Achieving Human Parity in Conversational Speech Recognition
• Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment
• Bias in Automated Speaker Recognition
• RAFT: Adapting Language Model to Domain Specific RAG
• Task-Level Authoring for Remote Robot Teleoperation
• In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss
• Beyond ChatBots: ExploreLLM for Structured Thoughts and Personalized Model Responses
• WhisperWand: Simultaneous Voice and Gesture Tracking Interface