Real-World Robot Applications of Foundation Models: A Review
arXiv:2402.05741 · doi:10.1080/01691864.2024.2408593
Abstract
Recent developments in foundation models, like Large Language Models (LLMs) and Vision-Language Models (VLMs), trained on extensive data, facilitate flexible application across different tasks and modalities. Their impact spans various fields, including healthcare, education, and robotics. This paper provides an overview of the practical application of foundation models in real-world robotics, with a primary emphasis on the replacement of specific components within existing robot systems. The summary encompasses the perspective of input-output relationships in foundation models, as well as their role in perception, motion planning, and control within the field of robotics. This paper concludes with a discussion of future challenges and implications for practical robot applications.
References in corpus (16)
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
- PaLM: Scaling Language Modeling with Pathways
- Large Language Models are Zero-Shot Reasoners
- Conservative Q-Learning for Offline Reinforcement Learning
- MOPO: Model-based Offline Policy Optimization
- Text2Motion: From Natural Language Instructions to Feasible Plans
- TidyBot: Personalized Robot Assistance with Large Language Models
- Symbol Emergence in Robotics: A Survey
- ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application
- DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics
- OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Robotics
- VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset
- VQA-based Robotic State Recognition Optimized with Genetic Algorithm
- Robotic Applications of Pre-Trained Vision-Language Models to Various Recognition Behaviors
- UniMASK: Unified Inference in Sequential Decision Problems
Cited by in corpus (6)
- Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
- Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Review
- RAMPA: Robotic Augmented Reality for Machine Programming by DemonstrAtion
- Skin-Machine Interface with Multimodal Contact Motion Classifier
- Variational Adaptive Noise and Dropout towards Stable Recurrent Neural Networks
- KeyMPs: One-Shot Vision-Language Guided Motion Generation by Sequencing DMPs for Occlusion-Rich Tasks