Real-Time Procedural Learning From Experience for AI Agents

Abstract

Learning how to do things from trial and error in real time is a hallmark of biological intelligence, yet most LLM-based agents lack mechanisms to acquire procedural knowledge after deployment. We propose Procedural Recall for Agents with eXperiences Indexed by State (PRAXIS), a lightweight post-training learning mechanism that stores the consequences of actions and retrieves them by jointly matching environmental and internal states of past episodes to the current state. PRAXIS augments agentic action selection with retrieved state-action-result exemplars that are generated in real time. When evaluated on the REAL web browsing benchmark, PRAXIS improves task completion accuracy, reliability, and cost efficiency across different foundation model backbones, and shows preliminary generalization to unseen tasks in similar environments. These results demonstrate that PRAXIS enables the practical adoption of AI agents in fast-evolving stateful environments by helping them learn new procedures effectively.

References in corpus (12)

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Reflexion: Language Agents with Verbal Reinforcement Learning
• Self-Refine: Iterative Refinement with Self-Feedback
• Mind2Web: Towards a Generalist Agent for the Web
• WebArena: A Realistic Web Environment for Building Autonomous Agents
• ExpeL: LLM Agents Are Experiential Learners
• MemoryBank: Enhancing Large Language Models with Long-Term Memory
• Agent Workflow Memory
• Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
• WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
• CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization
• Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control