Meta-Offline and Distributional Multi-Agent RL for Risk-Aware Decision-Making

Abstract

Mission critical applications, such as UAV-assisted IoT networks require risk-aware decision-making under dynamic topologies and uncertain channels. We propose meta-conservative quantile regression (M-CQR), a meta-offline distributional MARL algorithm that integrates conservative Q-learning (CQL) for safe offline learning, quantile regression DQN (QR-DQN) for risk-sensitive value estimation, and model-agnostic meta-learning (MAML) for rapid adaptation. Two variants are developed: meta-independent CQR (M-I-CQR) and meta-CTDE-CQR. In a UAV-based communication scenario, M-CTDE-CQR achieves up to 50% faster convergence and outperforms baseline MARL methods, offering improved scalability, robustness, and adaptability for risk-sensitive decision-making. Code is available at https://github.com/Eslam211/MA_Meta_ODRL

References in corpus (5)

• A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
• Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
• Meta-Reinforcement Learning Based Resource Allocation for Dynamic V2X Communications
• Multi-UAV Path Learning for Age and Power Optimization in IoT with UAV Battery Recharge
• Conservative and Risk-Aware Offline Multi-Agent Reinforcement Learning