Risk-Aware Continuous Control with Neural Contextual Bandits

Abstract

Recent advances in learning techniques have garnered attention for their applicability to a diverse range of real-world sequential decision-making problems. Yet, many practical applications have critical constraints for operation in real environments. Most learning solutions often neglect the risk of failing to meet these constraints, hindering their implementation in real-world contexts. In this paper, we propose a risk-aware decision-making framework for contextual bandit problems, accommodating constraints and continuous action spaces. Our approach employs an actor multi-critic architecture, with each critic characterizing the distribution of performance and constraint metrics. Our framework is designed to cater to various risk levels, effectively balancing constraint satisfaction against performance. To demonstrate the effectiveness of our approach, we first compare it against state-of-the-art baseline methods in a synthetic environment, highlighting the impact of intrinsic environmental noise across different risk configurations. Finally, we evaluate our framework in a real-world use case involving a 5G mobile network where only our approach consistently satisfies the system constraint (a signal processing reliability target) with a small performance toll (8.5% increase in power consumption).

12 pages, 13 figures

References in corpus (5)

• Worst Cases Policy Gradients
• Neural Contextual Bandits with Deep Representation and Shallow Exploration
• Constrained Combinatorial Optimization with Reinforcement Learning
• EE-Net: Exploitation-Exploration Neural Networks in Contextual Bandits
• Safe Contextual Bayesian Optimization for Sustainable Room Temperature PID Control Tuning