Calibrating AI Models for Few-Shot Demodulation via Conformal Prediction
arXiv:2210.04621 · doi:10.1109/TMLCN.2023.3319282
Abstract
AI tools can be useful to address model deficits in the design of communication systems. However, conventional learning-based AI algorithms yield poorly calibrated decisions, unabling to quantify their outputs uncertainty. While Bayesian learning can enhance calibration by capturing epistemic uncertainty caused by limited data availability, formal calibration guarantees only hold under strong assumptions about the ground-truth, unknown, data generation mechanism. We propose to leverage the conformal prediction framework to obtain data-driven set predictions whose calibration properties hold irrespective of the data distribution. Specifically, we investigate the design of baseband demodulators in the presence of hard-to-model nonlinearities such as hardware imperfections, and propose set-based demodulators based on conformal prediction. Numerical results confirm the theoretical validity of the proposed demodulators, and bring insights into their average prediction set size efficiency.
arXiv commentSubmitted for a conference publication
References in corpus (25)
- On Calibration of Modern Neural Networks
- Over the Air Deep Learning Based Radio Signal Classification
- Millimeter Wave Cellular Networks: A MAC Layer Perspective
- Estimating conditional quantiles with the help of the pinball loss
- Conformal Prediction: a Unified Review of Theory and New Challenges
- Explaining by Removing: A Unified Framework for Model Explanation
- Learning to Demodulate from Few Pilots via Offline and Online Meta-Learning
- Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning
- Evaluating model calibration in classification
- Verified Uncertainty Calibration
- Bayesian Optimization for Radio Resource Management: Open Loop Power Control
- Liberty or Depth: Deep Bayesian Neural Nets Do Not Need Complex Weight Posterior Approximations
- Calibrated Explanations: with Uncertainty Information and Counterfactuals
- Bayesian Active Meta-Learning for Few Pilot Demodulation and Equalization
- Adaptive Conformal Predictions for Time Series
- Training Uncertainty-Aware Classifiers with Conformalized Deep Learning
- Calibrated Prediction with Covariate Shift via Unsupervised Domain Adaptation
- Learning under Model Misspecification: Applications to Variational and Ensemble methods
- Investigating the Impact of Model Misspecification in Neural Simulation-based Inference
- Learning Optimal Conformal Classifiers
- Selection and Aggregation of Conformal Prediction Sets
- Achieving Risk Control in Online Learning Settings
- Beyond calibration: estimating the grouping loss of modern neural networks
- Probabilistic Conformal Prediction Using Conditional Random Samples
- Conformal Prediction with Temporal Quantile Adjustments