From Learning to Meta-Learning: Reduced Training Overhead and Complexity for Communication Systems

Abstract

Machine learning methods adapt the parameters of a model, constrained to lie in a given model class, by using a fixed learning procedure based on data or active observations. Adaptation is done on a per-task basis, and retraining is needed when the system configuration changes. The resulting inefficiency in terms of data and training time requirements can be mitigated, if domain knowledge is available, by selecting a suitable model class and learning procedure, collectively known as inductive bias. However, it is generally difficult to encode prior knowledge into an inductive bias, particularly with black-box model classes such as neural networks. Meta-learning provides a way to automatize the selection of an inductive bias. Meta-learning leverages data or active observations from tasks that are expected to be related to future, and a priori unknown, tasks of interest. With a meta-trained inductive bias, training of a machine learning model can be potentially carried out with reduced training data and/or time complexity. This paper provides a high-level introduction to meta-learning with applications to communication systems.

arXiv commentInvited to the 6G Wireless Summit 2020

References in corpus (17)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Reconciling modern machine learning practice and the bias-variance trade-off
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• On First-Order Meta-Learning Algorithms
• Gradient-based Hyperparameter Optimization through Reversible Learning
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
• Bayesian Model-Agnostic Meta-Learning
• Learning to Demodulate from Few Pilots via Offline and Online Meta-Learning
• A PAC-Bayesian bound for Lifelong Learning
• Learned Optimizers that Scale and Generalize
• Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory
• Meta-Learning without Memorization
• Meta-Learning with Warped Gradient Descent
• Meta-Learning to Communicate: Fast End-to-End Training for Fading Channels
• On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms
• Deep Transfer Learning Based Downlink Channel Prediction for FDD Massive MIMO Systems
• Modular Meta-Learning with Shrinkage

Cited by in corpus (12)

• Learning to Demodulate from Few Pilots via Offline and Online Meta-Learning
• Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization
• End-to-End Fast Training of Communication Links Without a Channel Model via Online Meta-Learning
• End-to-End Learning for VCSEL-based Optical Interconnects: State-of-the-Art, Challenges, and Opportunities
• Information-Theoretic Generalization Bounds for Meta-Learning and Applications
• Bayesian Active Meta-Learning for Few Pilot Demodulation and Equalization
• Joint PMD Tracking and Nonlinearity Compensation with Deep Neural Networks
• An Energy and Carbon Footprint Analysis of Distributed and Federated Learning
• Prediction of mmWave/THz Link Blockages through Meta-Learning and Recurrent Neural Networks
• Meta-ViterbiNet: Online Meta-Learned Viterbi Equalization for Non-Stationary Channels
• On the Energy and Communication Efficiency Tradeoffs in Federated and Multi-Task Learning
• Joint Design of Radar Waveform and Detector via End-to-end Learning with Waveform Constraints