Reinforcement Learning-powered Semantic Communication via Semantic Similarity

Abstract

We introduce a new semantic communication mechanism - SemanticRL, whose key idea is to preserve the semantic information instead of strictly securing the bit-level precision. Unlike previous methods that mainly concentrate on the network or structure design, we revisit the learning process and point out the semantic blindness of commonly used objective functions. To address this semantic gap, we introduce a schematic shift that learns from semantic similarity, instead of relying on conventional paired bit-level supervisions like cross entropy and bit error rate. However, developing such a semantic communication system is indeed a nontrivial task considering the non-differentiability of most semantic metrics as well as the instability from noisy channels. To further resolve these issues, we put forward a self-critic reinforcement learning (RL) solution which allows an efficient and stable learning on any user-defined semantic measurement, and take a step further to simultaneously tackle the non-differentiable semantic channel optimization problem via self-critic stochastic iterative updating (SCSIU) training on the decoupled semantic transceiver. We have firstly tested the proposed method in the challenging European-parliament dataset, which confirms the superiority of our method in revealing the semantic meanings, and better handling the semantic noise. Apart from the experimental results, we further provide an in-depth look at how the semantic model behaves, along with its superb generalization ability in real-life examples. An RL-based image transmission extension is also exemplified, so as to prove the generalization ability and motivate future discussion.

arXiv commentProject and codes: https://github.com/lukun199/SemanticRL

References in corpus (11)

• Efficient Estimation of Word Representations in Vector Space
• Language Models are Few-Shot Learners
• Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy
• Deep Learning Enabled Semantic Communication Systems
• Sequence Level Training with Recurrent Neural Networks
• Improved Image Captioning via Policy Gradient optimization of SPIDEr
• Deep Source-Channel Coding for Sentence Semantic Transmission with HARQ
• A Better Variant of Self-Critical Sequence Training
• On the Sentence Embeddings from Pre-trained Language Models
• Channel Estimation via Successive Denoising in MIMO OFDM Systems: A Reinforcement Learning Approach
• Semantic Communications for Speech Signals

Cited by in corpus (6)

• Overview of AI and Communication for 6G Network: Fundamentals, Challenges, and Future Research Opportunities
• Engineering Semantic Communication: A Survey
• Towards Ubiquitous Semantic Metaverse: Challenges, Approaches, and Opportunities
• Understand-Before-Talk (UBT): A Semantic Communication Approach to 6G Networks
• Semantic Edge Computing and Semantic Communications in 6G Networks: A Unifying Survey and Research Challenges
• Model-free Reinforcement Learning of Semantic Communication by Stochastic Policy Gradient