Reducing Computational Complexity of Neural Networks in Optical Channel Equalization: From Concepts to Implementation

Abstract

In this paper, a new methodology is proposed that allows for the low-complexity development of neural network (NN) based equalizers for the mitigation of impairments in high-speed coherent optical transmission systems. In this work, we provide a comprehensive description and comparison of various deep model compression approaches that have been applied to feed-forward and recurrent NN designs. Additionally, we evaluate the influence these strategies have on the performance of each NN equalizer. Quantization, weight clustering, pruning, and other cutting-edge strategies for model compression are taken into consideration. In this work, we propose and evaluate a Bayesian optimization-assisted compression, in which the hyperparameters of the compression are chosen to simultaneously reduce complexity and improve performance. In conclusion, the trade-off between the complexity of each compression approach and its performance is evaluated by utilizing both simulated and experimental data in order to complete the analysis. By utilizing optimal compression approaches, we show that it is possible to design an NN-based equalizer that is simpler to implement and has better performance than the conventional digital back-propagation (DBP) equalizer with only one step per span. This is accomplished by reducing the number of multipliers used in the NN equalizer after applying the weighted clustering and pruning algorithms. Furthermore, we demonstrate that an equalizer based on NN can also achieve superior performance while still maintaining the same degree of complexity as the full electronic chromatic dispersion compensation block. We conclude our analysis by highlighting open questions and existing challenges, as well as possible future research directions.

References in corpus (17)

• Practical Bayesian Optimization of Machine Learning Algorithms
• To prune, or not to prune: exploring the efficacy of pruning for model compression
• Comparing Rewinding and Fine-tuning in Neural Network Pruning
• Importance of Tuning Hyperparameters of Machine Learning Algorithms
• Training Quantized Nets: A Deeper Understanding
• Performance and Complexity Analysis of bi-directional Recurrent Neural Network Models vs. Volterra Nonlinear Equalizers in Digital Coherent Systems
• Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference
• Computational Complexity Evaluation of Neural Network Applications in Signal Processing
• ShiftAddNet: A Hardware-Inspired Deep Network
• Neural Network Quantization for Efficient Inference: A Survey
• Towards Efficient Post-training Quantization of Pre-trained Language Models
• Fixed-point Quantization of Convolutional Neural Networks for Quantized Inference on Embedded Platforms
• Power-of-Two Quantization for Low Bitwidth and Hardware Compliant Neural Networks
• Methods for Pruning Deep Neural Networks
• Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activations
• DKM: Differentiable K-Means Clustering Layer for Neural Network Compression
• PAC-Net: A Model Pruning Approach to Inductive Transfer Learning

Cited by in corpus (3)

• Implementing Neural Network-Based Equalizers in a Coherent Optical Transmission System Using Field-Programmable Gate Arrays
• Multichannel Nonlinear Equalization in Coherent WDM Systems based on Bi-directional Recurrent Neural Networks
• Artificial Intelligence Driven Channel Coding and Resource Optimization for Wireless Networks: A Systematic Survey