169 citations · 255 across the 16 of their papers we have counts for
4 papers · 1 filter
MoE-I: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
Cheng Yang, Yang Sui, Jinqi Xiao +7
The emergence of Mixture of Experts (MoE) LLMs has significantly advanced the development of language models. Compared to traditional LLMs, MoE LLMs outperform traditional LLMs by…
HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networks
Jinqi Xiao, Chengming Zhang, Yu Gong +5
Low-rank compression is an important model compression strategy for obtaining compact neural network models. In general, because the rank values directly determine the model comple…
Doubly Residual Neural Decoder: Towards Low-Complexity High-Performance Channel Decoding
Siyu Liao, Chunhua Deng, Miao Yin +1
Recently deep neural networks have been successfully applied in channel coding to improve the decoding performance. However, the state-of-the-art neural channel decoders cannot ach…
Compressing Recurrent Neural Networks Using Hierarchical Tucker Tensor Decomposition
Miao Yin, Siyu Liao, Xiao-Yang Liu +2
Recurrent Neural Networks (RNNs) have been widely used in sequence analysis and modeling. However, when processing high-dimensional data, RNNs typically require very large model si…