11 citations · 24 across the 12 of their papers we have counts for
5 papers · 1 filter
SlimKV: Joint Token-Feature KV Cache Compression with Reconstruction-Free Beacon Attention
Zihan Teng, Jiayu Zhao, Wentao Ren +4
Long-context LLM serving is increasingly bottlenecked by KV-cache memory, especially in resource-constrained scenarios. Among existing KV-cache compression strategies, token-wise m…
EvoLP: Self-Evolving Latency Predictor for Model Compression in Real-Time Edge Systems
Shuo Huai, Hao Kong, Shiqing Li +5
Edge devices are increasingly utilized for deploying deep learning applications on embedded systems. The real-time nature of many applications and the limited resources of edge dev…
BitsMoE: Cost-Aware Bit Allocation in Spectral Space for MoE LLM Quantization
Jiayu Zhao, Zihan Teng, Minhao Fan +4
Mixture-of-Experts (MoE) large language models incur substantial memory costs due to their large expert parameter counts. Mixed-precision quantization reduces these costs by alloca…
Exploring Deep-to-Shallow Transformable Neural Networks for Intelligent Embedded Systems
Xiangzhong Luo, Weichen Liu
Thanks to the evolving network depth, convolutional neural networks (CNNs) have achieved remarkable success across various embedded scenarios, paving the way for ubiquitous embedde…
Efficient Deep Learning Infrastructures for Embedded Computing Systems: A Comprehensive Survey and Future Envision
Xiangzhong Luo, Di Liu, Hao Kong +4
Deep neural networks (DNNs) have recently achieved impressive success across a wide range of real-world vision and language processing tasks, spanning from image classification to…