4 papers
HPIM: Heterogeneous Processing-In-Memory-based Accelerator for Large Language Models Inference
Cenlin Duan, Jianlei Yang, Rubing Yang +8
The deployment of large language models (LLMs) presents significant challenges due to their enormous memory footprints, low arithmetic intensity, and stringent latency requirements…
Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing
Lingkun Long, Yushi Huang, Shihao Bai +4
Diffusion Large Language Models (dLLMs) deliver strong long-context processing capability in a non-autoregressive decoding paradigm. However, the considerable computational cost of…
TinyFormer: Efficient Transformer Design and Deployment on Tiny Devices
Jianlei Yang, Jiacheng Liao, Fanding Lei +6
Developing deep learning models on tiny devices (e.g. Microcontroller units, MCUs) has attracted much attention in various embedded IoT applications. However, it is challenging to…
SlimInfer: Accelerating Long-Context LLM Inference via Dynamic Token Pruning
Lingkun Long, Rubing Yang, Yushi Huang +3
Long-context inference for Large Language Models (LLMs) is heavily limited by high computational demands. While several existing methods optimize attention computation, they still…