9 papers
Understand and Accelerate Memory Processing Pipeline for Large Language Model Inference
Zifan He, Rui Ma, Yizhou Sun +1
Modern large language models (LLMs) increasingly depends on efficient long-context processing and generation mechanisms, including sparse attention, retrieval-augmented generation…
HMT: Semantic Hierarchy-Aware Hierarchical Memory Transformer
Maryam Haghifam, Zifan He, Jason Cong +1
Transformer-based LLMs achieve strong results on many language tasks; however, long inputs remain challenging because context windows are finite, and prefill latency and memory gro…
LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs
Zifan He, Shengyu Ye, Rui Ma +2
The rapid development of large language models (LLM) has greatly enhanced everyday applications. While many FPGA-based accelerators, with flexibility for fine-grained data control,…
FlexLLM: Composable HLS Library for Flexible Hybrid LLM Accelerator Design
Jiahao Zhang, Zifan He, Nicholas Fraser +3
We present FlexLLM, a composable High-Level Synthesis (HLS) library for rapid development of domain-specific LLM accelerators. FlexLLM exposes key architectural degrees of freedom…
Optimized Multi-Token Joint Decoding with Auxiliary Model for LLM Inference
Zongyue Qin, Ziniu Hu, Zifan He +3
Large language models (LLMs) have achieved remarkable success across diverse tasks, yet their inference processes are hindered by substantial time and energy demands due to single-…
InTAR: Inter-Task Auto-Reconfigurable Accelerator Design for High Data Volume Variation in DNNs
Zifan He, Anderson Truong, Yingqi Cao +1
The rise of deep neural networks (DNNs) has driven an increased demand for computing power and memory. Modern DNNs exhibit high data volume variation (HDV) across tasks, which pose…