2 citations · 3 across the 9 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.AI2024
Dynamic-Width Speculative Beam Decoding for Efficient LLM Inference
Zongyue Qin, Zifan He, Neha Prakriya +2
Large language models (LLMs) have shown outstanding performance across numerous real-world tasks. However, the autoregressive nature of these models makes the inference process slo…
cs.CL2024
Optimized Multi-Token Joint Decoding with Auxiliary Model for LLM Inference
Zongyue Qin, Ziniu Hu, Zifan He +3
Large language models (LLMs) have achieved remarkable success across diverse tasks, yet their inference processes are hindered by substantial time and energy demands due to single-…
cs.CL2024★ 2 cited
HMT: Hierarchical Memory Transformer for Efficient Long Context Language Processing
Zifan He, Yingqi Cao, Zongyue Qin +3
Transformer-based large language models (LLM) have been widely used in language processing applications. However, due to the memory constraints of the devices, most of them restric…