27 citations · 48 across the 14 of their papers we have counts for
6 papers · 1 filter
Efficient LLM Inference on CPUs
Haihao Shen, Hanwen Chang, Bo Dong +2
Large language models (LLMs) have demonstrated remarkable performance and tremendous potential across a wide range of tasks. However, deploying these models has been challenging du…
Effective Quantization for Diffusion Models on CPUs
Hanwen Chang, Haihao Shen, Yiyang Cai +7
Diffusion models have gained popularity for generating images from textual descriptions. Nonetheless, the substantial need for computational resources continues to present a notewo…
TEQ: Trainable Equivalent Transformation for Quantization of LLMs
Wenhua Cheng, Yiyang Cai, Kaokao Lv +1
As large language models (LLMs) become more prevalent, there is a growing need for new and improved quantization methods that can meet the computationalast layer demands of these m…
Efficient Post-training Quantization with FP8 Formats
Haihao Shen, Naveen Mellempudi, Xin He +3
Recent advances in deep learning methods such as LLMs and Diffusion models have created a need for improved quantization methods that can meet the computational demands of these mo…
Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs
Wenhua Cheng, Weiwei Zhang, Haihao Shen +4
Large Language Models (LLMs) have demonstrated exceptional proficiency in language-related tasks, but their deployment poses significant challenges due to substantial memory and st…
An Efficient Sparse Inference Software Accelerator for Transformer-based Language Models on CPUs
Haihao Shen, Hengyu Meng, Bo Dong +9
In recent years, Transformer-based language models have become the standard approach for natural language processing tasks. However, stringent throughput and latency requirements i…