2 papers
cs.LG2024
LQER: Low-Rank Quantization Error Reconstruction for LLMs
Cheng Zhang, Jianyi Cheng, George A. Constantinides +1
Post-training quantization of Large Language Models (LLMs) is challenging. In this work, we introduce Low-rank Quantization Error Reduction (LQER), which combines quantization and…
cs.AR2024
A Dataflow Compiler for Efficient LLM Inference using Custom Microscaling Formats
Jianyi Cheng, Cheng Zhang, Zhewen Yu +3
Model quantization represents both parameters (weights) and intermediate values (activations) in a more compact format, thereby directly reducing both computational and memory cost…