6 papers
QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization
Yipin Guo, Arun M George, Jie Fu +3
Weight-only post-training quantization (PTQ) can alleviate the computational burden of serving large language models (LLMs) at scale. However, existing PTQ methods often fail to ge…
SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving
Yipin Guo, Siddharth Joshi
Contemporary systems serving large language models (LLMs) have adopted prefill-decode disaggregation to load-balance between the compute-bound prefill phase and the memory-bound de…
Efficient Fusion and Task Guided Embedding for End-to-end Autonomous Driving
Yipin Guo, Yilin Lang, Qinyuan Ren
To address the challenges of sensor fusion and safety risk prediction, contemporary closed-loop autonomous driving neural networks leveraging imitation learning typically require a…
GPTQT: Quantize Large Language Models Twice to Push the Efficiency
Yipin Guo, Yilin Lang, Qinyuan Ren
Due to their large size, generative Large Language Models (LLMs) require significant computing and storage resources. This paper introduces a new post-training quantization method,…
ShiftAddAug: Augment Multiplication-Free Tiny Neural Network with Hybrid Computation
Yipin Guo, Zihao Li, Yilin Lang +1
Operators devoid of multiplication, such as Shift and Add, have gained prominence for their compatibility with hardware. However, neural networks (NNs) employing these operators ty…
ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization
Haoran You, Yipin Guo, Yichao Fu +6
Large language models (LLMs) have shown impressive performance on language tasks but face challenges when deployed on resource-constrained devices due to their extensive parameters…