SDQ: Sparse Decomposed Quantization for LLM Inference

Geonhwa Jeong, Po-An Tsai, Stephen W. Keckler +1

Recently, large language models (LLMs) have shown surprising performance in task-specific workloads as well as general tasks with the given prompts. However, to achieve unprecedent…