UltraSketchLLM: Sub-1-Bit LLM Compression via Sketch and Hardware-Friendly Operators

Sunan Zou, Xueting Sun, Ziyun Zhang +1

Large language models (LLMs) require larger GPU memory size these days, necessitating efficient and extreme weight compression methods. Existing compression methods are either theo…