3 papers
cs.LG2025
SINQ: Sinkhorn-Normalized Quantization for Calibration-Free Low-Precision LLM Weights
Lorenz K. Müller, Philippe Bich, Jiawei Zhuang +3
Post-training quantization has emerged as the most widely used strategy for deploying large language models at low precision. Still, current methods show perplexity degradation at…
cs.LG2025
TyphoonMLA: A Mixed Naive-Absorb MLA Kernel For Shared Prefix
Ahmet Caner Yüzügüler, Ahmet Çelik, Jiawei Zhuang +1
Multi-Head Latent Attention (MLA) is a recent attention mechanism adopted in state-of-the-art LLMs such as DeepSeek-v3 and Kimi K2. Thanks to its novel formulation, MLA allows two…
cs.AI2025
PRESERVE: Prefetching Model Weights and KV-Cache in Distributed LLM Serving
Ahmet Caner Yüzügüler, Jiawei Zhuang, Lukas Cavigelli
Large language models (LLMs) are typically served from clusters of GPUs/NPUs that consist of large number of devices. Unfortunately, communication between these devices incurs sign…