3 citations · 3 across the 4 of their papers we have counts for
4 papers
Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs
Bakbergen Ryskulov, Iker García-Ferrero, David Montero +5
Serving large language models cheaply increasingly means shipping models that are both structurally compressed to a fraction of their parameters and quantized to 4 bits. Together t…
Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss
Bakbergen Ryskulov, Iker García-Ferrero, David Montero +5
Small language models are often the only option for deployment under tight latency, cost, and on-premises constraints, but they are rarely trained from scratch: a compressed model…
Distilling Lightweight Domain Experts from Large ML Models by Identifying Relevant Subspaces
Pattarawat Chormai, Ali Hashemi, Klaus-Robert Müller +1
Knowledge distillation involves transferring the predictive capabilities of large, high-performing AI models (teachers) to smaller models (students) that can operate in environment…
Enhancing Brain Source Reconstruction by Initializing 3D Neural Networks with Physical Inverse Solutions
Marco Morik, Ali Hashemi, Klaus-Robert Müller +2
Reconstructing brain sources is a fundamental challenge in neuroscience, crucial for understanding brain function and dysfunction. Electroencephalography (EEG) signals have a high…