4 citations · 4 across the 7 of their papers we have counts for
6 papers
Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs
Bakbergen Ryskulov, Iker García-Ferrero, David Montero +5
Serving large language models cheaply increasingly means shipping models that are both structurally compressed to a fraction of their parameters and quantized to 4 bits. Together t…
Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss
Bakbergen Ryskulov, Iker García-Ferrero, David Montero +5
Small language models are often the only option for deployment under tight latency, cost, and on-premises constraints, but they are rarely trained from scratch: a compressed model…
Efficient Model Compression Techniques with FishLeg
Jamie McGowan, Wei Sheng Lai, Weibin Chen +7
In many domains, the most successful AI models tend to be the largest, indeed often too large to be handled by AI players with limited computational resources. To mitigate this, a…
Uniform Generalization Bounds for Overparameterized Neural Networks
Sattar Vakili, Michael Bromberg, Jezabel Garcia +2
An interesting observation in artificial neural networks is their favorable generalization error despite typically being extremely overparameterized. It is well known that the clas…
Meta-Learning with MAML on Trees
Jezabel R. Garcia, Federica Freddi, Feng-Ting Liao +5
In meta-learning, the knowledge learned from previous tasks is transferred to new ones, but this transfer only works if tasks are related. Sharing information between unrelated tas…
Cyclic orthogonal convolutions for long-range integration of features
Federica Freddi, Jezabel R Garcia, Michael Bromberg +4
In Convolutional Neural Networks (CNNs) information flows across a small neighbourhood of each pixel of an image, preventing long-range integration of features before reaching deep…