From the 1 of 18 linked papers with an AI index.
14 papers · 1 filter
Can AI Weather Models Predict Beyond Two Weeks? A Quantitative Benchmark and Analysis of Long Rollouts
Fanny Lehmann, Firat Ozdemir, Yun Cheng +4
While AI weather models excel at short-to-medium range forecasts (up to 15 days), they frequently suffer from ill-defined "instabilities" when rolled out over longer horizons. This…
When Data Is Scarce: Scaling Sparse Language Models with Repeated Training
Boqian Wu, Qiao Xiao, Patrik Okanovic +6
Scaling laws for dense LLMs under infinite data are well explored, but how sparsity interacts with limited data is not. In this work, we study sparse training in data-constrained r…
Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling
Qiao Xiao, Boqian Wu, Patrik Okanovic +6
Dynamic Sparse Training (DST) offers a promising paradigm for improving the training and inference efficiency of deep neural networks; however, we find that in large language model…
WUSH: Near-Optimal Adaptive Transforms for LLM Quantization
Jiale Chen, Vage Egiazarian, Roberto L. Castro +2
Quantizing LLM weights and activations is a standard approach for efficient deployment, but a few extreme outliers can stretch the dynamic range and amplify low-bit quantization er…
The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm
Jiale Chen, Yalda Shabanzadeh, Elvir CrnÄeviÄ +2
Quantizing the weights of large language models (LLMs) from 16-bit to lower bitwidth is the de facto approach to deploy massive transformers onto more affordable accelerators. Whil…
Grid Games: The Power of Multiple Grids for Quantizing Large Language Models
Vage Egiazarian, Erik Schultheis, Andrei Panferov +3
A major recent advance in quantization is given by microscaled 4-bit formats such as NVFP4 and MXFP4, quantizing values into small groups sharing a scale, assuming a fixed floating…