1 citations · 1 across the 11 of their papers we have counts for
11 papers
GPTQ-2D: Cubic-Time Two-Sided Adaptive Rounding
Jiale Chen, Torsten Hoefler, Dan Alistarh
Adaptive rounding methods such as GPTQ, or equivalently Babai's nearest plane algorithm, round a real matrix to integers under a quadratic metric. They process the entries in a fix…
Can AI Weather Models Predict Beyond Two Weeks? A Quantitative Benchmark and Analysis of Long Rollouts
Fanny Lehmann, Firat Ozdemir, Yun Cheng +4
While AI weather models excel at short-to-medium range forecasts (up to 15 days), they frequently suffer from ill-defined "instabilities" when rolled out over longer horizons. This…
SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
Alexis Limozin, Eduard Durech, Torsten Hoefler +2
Recent mixed-policy optimization methods for LLM reasoning that interleave or blend supervised and reinforcement learning signals report improvements over the standard SFT-then-RL…
Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting
Firat Ozdemir, Yun Cheng, Salman Mohebi +11
Foundation models (FMs) for the Earth system learn statistical relationships between physical variables across massive datasets to enable versatile downstream applications through…
An Engineering Journey Training Large Language Models at Scale on Alps: The Apertus Experience
Jonathan Coles, Stefano Schuppli, Lukas Drescher +20
Large Language Models (LLMs) have surged as a transformative technology for science and society, prompting governments worldwide to pursue sovereign AI capabilities that ensure dat…
Beyond Outliers: A Study of Optimizers Under Quantization
Georgios Vlassis, Saleh Ashkboos, Alexandra Volkova +2
As new optimizers gain traction and model quantization becomes standard for efficient deployment, a key question arises: how does the choice of optimizer affect model performance i…
Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization
Vage Egiazarian, Roberto L. Castro, Denis Kuznedelev +8
The recent hardware-accelerated microscaling 4-bit floating-point formats such as MXFP4 and NVFP4, supported on NVIDIA and AMD GPUs, promise to revolutionize large language model (…
HALO: Hadamard-Assisted Lower-Precision Optimization for LLMs
Saleh Ashkboos, Mahdi Nikdan, Soroush Tabesh +3
Quantized training of Large Language Models (LLMs) remains an open challenge, as maintaining accuracy while performing all matrix multiplications in low precision has proven diffic…
EfQAT: An Efficient Framework for Quantization-Aware Training
Saleh Ashkboos, Bram Verhoef, Torsten Hoefler +2
Quantization-aware training (QAT) schemes have been shown to achieve near-full precision accuracy. They accomplish this by training a quantized model for multiple epochs. This is c…
MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models
Elias Frantar, Roberto L. Castro, Jiale Chen +2
As inference on Large Language Models (LLMs) emerges as an important workload in machine learning applications, weight quantization has become a standard technique for efficient GP…
SpComm3D: A Framework for Enabling Sparse Communication in 3D Sparse Kernels
Nabil Abubaker, Torsten Hoefler
Existing 3D algorithms for distributed-memory sparse kernels suffer from limited scalability due to reliance on bulk sparsity-agnostic communication. While easier to use, sparsity-…