9 citations · 12 across the 8 of their papers we have counts for
10 papers
Training an MoE LLM in MXFP4: How We Closed the Gap to BF16
Han Wang, Zhitao Wang, Natalia Frumkin +4
Training large language models (LLMs) in sub-8-bit arithmetic promises substantial gains in memory bandwidth and compute throughput, yet FP4 formats introduce quantization error se…
KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers
Yann Bouquet, Alireza Khodamoradi, Kristof Denolf +1
Post-training quantization (PTQ) of diffusion transformers (DiTs) to W4A4 severely degrades output quality, because activations entering each linear layer contain outliers that 4-b…
AdaHOP: Fast and Accurate Low-Precision Training via Outlier-Pattern-Aware Rotation
Seonggon Kim, Alireza Khodamoradi, Pranathi Vasireddy +2
Hadamard transforms have become a key tool for stabilizing low-precision training, but existing methods apply them uniformly across tensors and computation paths. We show that this…
LoRaQ: Optimized Low Rank Approximation for 4-bit Quantization
Yann Bouquet, Alireza Khodamoradi, Sophie Yáng Shen +2
Post-training quantization (PTQ) is essential for deploying large diffusion transformers on resource-constrained hardware, but aggressive 4-bit quantization significantly degrades…
Error Diffusion: Post Training Quantization with Block-Scaled Number Formats for Neural Networks
Alireza Khodamoradi, Kristof Denolf, Eric Dellinger
Quantization reduces the model's hardware costs, such as data movement, storage, and operations like multiply and addition. It also affects the model's behavior by degrading the ou…
Microscaling Data Formats for Deep Learning
Bita Darvish Rouhani, Ritchie Zhao, Ankit More +30
Narrow bit-width data formats are key to reducing the computational and storage costs of modern deep learning applications. This paper evaluates Microscaling (MX) data formats that…
SPARTA: Spatial Acceleration for Efficient and Scalable Horizontal Diffusion Weather Stencil Computation
Gagandeep Singh, Alireza Khodamoradi, Kristof Denolf +6
Fast and accurate climate simulations and weather predictions are critical for understanding and preparing for the impact of climate change. Real-world weather and climate modeling…
Tailor: Altering Skip Connections for Resource-Efficient Inference
Olivia Weng, Gabriel Marcano, Vladimir Loncar +7
Deep neural networks use skip connections to improve training convergence. However, these skip connections are costly in hardware, requiring extra buffers and increasing on- and of…
RUBICON: A Framework for Designing Efficient Deep Learning-Based Genomic Basecallers
Gagandeep Singh, Mohammed Alser, Kristof Denolf +5
Nanopore sequencing generates noisy electrical signals that need to be converted into a standard string of DNA nucleotide bases using a computational step called basecalling. The a…
Hardware-efficient Residual Networks for FPGAs
Olivia Weng, Alireza Khodamoradi, Ryan Kastner
Residual networks (ResNets) employ skip connections in their networks -- reusing activations from previous layers -- to improve training convergence, but these skip connections cre…