1 citations · 2 across the 7 of their papers we have counts for
8 papers
Training an MoE LLM in MXFP4: How We Closed the Gap to BF16
Han Wang, Zhitao Wang, Natalia Frumkin +4
Training large language models (LLMs) in sub-8-bit arithmetic promises substantial gains in memory bandwidth and compute throughput, yet FP4 formats introduce quantization error se…
DARE: Diffusion Language Model Activation Reuse for Efficient Inference
Natalia Frumkin, Bokun Wang, Hung-Yueh Chiang +3
Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to auto-regressive (AR) models, offering greater expressive capacity and potential for parallel gene…
Q-Sched: Pushing the Boundaries of Few-Step Diffusion Models with Quantization-Aware Scheduling
Natalia Frumkin, Diana Marculescu
Text-to-image diffusion models are computationally intensive, often requiring dozens of forward passes through large transformer backbones. For instance, Stable Diffusion XL genera…
Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models
Hung-Yueh Chiang, Chi-Chih Chang, Natalia Frumkin +3
State Space Models (SSMs) are emerging as a compelling alternative to Transformers because of their consistent memory usage and high performance. Despite this, scaling up SSMs on c…
Quamba: A Post-Training Quantization Recipe for Selective State Space Models
Hung-Yueh Chiang, Chi-Chih Chang, Natalia Frumkin +2
State Space Models (SSMs) have emerged as an appealing alternative to Transformers for large language models, achieving state-of-the-art accuracy with constant memory complexity wh…
Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers
Natalia Frumkin, Dibakar Gope, Diana Marculescu
Quantization scale and bit-width are the most important parameters when considering how to quantize a neural network. Prior work focuses on optimizing quantization scales in a glob…
MobileTL: On-device Transfer Learning with Inverted Residual Blocks
Hung-Yueh Chiang, Natalia Frumkin, Feng Liang +1
Transfer learning on edge is challenging due to on-device limited resources. Existing work addresses this issue by training a subset of parameters or adding model patches. Develope…
CPT-V: A Contrastive Approach to Post-Training Quantization of Vision Transformers
Natalia Frumkin, Dibakar Gope, Diana Marculescu
When considering post-training quantization, prior work has typically focused on developing a mixed precision scheme or learning the best way to partition a network for quantizatio…