7 papers
FORGE: Procedural Weight Generation for Sparse MoE Inference
Chanyoung Moon, Travis Pence, Daisuke Yamada +4
We introduce FORGE (Fast On-chip Rotor Generation for Experts), a procedural weight generation system for Sparse Mixture-of-Experts (MoE) inference. Rather than storing and loading…
Fine-Tuning of Transformer models with Frames
Harshavardhan Adepu, Li Zhang, Sanjiv Kumar +1
Parameter-Efficient Fine-Tuning (PEFT) strategies such as Low-Rank Adaptation (LoRA) are effective solutions for fine-tuning large-scale pre-trained models; however, their memory r…
Beyond Static Policies: Dynamic Selection Among Modern Microarchitectural Policies
Yanxin Zhang, Ian McDougall, Junnan Li +3
Modern processors gain performance from interacting policies: prefetchers, predictors, replacement rules, and schedulers. These policies are often evaluated one at a time, yet a po…
Beyond Static Policies: Exploring Dynamic Policy Selection for Single-Thread Performance Optimization
Yanxin Zhang, Ian McDougall, Junnan Li +3
For over a decade, processor design has focused on implementing sophisticated policies for various components of the out-of-order pipeline, including cache replacement and prefetch…
NeuroScalar: A Deep Learning Framework for Fast, Accurate, and In-the-Wild Cycle-Level Performance Prediction
Shayne Wadle, Yanxin Zhang, Vikas Singh +1
The evaluation of new microprocessor designs is constrained by slow, cycle-accurate simulators that rely on unrepresentative benchmark traces. This paper introduces a novel deep le…
IM-Unpack: Training and Inference with Arbitrarily Low Precision Integers
Zhanpeng Zeng, Karthikeyan Sankaralingam, Vikas Singh
GEneral Matrix Multiply (GEMM) is a central operation in deep learning and corresponds to the largest chunk of the compute footprint. Therefore, improving its efficiency is an acti…
FrameQuant: Flexible Low-Bit Quantization for Transformers
Harshavardhan Adepu, Zhanpeng Zeng, Li Zhang +1
Transformers are the backbone of powerful foundation models for many Vision and Natural Language Processing tasks. But their compute and memory/storage footprint is large, and so,…