4 papers
To SIMD or Not to SIMD: Comparing RVV vs. CGRA SoCs for Non-Linear Functions in AI
Sameer Deshmukh, Naoya Mizuki, Takumi Honda
Non-linear functions are an important class of operations in modern large language models. However, chip vendors building for AI have so far relied on traditional vector or scalar…
OneComp: One-Line Revolution for Generative AI Model Compression
Yuma Ichikawa, Keiji Kimura, Akihiro Yoshida +11
Deploying foundation models is increasingly constrained by memory footprint, latency, and hardware costs. Post-training compression can mitigate these bottlenecks by reducing the p…
BLaST: High Performance Inference and Pretraining using BLock Sparse Transformers
Patrik Okanovic, Sameer Deshmukh, Grzegorz Kwasniewski +8
The energy consumption of large-scale ML models is dominated by data movement, shuffling billions of parameters across memory hierarchies and data centers. Sparsification offers a…
Direct Quantized Training of Language Models with Stochastic Rounding
Kaiyan Zhao, Tsuguchika Tabaru, Kenichi Kobayashi +3
Although recent quantized Large Language Models (LLMs), such as BitNet, have paved the way for significant reduction in memory usage during deployment with binary or ternary weight…