4 papers · 1 filter
LATMiX: Learnable Affine Transformations for Microscaling Quantization of LLMs
Ofir Gordon, Lior Dikstein, Arnon Netzer +2
Post-training quantization (PTQ) is a widely used approach for reducing the memory and compute costs of large language models (LLMs). Recent studies have shown that applying invert…
MLoRQ: Bridging Low-Rank and Quantization for Transformer Compression
Ofir Gordon, Ariel Lapid, Elad Cohen +3
Deploying transformer-based neural networks on resource-constrained edge devices presents a significant challenge. This challenge is often addressed through various techniques, suc…
Data Generation for Hardware-Friendly Post-Training Quantization
Lior Dikstein, Ariel Lapid, Arnon Netzer +1
Zero-shot quantization (ZSQ) using synthetic data is a key approach for post-training quantization (PTQ) under privacy and security constraints. However, existing data generation m…
Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learning
Idan Achituve, Idit Diamant, Arnon Netzer +2
As machine learning becomes more prominent there is a growing demand to perform several inference tasks in parallel. Running a dedicated model for each task is computationally expe…