9 papers
SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices
Ernests Lavrinovics, Marco Letizia, Roy Janco +3
We present SigmaScale, a method for learning auxiliary scaling matrices to aid truncated Singular Value Decomposition (SVD) based Large Language Model (LLM) compression. Instea…
Taming the Exponential: A Fast Softmax Surrogate for Integer-Native Edge Inference
Dimitrios Danopoulos, Enrico Lupi, Michael Kagan +1
Softmax can become a computational bottleneck in the Transformer model's Multi-Head Attention (MHA) block, particularly in small models under low-precision inference, where exponen…
PQuantML: A Tool for End-to-End Hardware-aware Model Compression
Roope Niemi, Anastasiia Petrovych, Arghya Ranjan Das +9
PQuantML is a new open-source, hardware-aware neural network model compression library tailored to end-to-end workflows. Motivated by the need to deploy performant models to enviro…
CHLU: The Causal Hamiltonian Learning Unit as a Symplectic Primitive for Deep Learning
Pratik Jawahar, Maurizio Pierini
Current deep learning primitives dealing with temporal dynamics suffer from a fundamental dichotomy: they are either discrete and unstable (LSTMs) \citep{pascanu_difficulty_2013},…
Towards Tensor Network Models for Low-Latency Jet Tagging on FPGAs
Alberto Coppi, Ema Puljak, Lorenzo Borella +6
We present a systematic study of Tensor Network (TN) models $\unicode{x2013}$ Matrix Product States (MPS) and Tree Tensor Networks (TTN) $\unicode{x2013}$ for real-time jet tagging…
AIE4ML: An End-to-End Framework for Compiling Neural Networks for the Next Generation of AMD AI Engines
Dimitrios Danopoulos, Enrico Lupi, Chang Sun +4
Efficient AI inference on AMD's Versal AI Engine (AIE) is challenging due to tightly coupled VLIW execution, explicit datapaths, and local memory management. Prior work focused on…