7 papers
STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU
Victor J. B. Jung, Gagandeep Singh, Joseph Melber +3
The growing adoption of large language model-based agents within operating system workflows has increased the importance of energy-efficient inference on laptop-class systems-on-ch…
BioTrain: Sub-MB, Sub-50mW On-Device Fine-Tuning for Edge-AI on Biosignals
Run Wang, Victor J. B. Jung, Philip Wiese +5
Biosignals exhibit substantial cross-subject and cross-session variability, inducing severe domain shifts that degrade post-deployment performance for small, edge-oriented AI model…
TrainDeeploy: Hardware-Accelerated Parameter-Efficient Fine-Tuning of Small Transformer Models at the Extreme Edge
Run Wang, Victor J. B. Jung, Philip Wiese +3
On-device tuning of deep neural networks enables long-term adaptation at the edge while preserving data privacy. However, the high computational and memory demands of backpropagati…
A Dynamic Allocation Scheme for Adaptive Shared-Memory Mapping on Kilo-core RV Clusters for Attention-Based Model Deployment
Bowen Wang, Marco Bertuletti, Yichao Zhang +2
Attention-based models demand flexible hardware to manage diverse kernels with varying arithmetic intensities and memory access patterns. Large clusters with shared L1 memory, a co…
Distributed Inference with Minimal Off-Chip Traffic for Transformers on Low-Power MCUs
Severin Bochem, Victor J. B. Jung, Arpan Prasad +2
Contextual Artificial Intelligence (AI) based on emerging Transformer models is predicted to drive the next technology revolution in interactive wearable devices such as new-genera…
Fused-Tiled Layers: Minimizing Data Movement on RISC-V SoCs with Software-Managed Caches
Victor J. B. Jung, Alessio Burrello, Francesco Conti +1
The success of DNNs and their high computational requirements pushed for large codesign efforts aiming at DNN acceleration. Since DNNs can be represented as static computational gr…