51 citations · 56 across the 5 of their papers we have counts for
8 papers
Continual Low-Rank Scaled Dot-product Attention
Ginés Carreto Picón, Illia Oleksiienko, Lukas Hedegaard +2
Transformers are widely used for their ability to capture data relations in sequence processing, with great success for a wide range of static tasks. However, the computational and…
Efficient Online Processing with Deep Neural Networks
Lukas Hedegaard
The capabilities and adoption of deep neural networks (DNNs) grow at an exhilarating pace: Vision models accurately classify human actions in videos and identify cancerous tissue i…
Structured Pruning Adapters
Lukas Hedegaard, Aman Alok, Juby Jose +1
Adapters are a parameter-efficient alternative to fine-tuning, which augment a frozen base network to learn new tasks. Yet, the inference of the adapted model is often slower than…
Continual Inference: A Library for Efficient Online Inference with Deep Neural Networks in PyTorch
Lukas Hedegaard, Alexandros Iosifidis
We present Continual Inference, a Python library for implementing Continual Inference Networks (CINs) in PyTorch, a class of Neural Networks designed specifically for efficient inf…
Continual Spatio-Temporal Graph Convolutional Networks
Lukas Hedegaard, Negar Heidari, Alexandros Iosifidis
Graph-based reasoning over skeleton data has emerged as a promising approach for human action recognition. However, the application of prior graph-based methods, which predominantl…
Continual Transformers: Redundancy-Free Attention for Online Inference
Lukas Hedegaard, Arian Bakhtiarnia, Alexandros Iosifidis
Transformers in their common form are inherently limited to operate on whole token sequences rather than on one token at a time. Consequently, their use during online inference on…