8 papers
Sub-Model Short-Term Memory Convolutions for Keyword Spotting Systems on Device
Paweł Warlewski, Artur Czeczko, Artur Szumaczuk +2
Keyword Spotting (KWS) is becoming increasingly important as voice-controlled devices grow more widespread. While voice interaction with smartphones and smart TVs is already common…
Pocket-STVG: lightweight architecture for Spatio-Temporal Video Grounding
Alberto Presta, Michal Byra, Grzegorz Stefański +3
Spatio-Temporal Video Grounding (STVG) aims to localize the spatio-temporal tube in a video corresponding to a natural language query. While recent methods achieve strong performan…
Training Crossroads for Recurrent Vision Transformers: Recurrence, Neural ODEs, and Deep Supervision
Grzegorz Gruszczynski, Pawel Olszowiec, Michal Byra +2
Vision Transformers (ViTs) achieve strong image-recognition performance, but their parameter count grows linearly with depth when each block is independently parameterized. Single-…
t-gems: text-guided exit modules for decreasing clip image encoder
Alberto Presta, Grzegorz Stefanski, Michal Byra +1
Multimodal deep neural networks enhance deep comprehension by integrating diverse data modalities. Data from different modalities are typically projected into a shared latent space…
Investigating Single-Block Recurrence in Vision Transformers for Image Recognition
Michal Byra, Pawel Olszowiec, Grzegorz Stefanski +2
Vision Transformers (ViTs) implement depth by stacking independently parameterized blocks, but it remains unclear how much of this parameterization is necessary and how much can be…
Routing the Lottery: Adaptive Subnetworks for Heterogeneous Data
Grzegorz Stefanski, Alberto Presta, Michal Byra
In pruning, the Lottery Ticket Hypothesis posits that large networks contain sparse subnetworks, or winning tickets, that can be trained in isolation to match the performance of th…
SOI: Scaling Down Computational Complexity by Estimating Partial States of the Model
Grzegorz Stefański, Paweł Daniluk, Artur Szumaczuk +1
Consumer electronics used to follow the miniaturization trend described by Moore's Law. Despite increased processing power in Microcontroller Units (MCUs), MCUs used in the smalles…
Short-Term Memory Convolutions
Grzegorz Stefański, Krzysztof Arendt, Paweł Daniluk +2
The real-time processing of time series signals is a critical issue for many real-life applications. The idea of real-time processing is especially important in audio domain as the…