108 citations · 234 across the 37 of their papers we have counts for
8 papers · 1 filter
GatedFWA: Linear Flash Windowed Attention with Gated Associative Memory
Jiaxu Liu, Yuhe Bai, Xiangyu Yin +1
Modern autoregressive models rely on attention, yet the Softmax full attention in Transformers scales quadratically with sequence length. Sliding Window Attention (SWA) achieves li…
Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix It
Guoxuan Xia, Olivier Laurent, Gianni Franchi +1
Label smoothing (LS) is a popular regularisation method for training neural networks as it is effective in improving test accuracy and is simple to implement. ``Hard'' one-hot labe…
Mixed-TD: Efficient Neural Network Accelerator with Layer-Specific Tensor Decomposition
Zhewen Yu, Christos-Savvas Bouganis
Neural Network designs are quite diverse, from VGG-style to ResNet-style, and from Convolutional Neural Networks to Transformers. Towards the design of efficient accelerators, many…
Window-Based Early-Exit Cascades for Uncertainty Estimation: When Deep Ensembles are More Efficient than Single Models
Guoxuan Xia, Christos-Savvas Bouganis
Deep Ensembles are a simple, reliable, and effective method of improving both the predictive performance and uncertainty estimates of deep learning approaches. However, they are wi…
SVD-NAS: Coupling Low-Rank Approximation and Neural Architecture Search
Zhewen Yu, Christos-Savvas Bouganis
The task of compressing pre-trained Deep Neural Networks has attracted wide interest of the research community due to its great benefits in freeing practitioners from data access r…
On the Usefulness of Deep Ensemble Diversity for Out-of-Distribution Detection
Guoxuan Xia, Christos-Savvas Bouganis
The ability to detect Out-of-Distribution (OOD) data is important in safety-critical applications of deep learning. The aim is to separate In-Distribution (ID) data drawn from the…
Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data
Guoxuan Xia, Christos-Savvas Bouganis
Detecting out-of-distribution (OOD) data is a task that is receiving an increasing amount of research attention in the domain of deep learning for computer vision. However, the per…
perf4sight: A toolflow to model CNN training performance on Edge GPUs
Aditya Rajagopal, Christos-Savvas Bouganis
The increased memory and processing capabilities of today's edge devices create opportunities for greater edge intelligence. In the domain of vision, the ability to adapt a Convolu…