5 papers
HEM: a margin-based loss for visual categorisation tasks
Michael W. Spratling, Heiko H. Schütt
Training deep neural networks (DNNs) on classification tasks can be performed with a number of different losses, but cross-entropy (CE) loss is the de-facto standard. Here, we prop…
Softplus Attention with Re-weighting Boosts Length Extrapolation in Large Language Models
Bo Gao, Michael W. Spratling, Letizia Gionfrida
Large language models have achieved remarkable success in recent years, primarily due to self-attention. However, traditional Softmax attention suffers from numerical instability a…
FAAR: Efficient Frequency-Aware Multi-Task Fine-Tuning via Automatic Rank Selection
Maxime Fontana, Michael Spratling, Miaojing Shi
Adapting models pre-trained on large-scale datasets is a proven way to reach strong performance quickly for down-stream tasks. However, the growth of state-of-the-art mod-els makes…
Optimizing Dense Visual Predictions Through Multi-Task Coherence and Prioritization
Maxime Fontana, Michael Spratling, Miaojing Shi
Multi-Task Learning (MTL) involves the concurrent training of multiple tasks, offering notable advantages for dense prediction tasks in computer vision. MTL not only reduces traini…
Few-Shot Anomaly Detection via Category-Agnostic Registration Learning
Chaoqin Huang, Haoyan Guan, Aofan Jiang +4
Most existing anomaly detection (AD) methods require a dedicated model for each category. Such a paradigm, despite its promising results, is computationally expensive and inefficie…