183 citations · 552 across the 27 of their papers we have counts for
4 papers · 1 filter
A Functional Perspective on Knowledge Distillation in Neural Networks
Israel Mason-Williams, Gabryel Mason-Williams, Helen Yannakoudakis
Knowledge distillation is considered a compression mechanism when judged on the resulting student's accuracy and loss, yet its functional impact is poorly understood. We quantify t…
A Function-Centric Perspective on Flat and Sharp Minima
Israel Mason-Williams, Gabryel Mason-Williams, Helen Yannakoudakis
Flat minima are strongly associated with improved generalisation in deep neural networks. However, this connection has proven nuanced in recent studies, with both theoretical count…
A (More) Realistic Evaluation Setup for Generalisation of Community Models on Malicious Content Detection
Ivo Verhoeven, Pushkar Mishra, Rahel Beloch +2
Community models for malicious content detection, which take into account the context from a social graph alongside the content itself, have shown remarkable performance on benchma…
Learning New Tasks from a Few Examples with Soft-Label Prototypes
Avyav Kumar Singh, Ekaterina Shutova, Helen Yannakoudakis
Existing approaches to few-shot learning in NLP rely on large language models (LLMs) and/or fine-tuning of these to generalise on out-of-distribution data. In this work, we propose…