95 citations · 269 across the 19 of their papers we have counts for
21 papers
Heterogeneous Structured Pruning Under Resource Constraints for Hardware-Efficient Large Language Models
Cameron Barker, Andreas Paraskeva, Jan N. van Rijn +2
Training-free pruning removes weights from pretrained large language models to reduce their deployment cost. However, existing approaches typically consider only the total number o…
Benchmarking Open-Ended Multi-Agent Coordination in Language Agents
Kale-ab Abebe Tessera, Andras Szecsenyi, Cameron Barker +7
As language models are increasingly deployed as autonomous agents, they must coordinate with others over long horizons in open-ended interactive tasks. Yet existing evaluations rar…
Kalman Linear Attention: Parallel Bayesian Filtering For Efficient Language Modelling and State Tracking
Vaisakh Shaj, Cameron Barker, Aidan Scannell +3
State-space language models such as Mamba and gated linear attention (GLA) offer linear-complexity, parallelisable alternatives to transformers, but their linear state updates limi…
DLAS: An Exploration and Assessment of the Deep Learning Acceleration Stack
Perry Gibson, José Cano, Elliot J. Crowley +2
Deep Neural Networks (DNNs) are extremely computationally demanding, which presents a large barrier to their deployment on resource-constrained devices. Since such devices are wher…
Generate Your Own Scotland: Satellite Image Generation Conditioned on Maps
Miguel Espinosa, Elliot J. Crowley
Despite recent advancements in image generation, diffusion models still remain largely underexplored in Earth Observation. In this paper we show that state-of-the-art pretrained di…
GPViT: A High Resolution Non-Hierarchical Vision Transformer with Group Propagation
Chenhongyi Yang, Jiarui Xu, Shalini De Mello +2
We present the Group Propagation Vision Transformer (GPViT): a novel nonhierarchical (i.e. non-pyramidal) transformer model designed for general visual recognition with high-resolu…
Plug and Play Active Learning for Object Detection
Chenhongyi Yang, Lichao Huang, Elliot J. Crowley
Annotating datasets for object detection is an expensive and time-consuming endeavor. To minimize this burden, active learning (AL) techniques are employed to select the most infor…
Prediction-Guided Distillation for Dense Object Detection
Chenhongyi Yang, Mateusz Ochal, Amos Storkey +1
Real-world object detection models should be cheap and accurate. Knowledge distillation (KD) can boost the accuracy of a small, cheap detection model by leveraging useful informati…
Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning
Chenhongyi Yang, Lichao Huang, Elliot J. Crowley
The goal of contrastive learning based pre-training is to leverage large quantities of unlabeled data to produce a model that can be readily adapted downstream. Current approaches…
Neural Architecture Search as Program Transformation Exploration
Jack Turner, Elliot J. Crowley, Michael O'Boyle
Improving the performance of deep neural networks (DNNs) is important to both the compiler and neural architecture search (NAS) communities. Compilers apply program transformations…
Optimizing Grouped Convolutions on Edge Devices
Perry Gibson, José Cano, Jack Turner +3
When deploying a deep neural network on constrained hardware, it is possible to replace the network's standard convolutions with grouped convolutions. This allows for substantial m…
Neural Architecture Search without Training
Joseph Mellor, Jack Turner, Amos Storkey +1
The time and effort involved in hand-designing deep neural networks is immense. This has prompted the development of Neural Architecture Search (NAS) techniques to automate this de…
Performance Aware Convolutional Neural Network Channel Pruning for Embedded GPUs
Valentin Radu, Kuba Kaszyk, Yuan Wen +6
Convolutional Neural Networks (CNN) are becoming a common presence in many applications and services, due to their superior recognition accuracy. They are increasingly being used o…
Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels
Massimiliano Patacchiola, Jack Turner, Elliot J. Crowley +2
Recently, different machine learning methods have been introduced to tackle the challenging few-shot learning scenario that is, learning from a small labeled dataset related to a s…
Separable Layers Enable Structured Efficient Linear Substitutions
Gavin Gray, Elliot J. Crowley, Amos Storkey
In response to the development of recent efficient dense layers, this paper shows that something as simple as replacing linear components in pointwise convolutions with structured…
BlockSwap: Fisher-guided Block Substitution for Network Compression on a Budget
Jack Turner, Elliot J. Crowley, Michael O'Boyle +2
The desire to map neural networks to varying-capacity devices has led to the development of a wealth of compression techniques, many of which involve replacing standard convolution…
Dilated DenseNets for Relational Reasoning
Antreas Antoniou, Agnieszka Słowik, Elliot J. Crowley +1
Despite their impressive performance in many tasks, deep neural networks often struggle at relational reasoning. This has recently been remedied with the introduction of a plug-in…
Distilling with Performance Enhanced Students
Jack Turner, Elliot J. Crowley, Valentin Radu +3
The task of accelerating large neural networks on general purpose hardware has, in recent years, prompted the use of channel pruning to reduce network size. However, the efficacy o…
A Closer Look at Structured Pruning for Neural Network Compression
Elliot J. Crowley, Jack Turner, Amos Storkey +1
Structured pruning is a popular method for compressing a neural network: given a large trained network, one alternates between removing channel connections and fine-tuning; reducin…
Characterising Across-Stack Optimisations for Deep Convolutional Neural Networks
Jack Turner, José Cano, Valentin Radu +3
Convolutional Neural Networks (CNNs) are extremely computationally demanding, presenting a large barrier to their deployment on resource-constrained devices. Since such systems are…
Moonshine: Distilling with Cheap Convolutions
Elliot J. Crowley, Gavin Gray, Amos Storkey
Many engineers wish to deploy modern neural networks in memory-limited settings; but the development of flexible methods for reducing memory use is in its infancy, and there is lit…