7 citations · 14 across the 7 of their papers we have counts for
7 papers
Learning to Project for Cross-Task Knowledge Distillation
Dylan Auty, Roy Miles, Benedikt Kolbeinsson +1
Traditional knowledge distillation (KD) relies on a proficient teacher trained on the target task, which is not always available. In this setting, cross-task distillation can be us…
MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation
Roy Miles, Mehmet Kerim Yucel, Bruno Manganelli +1
This paper tackles the problem of semi-supervised video object segmentation on resource-constrained devices, such as mobile phones. We formulate this problem as a distillation task…
Understanding the Role of the Projector in Knowledge Distillation
Roy Miles, Krystian Mikolajczyk
In this paper we revisit the efficacy of knowledge distillation as a function matching and metric learning problem. In doing so we verify three important design decisions, namely t…
Information Theoretic Representation Distillation
Roy Miles, Adrian Lopez Rodriguez, Krystian Mikolajczyk
Despite the empirical success of knowledge distillation, current state-of-the-art methods are computationally expensive to train, which makes them difficult to adopt in practice. T…
Reconstructing Pruned Filters using Cheap Spatial Transformations
Roy Miles, Krystian Mikolajczyk
We present an efficient alternative to the convolutional layer using cheap spatial transformations. This construction exploits an inherent spatial redundancy of the learned convolu…
Cascaded channel pruning using hierarchical self-distillation
Roy Miles, Krystian Mikolajczyk
In this paper, we propose an approach for filter-level pruning with hierarchical knowledge distillation based on the teacher, teaching-assistant, and student framework. Our method…
Compression of descriptor models for mobile applications
Roy Miles, Krystian Mikolajczyk
Deep neural networks have demonstrated state-of-the-art performance for feature-based image matching through the advent of new large and diverse datasets. However, there has been l…