Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification
arXiv:1804.10069
Abstract
Video representation learning is a vital problem for classification task. Recently, a promising unsupervised paradigm termed self-supervised learning has emerged, which explores inherent supervisory signals implied in massive data for feature learning via solving auxiliary tasks. However, existing methods in this regard suffer from two limitations when extended to video classification. First, they focus only on a single task, whereas ignoring complementarity among different task-specific features and thus resulting in suboptimal video representation. Second, high computational and memory cost hinders their application in real-world scenarios. In this paper, we propose a graph-based distillation framework to address these problems: (1) We propose logits graph and representation graph to transfer knowledge from multiple self-supervised tasks, where the former distills classifier-level knowledge by solving a multi-distribution joint matching problem, and the latter distills internal feature knowledge from pairwise ensembled representations with tackling the challenge of heterogeneity among different features; (2) The proposal that adopts a teacher-student framework can reduce the redundancy of knowledge learnt from teachers dramatically, leading to a lighter student model that solves classification task more efficiently. Experimental results on 3 video datasets validate that our proposal not only helps learn better video representation but also compress model for faster inference.
arXiv comment7 pages, accepted by International Joint Conference on Artificial Intelligence (IJCAI) 2018
References in corpus (7)
- Distilling the Knowledge in a Neural Network
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
- Unsupervised Learning of Visual Representations using Videos
- Learning to See by Moving
- Pose from Action: Unsupervised Learning of Pose Features based on Motion
- Graph Distillation for Action Detection with Privileged Modalities