Sketch-a-Net that Beats Humans

Abstract

We propose a multi-scale multi-channel deep neural network framework that, for the first time, yields sketch recognition performance surpassing that of humans. Our superior performance is a result of explicitly embedding the unique characteristics of sketches in our model: (i) a network architecture designed for sketch rather than natural photo statistics, (ii) a multi-channel generalisation that encodes sequential ordering in the sketching process, and (iii) a multi-scale network ensemble with joint Bayesian fusion that accounts for the different levels of abstraction exhibited in free-hand sketches. We show that state-of-the-art deep networks specifically engineered for photos of natural objects fail to perform well on sketch recognition, regardless whether they are trained using photo or sketch. Our network on the other hand not only delivers the best performance on the largest human sketch dataset to date, but also is small in size making efficient training possible using just CPUs.

Accepted to BMVC 2015 (oral)

References in corpus (5)

• Improving neural networks by preventing co-adaptation of feature detectors
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• MatConvNet - Convolutional Neural Networks for MATLAB
• Sketch-based 3D Shape Retrieval using Convolutional Neural Networks
• Deep Networks with Internal Selective Attention through Feedback Connections

Cited by in corpus (15)

• Generalisation and Sharing in Triplet Convnets for Sketch based Visual Search
• Learning Robust Representations via Multi-View Information Bottleneck
• The Devil is in the Middle: Exploiting Mid-level Representations for Cross-Domain Instance Matching
• Multi-Graph Transformer for Free-Hand Sketch Recognition
• Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval
• The SP theory of intelligence: distinctive features and advantages
• Sketch-R2CNN: An Attentive Network for Vector Sketch Recognition
• SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval
• Sequential Dual Deep Learning with Shape and Texture Features for Sketch Recognition
• Zero-Shot Sketch-Image Hashing
• Cross-modal Subspace Learning for Fine-grained Sketch-based Image Retrieval
• Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketches
• Instance-level Sketch-based Retrieval by Deep Triplet Classification Siamese Network
• Stacked Semantic-Guided Network for Zero-Shot Sketch-Based Image Retrieval
• Asymmetric Feature Maps with Application to Sketch Based Retrieval