Using Cross-Model EgoSupervision to Learn Cooperative Basketball Intention

Abstract

We present a first-person method for cooperative basketball intention prediction: we predict with whom the camera wearer will cooperate in the near future from unlabeled first-person images. This is a challenging task that requires inferring the camera wearer's visual attention, and decoding the social cues of other players. Our key observation is that a first-person view provides strong cues to infer the camera wearer's momentary visual attention, and his/her intentions. We exploit this observation by proposing a new cross-model EgoSupervision learning scheme that allows us to predict with whom the camera wearer will cooperate in the near future, without using manually labeled intention labels. Our cross-model EgoSupervision operates by transforming the outputs of a pretrained pose-estimation network, into pseudo ground truth labels, which are then used as a supervisory signal to train a new network for a cooperative intention task. We evaluate our method, and show that it achieves similar or even better accuracy than the fully supervised methods do.

References in corpus (14)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Distilling the Knowledge in a Neural Network
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• Fully Convolutional Networks for Semantic Segmentation
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Pyramid Scene Parsing Network
• Deep Contrast Learning for Salient Object Detection
• Cross Modal Distillation for Supervision Transfer
• First Person Action-Object Detection with EgoNet
• DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection