Classification of sparsely labeled spatio-temporal data through semi-supervised adversarial learning

Abstract

In recent years, Generative Adversarial Networks (GAN) have emerged as a powerful method for learning the mapping from noisy latent spaces to realistic data samples in high-dimensional space. So far, the development and application of GANs have been predominantly focused on spatial data such as images. In this project, we aim at modeling of spatio-temporal sensor data instead, i.e. dynamic data over time. The main goal is to encode temporal data into a global and low-dimensional latent vector that captures the dynamics of the spatio-temporal signal. To this end, we incorporate auto-regressive RNNs, Wasserstein GAN loss, spectral norm weight constraints and a semi-supervised learning scheme into InfoGAN, a method for retrieval of meaningful latents in adversarial learning. To demonstrate the modeling capability of our method, we encode full-body skeletal human motion from a large dataset representing 60 classes of daily activities, recorded in a multi-Kinect setup. Initial results indicate competitive classification performance of the learned latent representations, compared to direct CNN/RNN inference. In future work, we plan to apply this method on a related problem in the medical domain, i.e. on recovery of meaningful latents in gait analysis of patients with vertigo and balance disorders.

References in corpus (17)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Improved Training of Wasserstein GANs
• Improved Techniques for Training GANs
• Generating Videos with Scene Dynamics
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Adversarial Feature Learning
• Adversarially Learned Inference
• Achieving Human Parity in Conversational Speech Recognition
• NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
• Towards Principled Methods for Training Generative Adversarial Networks
• Generative Adversarial Imitation Learning
• Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
• Adversarial Learning for Neural Dialogue Generation
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment
• Language Generation with Recurrent Generative Adversarial Networks without Pre-training