An Overview on Data Representation Learning: From Traditional Feature Learning to Recent Deep Learning

Abstract

Since about 100 years ago, to learn the intrinsic structure of data, many representation learning approaches have been proposed, including both linear ones and nonlinear ones, supervised ones and unsupervised ones. Particularly, deep architectures are widely applied for representation learning in recent years, and have delivered top results in many tasks, such as image classification, object detection and speech recognition. In this paper, we review the development of data representation learning methods. Specifically, we investigate both traditional feature learning algorithms and state-of-the-art deep learning models. The history of data representation learning is introduced, while available resources (e.g. online course, tutorial and book information) and toolboxes are provided. Finally, we conclude this paper with remarks and some interesting research directions on data representation learning.

About 20 pages. Submitted to Journal of Finance and Data Science as an invited paper

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning in Neural Networks: An Overview
• Sequence to Sequence Learning with Neural Networks
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Deep Residual Learning for Image Recognition
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Attention-Based Models for Speech Recognition
• Going Deeper with Convolutions
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• Deep Learning in Bioinformatics

Cited by in corpus (2)

• Expressive TTS Training with Frame and Style Reconstruction Loss
• Learning Optimal Representations with the Decodable Information Bottleneck