Reducing Data Complexity using Autoencoders with Class-informed Loss Functions

Abstract

Available data in machine learning applications is becoming increasingly complex, due to higher dimensionality and difficult classes. There exists a wide variety of approaches to measuring complexity of labeled data, according to class overlap, separability or boundary shapes, as well as group morphology. Many techniques can transform the data in order to find better features, but few focus on specifically reducing data complexity. Most data transformation methods mainly treat the dimensionality aspect, leaving aside the available information within class labels which can be useful when classes are somehow complex. This paper proposes an autoencoder-based approach to complexity reduction, using class labels in order to inform the loss function about the adequacy of the generated variables. This leads to three different new feature learners, Scorer, Skaler and Slicer. They are based on Fisher's discriminant ratio, the Kullback-Leibler divergence and least-squares support vector machines, respectively. They can be applied as a preprocessing stage for a binary classification problem. A thorough experimentation across a collection of 27 datasets and a range of complexity and classification metrics shows that class-informed autoencoders perform better than 4 other popular unsupervised feature extraction techniques, especially when the final objective is using the data for a classification task.

This paper has been accepted for publication by IEEE Transactions on Pattern Analysis and Machine Intelligence

References in corpus (4)

• Semi-Supervised Learning with Deep Generative Models
• A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines
• An analysis on the use of autoencoders for representation learning: fundamentals, learning task case studies, explainability and challenges
• Classification of Imbalanced Data with a Geometric Digraph Family

Cited by in corpus (1)

• MoE-Enhanced Explainable Deep Manifold Transformation for Complex Data Embedding and Visualization