Forced to Learn: Discovering Disentangled Representations Without Exhaustive Labels

Abstract

Learning a better representation with neural networks is a challenging problem, which was tackled extensively from different prospectives in the past few years. In this work, we focus on learning a representation that could be used for a clustering task and introduce two novel loss components that substantially improve the quality of produced clusters, are simple to apply to an arbitrary model and cost function, and do not require a complicated training procedure. We evaluate them on two most common types of models, Recurrent Neural Networks and Convolutional Neural Networks, showing that the approach we propose consistently improves the quality of KMeans clustering in terms of Adjusted Mutual Information score and outperforms previously proposed methods.

Abstract accepted at ICLR 2017 Workshop: https://openreview.net/pdf?id=SkCmfeSFg

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Visualizing and Understanding Recurrent Networks
• Discovering Hidden Factors of Variation in Deep Networks
• Incremental Sequence Learning