Label Embedding Network: Learning Label Representation for Soft Training of Deep Networks

Abstract

We propose a method, called Label Embedding Network, which can learn label representation (label embedding) during the training process of deep networks. With the proposed method, the label embedding is adaptively and automatically learned through back propagation. The original one-hot represented loss function is converted into a new loss function with soft distributions, such that the originally unrelated labels have continuous interactions with each other during the training process. As a result, the trained model can achieve substantially higher accuracy and with faster convergence speed. Experimental results based on competitive tasks demonstrate the effectiveness of the proposed method, and the learned label embedding is reasonable and interpretable. The proposed method achieves comparable or even better results than the state-of-the-art systems. The source code is available at \url{https://github.com/lancopku/LabelEmb}.

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• Wide Residual Networks
• Training Very Deep Networks
• Multi-Label Prediction via Compressed Sensing
• A Semantic Relevance Based Neural Network for Text Summarization and Text Simplification

Cited by in corpus (12)

• Hierarchy-based Image Embeddings for Semantic Image Retrieval
• SGM: Sequence Generation Model for Multi-label Classification
• Deconvolution-Based Global Decoding for Neural Machine Translation
• Bag-of-Words as Target for Neural Machine Translation
• A Hierarchical End-to-End Model for Jointly Improving Text Summarization and Sentiment Classification
• Global Semantic Consistency for Zero-Shot Learning
• Improving Social Media Text Summarization by Learning Sentence Weight Distribution
• SHREWD: Semantic Hierarchy-based Relational Embeddings for Weakly-supervised Deep Hashing
• GREEN: a Graph REsidual rE-ranking Network for Grading Diabetic Retinopathy
• LabelEnc: A New Intermediate Supervision Method for Object Detection
• Exploration on Grounded Word Embedding: Matching Words and Images with Image-Enhanced Skip-Gram Model
• Learning to hash with semantic similarity metrics and empirical KL divergence