Co-training for Demographic Classification Using Deep Learning from Label Proportions

Abstract

Deep learning algorithms have recently produced state-of-the-art accuracy in many classification tasks, but this success is typically dependent on access to many annotated training examples. For domains without such data, an attractive alternative is to train models with light, or distant supervision. In this paper, we introduce a deep neural network for the Learning from Label Proportion (LLP) setting, in which the training data consist of bags of unlabeled instances with associated label distributions for each bag. We introduce a new regularization layer, Batch Averager, that can be appended to the last layer of any deep neural network to convert it from supervised learning to LLP. This layer can be implemented readily with existing deep learning packages. To further support domains in which the data consist of two conditionally independent feature views (e.g. image and text), we propose a co-training algorithm that iteratively generates pseudo bags and refits the deep LLP model to improve classification accuracy. We demonstrate our models on demographic attribute classification (gender and race/ethnicity), which has many applications in social media analysis, public health, and marketing. We conduct experiments to predict demographics of Twitter users based on their tweets and profile image, without requiring any user-level annotations for training. We find that the deep LLP approach outperforms baselines for both text and image features separately. Additionally, we find that co-training algorithm improves image and text classification by 4% and 8% absolute F1, respectively. Finally, an ensemble of text and image classifiers further improves the absolute F1 measure by 4% on average.

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Deep Residual Learning for Image Recognition
• Rethinking the Inception Architecture for Computer Vision
• Xception: Deep Learning with Depthwise Separable Convolutions
• Rigid-Motion Scattering for Texture Classification
• Deep Learning Face Attributes in the Wild
• PANDA: Pose Aligned Networks for Deep Attribute Modeling

Cited by in corpus (8)

• An Overview of Deep Semi-Supervised Learning
• Demographic Inference and Representative Population Estimates from Multilingual Social Media Data
• Crossbar-Net: A Novel Convolutional Network for Kidney Tumor Segmentation in CT Images
• Deep multi-class learning from label proportions
• Learning from Label Proportions with Consistency Regularization
• Training image classifiers using Semi-Weak Label Data
• Two-stage Training for Learning from Label Proportions
• Fast learning from label proportions with small bags