On Class Imbalance and Background Filtering in Visual Relationship Detection

Abstract

In this paper we investigate the problems of class imbalance and irrelevant relationships in Visual Relationship Detection (VRD). State-of-the-art deep VRD models still struggle to predict uncommon classes, limiting their applicability. Moreover, many methods are incapable of properly filtering out background relationships while predicting relevant ones. Although these problems are very apparent, they have both been overlooked so far. We analyse why this is the case and propose modifications to both model and training to alleviate the aforementioned issues, as well as suggesting new measures to complement existing ones and give a more holistic picture of the efficacy of a model.

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• On Calibration of Modern Neural Networks
• Cost-aware Pre-training for Multiclass Cost-sensitive Deep Learning
• Deep Imbalanced Learning for Face Recognition and Attribute Prediction
• Class Rectification Hard Mining for Imbalanced Deep Learning
• Graph R-CNN for Scene Graph Generation
• Deep Imbalanced Attribute Classification using Visual Attention Aggregation
• Visual Relationship Prediction via Label Clustering and Incorporation of Depth Information