Deep Continuous Conditional Random Fields with Asymmetric Inter-object Constraints for Online Multi-object Tracking

Abstract

Online Multi-Object Tracking (MOT) is a challenging problem and has many important applications including intelligence surveillance, robot navigation and autonomous driving. In existing MOT methods, individual object's movements and inter-object relations are mostly modeled separately and relations between them are still manually tuned. In addition, inter-object relations are mostly modeled in a symmetric way, which we argue is not an optimal setting. To tackle those difficulties, in this paper, we propose a Deep Continuous Conditional Random Field (DCCRF) for solving the online MOT problem in a track-by-detection framework. The DCCRF consists of unary and pairwise terms. The unary terms estimate tracked objects' displacements across time based on visual appearance information. They are modeled as deep Convolution Neural Networks, which are able to learn discriminative visual features for tracklet association. The asymmetric pairwise terms model inter-object relations in an asymmetric way, which encourages high-confidence tracklets to help correct errors of low-confidence tracklets and not to be affected by low-confidence ones much. The DCCRF is trained in an end-to-end manner for better adapting the influences of visual information as well as inter-object relations. Extensive experimental comparisons with state-of-the-arts as well as detailed component analysis of our proposed DCCRF on two public benchmarks demonstrate the effectiveness of our proposed MOT framework.

arXiv commentAccepted to TCSVT

References in corpus (10)

• Conditional Random Fields as Recurrent Neural Networks
• MOT16: A Benchmark for Multi-Object Tracking
• Multiple Object Tracking: A Literature Review
• MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking
• Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network
• Siamese Instance Search for Tracking
• Crafting GBD-Net for Object Detection
• Fully-Convolutional Siamese Networks for Object Tracking
• Fusion of Head and Full-Body Detectors for Multi-Object Tracking
• Multi-Person Tracking by Multicut and Deep Matching

Cited by in corpus (12)

• Deep Learning in Video Multi-Object Tracking: A Survey
• FishNet: A Versatile Backbone for Image, Region, and Pixel Level Prediction
• MOANA: An Online Learned Adaptive Appearance Model for Robust Multiple Object Tracking in 3D
• Machine Learning Methods for Data Association in Multi-Object Tracking
• Instance-Aware Representation Learning and Association for Online Multi-Person Tracking
• A CRF-based Framework for Tracklet Inactivation in Online Multi-Object Tracking
• Joint Detection and Tracking in Videos with Identification Features
• End-to-End Learning Deep CRF models for Multi-Object Tracking
• Robust Multi-Modality Multi-Object Tracking
• Online Multiple Pedestrians Tracking using Deep Temporal Appearance Matching Association
• Learning a Proposal Classifier for Multiple Object Tracking
• Online Multi-Object Tracking with delta-GLMB Filter based on Occlusion and Identity Switch Handling