Robust Multi-Task Learning and Online Refinement for Spacecraft Pose Estimation across Domain Gap

Abstract

This work presents Spacecraft Pose Network v2 (SPNv2), a Convolutional Neural Network (CNN) for pose estimation of noncooperative spacecraft across domain gap. SPNv2 is a multi-scale, multi-task CNN which consists of a shared multi-scale feature encoder and multiple prediction heads that perform different tasks on a shared feature output. These tasks are all related to detection and pose estimation of a target spacecraft from an image, such as prediction of pre-defined satellite keypoints, direct pose regression, and binary segmentation of the satellite foreground. It is shown that by jointly training on different yet related tasks with extensive data augmentations on synthetic images only, the shared encoder learns features that are common across image domains that have fundamentally different visual characteristics compared to synthetic images. This work also introduces Online Domain Refinement (ODR) which refines the parameters of the normalization layers of SPNv2 on the target domain images online at deployment. Specifically, ODR performs self-supervised entropy minimization of the predicted satellite foreground, thereby improving the CNN's performance on the target domain images without their pose labels and with minimal computational efforts. The GitHub repository for SPNv2 is available at https://github.com/tpark94/spnv2.

Accepted to Advances in Space Research; fixed error on reporting translation from heatmaps

References in corpus (5)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Unsupervised Domain Adaptation by Backpropagation
• Model Adaptation: Unsupervised Domain Adaptation without Source Data
• EfficientPose: An efficient, accurate and scalable end-to-end 6D multi object pose estimation approach
• Robotic Testbed for Rendezvous and Optical Navigation: Multi-Source Calibration and Machine Learning Use Cases

Cited by in corpus (10)

• A Survey on Deep Learning-Based Monocular Spacecraft Pose Estimation: Current State, Limitations and Prospects
• SU-Net: Pose estimation network for non-cooperative spacecraft on-orbit
• Online Supervised Training of Spaceborne Vision during Proximity Operations using Adaptive Kalman Filtering
• Bridging the Domain Gap for Flight-Ready Spaceborne Vision
• Domain Generalization for In-Orbit 6D Pose Estimation
• Test-Time Adaptation for Keypoint-Based Spacecraft Pose Estimation Based on Predicted-View Synthesis
• Event-RGB Fusion for Spacecraft Pose Estimation Under Harsh Lighting
• Improved 3D Gaussian Splatting of Unknown Spacecraft Structure Using Space Environment Illumination Knowledge
• Uncertainty-Aware Knowledge Distillation for Compact and Efficient 6DoF Pose Estimation
• FPG-NAS: FLOPs-Aware Gated Differentiable Neural Architecture Search for Efficient 6DoF Pose Estimation