A Survey on Deep Learning-Based Monocular Spacecraft Pose Estimation: Current State, Limitations and Prospects

Abstract

Estimating the pose of an uncooperative spacecraft is an important computer vision problem for enabling the deployment of automatic vision-based systems in orbit, with applications ranging from on-orbit servicing to space debris removal. Following the general trend in computer vision, more and more works have been focusing on leveraging Deep Learning (DL) methods to address this problem. However and despite promising research-stage results, major challenges preventing the use of such methods in real-life missions still stand in the way. In particular, the deployment of such computation-intensive algorithms is still under-investigated, while the performance drop when training on synthetic and testing on real images remains to mitigate. The primary goal of this survey is to describe the current DL-based methods for spacecraft pose estimation in a comprehensive manner. The secondary goal is to help define the limitations towards the effective deployment of DL-based spacecraft pose estimation solutions for reliable autonomous vision-based applications. To this end, the survey first summarises the existing algorithms according to two approaches: hybrid modular pipelines and direct end-to-end regression methods. A comparison of algorithms is presented not only in terms of pose accuracy but also with a focus on network architectures and models' sizes keeping potential deployment in mind. Then, current monocular spacecraft pose estimation datasets used to train and test these methods are discussed. The data generation methods: simulators and testbeds, the domain gap and the performance drop between synthetically generated and lab/space collected images and the potential solutions are also discussed. Finally, the paper presents open research questions and future directions in the field, drawing parallels with other computer vision applications.

References in corpus (23)

• YOLOv4: Optimal Speed and Accuracy of Object Detection
• YOLOX: Exceeding YOLO Series in 2021
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• An Overview of Multi-Task Learning in Deep Neural Networks
• Deep Visual Domain Adaptation: A Survey
• A Survey of Deep Learning-based Object Detection
• Deep Learning in Video Multi-Object Tracking: A Survey
• High-Resolution Representations for Labeling Pixels and Regions
• A tutorial on conformal prediction
• Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation
• Satellite Pose Estimation Challenge: Dataset, Competition Design and Results
• PP-YOLO: An Effective and Efficient Implementation of Object Detector
• A Survey on Neural Architecture Search
• Image Segmentation Algorithms Overview
• Robust Multi-Task Learning and Online Refinement for Spacecraft Pose Estimation across Domain Gap
• EfficientPose: An efficient, accurate and scalable end-to-end 6D multi object pose estimation approach
• Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering
• MobilePose: Real-Time Pose Estimation for Unseen Objects with Weak Shape Supervision
• A Principled Approach for Learning Task Similarity in Multitask Learning
• GPU4S: Embedded GPUs in Space -- Latest Project Updates
• Lessons from a Space Lab -- An Image Acquisition Perspective
• Source-Free Unsupervised Domain Adaptation: A Survey
• Unseen Object 6D Pose Estimation: A Benchmark and Baselines

Cited by in corpus (5)

• Stereo Event-based, 6-DOF Pose Tracking for Uncooperative Spacecraft
• Lessons from a Space Lab -- An Image Acquisition Perspective
• Object-centric Reconstruction and Tracking of Dynamic Unknown Objects using 3D Gaussian Splatting
• Event-RGB Fusion for Spacecraft Pose Estimation Under Harsh Lighting
• FPG-NAS: FLOPs-Aware Gated Differentiable Neural Architecture Search for Efficient 6DoF Pose Estimation