RadarMOSEVE: A Spatial-Temporal Transformer Network for Radar-Only Moving Object Segmentation and Ego-Velocity Estimation

Abstract

Moving object segmentation (MOS) and Ego velocity estimation (EVE) are vital capabilities for mobile systems to achieve full autonomy. Several approaches have attempted to achieve MOSEVE using a LiDAR sensor. However, LiDAR sensors are typically expensive and susceptible to adverse weather conditions. Instead, millimeter-wave radar (MWR) has gained popularity in robotics and autonomous driving for real applications due to its cost-effectiveness and resilience to bad weather. Nonetheless, publicly available MOSEVE datasets and approaches using radar data are limited. Some existing methods adopt point convolutional networks from LiDAR-based approaches, ignoring the specific artifacts and the valuable radial velocity information of radar measurements, leading to suboptimal performance. In this paper, we propose a novel transformer network that effectively addresses the sparsity and noise issues and leverages the radial velocity measurements of radar points using our devised radar self- and cross-attention mechanisms. Based on that, our method achieves accurate EVE of the robot and performs MOS using only radar data simultaneously. To thoroughly evaluate the MOSEVE performance of our method, we annotated the radar points in the public View-of-Delft (VoD) dataset and additionally constructed a new radar dataset in various environments. The experimental results demonstrate the superiority of our approach over existing state-of-the-art methods. The code is available at https://github.com/ORCA-Uboat/RadarMOSEVE.

arXiv commentAccepted at AAAI-24

References in corpus (16)

• Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data
• Automatic Labeling to Generate Training Data for Online LiDAR-based Moving Object Segmentation
• Unsupervised Video Object Segmentation for Deep Reinforcement Learning
• Building Volumetric Beliefs for Dynamic Environments Exploiting Map-Based Moving Object Segmentation
• Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling
• SegFlow: Joint Learning for Video Object Segmentation and Optical Flow
• Self-Supervised Scene Flow Estimation with 4-D Automotive Radar
• Gaussian Radar Transformer for Semantic Segmentation in Noisy Radar Data
• A Credible and Robust approach to Ego-Motion Estimation using an Automotive Radar
• See Through Smoke: Robust Indoor Mapping with Low-cost mmWave Radar
• Unsupervised Moving Object Detection via Contextual Information Separation
• Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions
• Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervision
• Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving Object Segmentation
• InsMOS: Instance-Aware Moving Object Segmentation in LiDAR Data
• Exploiting Temporal Relations on Radar Perception for Autonomous Driving