RadarMOSEVE: A Spatial-Temporal Transformer Network for Radar-Only Moving Object Segmentation and Ego-Velocity Estimation

Abstract

Moving object segmentation (MOS) and Ego velocity estimation (EVE) are vital capabilities for mobile systems to achieve full autonomy. Several approaches have attempted to achieve MOSEVE using a LiDAR sensor. However, LiDAR sensors are typically expensive and susceptible to adverse weather conditions. Instead, millimeter-wave radar (MWR) has gained popularity in robotics and autonomous driving for real applications due to its cost-effectiveness and resilience to bad weather. Nonetheless, publicly available MOSEVE datasets and approaches using radar data are limited. Some existing methods adopt point convolutional networks from LiDAR-based approaches, ignoring the specific artifacts and the valuable radial velocity information of radar measurements, leading to suboptimal performance. In this paper, we propose a novel transformer network that effectively addresses the sparsity and noise issues and leverages the radial velocity measurements of radar points using our devised radar self- and cross-attention mechanisms. Based on that, our method achieves accurate EVE of the robot and performs MOS using only radar data simultaneously. To thoroughly evaluate the MOSEVE performance of our method, we annotated the radar points in the public View-of-Delft (VoD) dataset and additionally constructed a new radar dataset in various environments. The experimental results demonstrate the superiority of our approach over existing state-of-the-art methods. The code is available at https://github.com/ORCA-Uboat/RadarMOSEVE.

Accepted at AAAI-24

References in corpus (10)

• Automatic Labeling to Generate Training Data for Online LiDAR-based Moving Object Segmentation
• Building Volumetric Beliefs for Dynamic Environments Exploiting Map-Based Moving Object Segmentation
• SegFlow: Joint Learning for Video Object Segmentation and Optical Flow
• Gaussian Radar Transformer for Semantic Segmentation in Noisy Radar Data
• A Credible and Robust approach to Ego-Motion Estimation using an Automotive Radar
• Unsupervised Moving Object Detection via Contextual Information Separation
• Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervision
• Exploiting Temporal Relations on Radar Perception for Autonomous Driving
• Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving Object Segmentation
• InsMOS: Instance-Aware Moving Object Segmentation in LiDAR Data