Gaussian Radar Transformer for Semantic Segmentation in Noisy Radar Data

Abstract

Scene understanding is crucial for autonomous robots in dynamic environments for making future state predictions, avoiding collisions, and path planning. Camera and LiDAR perception made tremendous progress in recent years, but face limitations under adverse weather conditions. To leverage the full potential of multi-modal sensor suites, radar sensors are essential for safety critical tasks and are already installed in most new vehicles today. In this paper, we address the problem of semantic segmentation of moving objects in radar point clouds to enhance the perception of the environment with another sensor modality. Instead of aggregating multiple scans to densify the point clouds, we propose a novel approach based on the self-attention mechanism to accurately perform sparse, single-scan segmentation. Our approach, called Gaussian Radar Transformer, includes the newly introduced Gaussian transformer layer, which replaces the softmax normalization by a Gaussian function to decouple the contribution of individual points. To tackle the challenge of the transformer to capture long-range dependencies, we propose our attentive up- and downsampling modules to enlarge the receptive field and capture strong spatial relations. We compare our approach to other state-of-the-art methods on the RadarScenes data set and show superior segmentation quality in diverse environments, even without exploiting temporal information.

arXiv commentAccepted for publication in IEEE Robotics and Automation Letters (RA-L)

References in corpus (7)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• PCT: Point cloud transformer
• Seeing Around Street Corners: Non-Line-of-Sight Detection and Tracking In-the-Wild Using Doppler Radar
• Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction
• Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding
• (AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Network
• Stratified Transformer for 3D Point Cloud Segmentation

Cited by in corpus (8)

• Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review
• A New Wave in Robotics: Survey on Recent mmWave Radar Applications in Robotics
• EFEAR-4D: Ego-Velocity Filtering for Efficient and Accurate 4D radar Odometry
• Improved Multi-Scale Grid Rendering of Point Clouds for Radar Object Detection Networks
• Radar Velocity Transformer: Single-scan Moving Object Segmentation in Noisy Radar Point Clouds
• RadarMOSEVE: A Spatial-Temporal Transformer Network for Radar-Only Moving Object Segmentation and Ego-Velocity Estimation
• SemRaFiner: Panoptic Segmentation in Sparse and Noisy Radar Point Clouds
• Radar Tracker: Moving Instance Tracking in Sparse and Noisy Radar Point Clouds