A Convolutional Vision Transformer for Semantic Segmentation of Side-Scan Sonar Data

Abstract

Distinguishing among different marine benthic habitat characteristics is of key importance in a wide set of seabed operations ranging from installations of oil rigs to laying networks of cables and monitoring the impact of humans on marine ecosystems. The Side-Scan Sonar (SSS) is a widely used imaging sensor in this regard. It produces high-resolution seafloor maps by logging the intensities of sound waves reflected back from the seafloor. In this work, we leverage these acoustic intensity maps to produce pixel-wise categorization of different seafloor types. We propose a novel architecture adapted from the Vision Transformer (ViT) in an encoder-decoder framework. Further, in doing so, the applicability of ViTs is evaluated on smaller datasets. To overcome the lack of CNN-like inductive biases, thereby making ViTs more conducive to applications in low data regimes, we propose a novel feature extraction module to replace the Multi-layer Perceptron (MLP) block within transformer layers and a novel module to extract multiscale patch embeddings. A lightweight decoder is also proposed to complement this design in order to further boost multiscale feature extraction. With the modified architecture, we achieve state-of-the-art results and also meet real-time computational requirements. We make our code available at ~\url{https://github.com/hayatrajani/s3seg-vit

arXiv commentSubmitted to Ocean Engineering special issue "Autonomous Marine Robotics Operations"

References in corpus (9)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation
• ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
• XCiT: Cross-Covariance Image Transformers
• MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
• Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
• Efficient Self-Ensemble for Semantic Segmentation
• MoCoViT: Mobile Convolutional Vision Transformer
• SimA: Simple Softmax-free Attention for Vision Transformers

Cited by in corpus (1)

• EchoPT: A Pretrained Transformer Architecture that Predicts 2D In-Air Sonar Images for Mobile Robotics