Large Scale Image Segmentation with Structured Loss based Deep Learning for Connectome Reconstruction

Abstract

We present a method combining affinity prediction with region agglomeration, which improves significantly upon the state of the art of neuron segmentation from electron microscopy (EM) in accuracy and scalability. Our method consists of a 3D U-NET, trained to predict affinities between voxels, followed by iterative region agglomeration. We train using a structured loss based on MALIS, encouraging topologically correct segmentations obtained from affinity thresholding. Our extension consists of two parts: First, we present a quasi-linear method to compute the loss gradient, improving over the original quadratic algorithm. Second, we compute the gradient in two separate passes to avoid spurious gradient contributions in early training stages. Our predictions are accurate enough that simple learning-free percentile-based agglomeration outperforms more involved methods used earlier on inferior predictions. We present results on three diverse EM datasets, achieving relative improvements over previous results of 27%, 15%, and 250%. Our findings suggest that a single method can be applied to both nearly isotropic block-face EM data and anisotropic serial sectioned EM data. The runtime of our method scales linearly with the size of the volume and achieves a throughput of about 2.6 seconds per megavoxel, qualifying our method for the processing of very large datasets.

References in corpus (3)

• Image Segmentation by Size-Dependent Single Linkage Clustering of a Watershed Basin Graph
• Flood-Filling Networks
• Anisotropic EM Segmentation by 3D Affinity Learning and Agglomeration

Cited by in corpus (25)

• Segmentation in large-scale cellular electron microscopy with deep learning: A literature survey
• The Mutex Watershed and its Objective: Efficient, Parameter-Free Graph Partitioning
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning
• Improving Representation of High-frequency Components for Medical Visual Foundation Models
• Enhance Connectivity of Promising Regions for Sampling-based Path Planning
• clDice -- A Novel Topology-Preserving Loss Function for Tubular Structure Segmentation
• Cylindrical Shape Decomposition for 3D Segmentation of Tubular Objects
• Chunkflow: Distributed Hybrid Cloud Processing of Large 3D Images by Convolutional Nets
• An Error Detection and Correction Framework for Connectomics
• Ultrametric Fitting by Gradient Descent
• Triplet-Watershed for Hyperspectral Image Classification
• Large-scale image segmentation based on distributed clustering algorithms
• PatchPerPix for Instance Segmentation
• An Auxiliary Task for Learning Nuclei Segmentation in 3D Microscopy Images
• Enforcing connectivity of 3D linear structures using their 2D projections
• Learning and Segmenting Dense Voxel Embeddings for 3D Neuron Reconstruction
• Probabilistic Deep Learning for Instance Segmentation
• Universal Vessel Segmentation for Multi-Modality Retinal Images
• Detecting Synapse Location and Connectivity by Signed Proximity Estimation and Pruning with Deep Nets
• CAPE: Connectivity-Aware Path Enforcement Loss for Curvilinear Structure Delineation
• HIVE-Net: Centerline-Aware HIerarchical View-Ensemble Convolutional Network for Mitochondria Segmentation in EM Images
• Scaling Distributed Training of Flood-Filling Networks on HPC Infrastructure for Brain Mapping
• End-to-End Learned Random Walker for Seeded Image Segmentation
• Stateless actor-critic for instance segmentation with high-level priors
• AxonEM Dataset: 3D Axon Instance Segmentation of Brain Cortical Regions