PYRO-NN: Python Reconstruction Operators in Neural Networks

Abstract

Purpose: Recently, several attempts were conducted to transfer deep learning to medical image reconstruction. An increasingly number of publications follow the concept of embedding the CT reconstruction as a known operator into a neural network. However, most of the approaches presented lack an efficient CT reconstruction framework fully integrated into deep learning environments. As a result, many approaches are forced to use workarounds for mathematically unambiguously solvable problems. Methods: PYRO-NN is a generalized framework to embed known operators into the prevalent deep learning framework Tensorflow. The current status includes state-of-the-art parallel-, fan- and cone-beam projectors and back-projectors accelerated with CUDA provided as Tensorflow layers. On top, the framework provides a high level Python API to conduct FBP and iterative reconstruction experiments with data from real CT systems. Results: The framework provides all necessary algorithms and tools to design end-to-end neural network pipelines with integrated CT reconstruction algorithms. The high level Python API allows a simple use of the layers as known from Tensorflow. To demonstrate the capabilities of the layers, the framework comes with three baseline experiments showing a cone-beam short scan FDK reconstruction, a CT reconstruction filter learning setup, and a TV regularized iterative reconstruction. All algorithms and tools are referenced to a scientific publication and are compared to existing non deep learning reconstruction frameworks. The framework is available as open-source software at \url{https://github.com/csyben/PYRO-NN}. Conclusions: PYRO-NN comes with the prevalent deep learning framework Tensorflow and allows to setup end-to-end trainable neural networks in the medical image reconstruction context. We believe that the framework will be a step towards reproducible research

V1: Submitted to Medical Physics, 11 pages, 7 figures

References in corpus (4)

• Deep Convolutional Neural Network for Inverse Problems in Imaging
• Image reconstruction by domain transform manifold learning
• Learned Primal-dual Reconstruction
• PYRO-NN: Python Reconstruction Operators in Neural Networks

Cited by in corpus (10)

• PYRO-NN: Python Reconstruction Operators in Neural Networks
• WNet: A data-driven dual-domain denoising model for sparse-view computed tomography with a trainable reconstruction layer
• Known Operator Learning and Hybrid Machine Learning in Medical Imaging -- A Review of the Past, the Present, and the Future
• Ultra Low-Parameter Denoising: Trainable Bilateral Filter Layers in Computed Tomography
• Artifact Reduction in 3D and 4D Cone-beam Computed Tomography Images with Deep Learning -- A Review
• End-to-end Memory-Efficient Reconstruction for Cone Beam CT
• Vanishing Point Detection with Direct and Transposed Fast Hough Transform inside the neural network
• A gradient-based approach to fast and accurate head motion compensation in cone-beam CT
• Convergence analysis of pixel-driven Radon and fanbeam transforms
• A Conditional Point Cloud Diffusion Model for Deformable Liver Motion Tracking Via a Single Arbitrarily-Angled X-ray Projection