Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue

Abstract

Histological staining is a vital step used to diagnose various diseases and has been used for more than a century to provide contrast to tissue sections, rendering the tissue constituents visible for microscopic analysis by medical experts. However, this process is time-consuming, labor-intensive, expensive and destructive to the specimen. Recently, the ability to virtually-stain unlabeled tissue sections, entirely avoiding the histochemical staining step, has been demonstrated using tissue-stain specific deep neural networks. Here, we present a new deep learning-based framework which generates virtually-stained images using label-free tissue, where different stains are merged following a micro-structure map defined by the user. This approach uses a single deep neural network that receives two different sources of information at its input: (1) autofluorescence images of the label-free tissue sample, and (2) a digital staining matrix which represents the desired microscopic map of different stains to be virtually generated at the same tissue section. This digital staining matrix is also used to virtually blend existing stains, digitally synthesizing new histological stains. We trained and blindly tested this virtual-staining network using unlabeled kidney tissue sections to generate micro-structured combinations of Hematoxylin and Eosin (H&E), Jones silver stain, and Masson's Trichrome stain. Using a single network, this approach multiplexes virtual staining of label-free tissue with multiple types of stains and paves the way for synthesizing new digital histological stains that can be created on the same tissue cross-section, which is currently not feasible with standard histochemical staining methods.

arXiv comment19 pages, 5 figures, 2 tables

References in corpus (6)

• Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue
• Deep Learning Microscopy
• PhaseStain: Digital staining of label-free quantitative phase microscopy images using deep learning
• Deep learning enhanced mobile-phone microscopy
• Cross-modality deep learning brings bright-field microscopy contrast to holography
• Unsupervised Prostate Cancer Detection on H&E using Convolutional Adversarial Autoencoders

Cited by in corpus (26)

• Deep Learning-enabled Virtual Histological Staining of Biological Samples
• Deep learning-based transformation of the H&E stain into special stains
• Misalignment Resilient Diffractive Optical Networks
• Self-supervised learning of hologram reconstruction using physics consistency
• Deep learning in biomedical optics
• Scale-, shift- and rotation-invariant diffractive optical networks
• Label-free virtual HER2 immunohistochemical staining of breast tissue using deep learning
• Nonlinear imaging of nanoscale topological corner states
• Nonlinear imaging of nanoscale topological corner states
• Virtual histological staining of unlabeled autopsy tissue
• Virtual stain transfer in histology via cascaded deep neural networks
• Digital staining in optical microscopy using deep learning -- a review
• Extracting quantitative biological information from brightfield cell images using deep learning
• Virtual staining of defocused autofluorescence images of unlabeled tissue using deep neural networks
• Pixel super-resolved virtual staining of label-free tissue using diffusion models
• Virtual birefringence imaging and histological staining of amyloid deposits in label-free tissue using autofluorescence microscopy and deep learning
• Virtual Gram staining of label-free bacteria using darkfield microscopy and deep learning
• A robust and scalable framework for hallucination detection in virtual tissue staining and digital pathology
• Automated Whole Slide Imaging for Label-Free Histology using Photon Absorption Remote Sensing Microscopy
• Virtual Staining of Label-Free Tissue in Imaging Mass Spectrometry
• High-throughput fast full-color digital pathology based on Fourier ptychographic microscopy via color transfer
• Cycle Consistency-based Uncertainty Quantification of Neural Networks in Inverse Imaging Problems
• Morphomics via Next-generation Electron Microscopy
• Label-free evaluation of lung and heart transplant biopsies using tissue autofluorescence-based virtual staining
• MIPHEI-ViT: Multiplex Immunofluorescence Prediction from H&E Images using ViT Foundation Models
• Detecting immune cells with label-free two-photon autofluorescence and deep learning