A broadband hyperspectral image sensor with high spatio-temporal resolution

Abstract

Hyperspectral imaging provides high-dimensional spatial-temporal-spectral information revealing intrinsic matter characteristics. Here we report an on-chip computational hyperspectral imaging framework with high spatial and temporal resolution. By integrating different broadband modulation materials on the image sensor chip, the target spectral information is non-uniformly and intrinsically coupled on each pixel with high light throughput. Using intelligent reconstruction algorithms, multi-channel images can be recovered from each frame, realizing real-time hyperspectral imaging. Following such a framework, we for the first time fabricated a broadband VIS-NIR (400-1700 nm) hyperspectral imaging sensor using photolithography, with an average light throughput of 74.8% and 96 wavelength channels. The demonstrated resolution is 1,024*1,024 pixels at 124 fps. We demonstrated its wide applications including chlorophyll and sugar quantification for intelligent agriculture, blood oxygen and water quality monitoring for human health, textile classification and apple bruise detection for industrial automation, and remote lunar detection for astronomy. The integrated hyperspectral image sensor weighs only tens of grams, and can be assembled on various resource-limited platforms or equipped with off-the-shelf optical systems. The technique transforms the challenge of high-dimensional imaging from a high-cost manufacturing and cumbersome system to one that is solvable through on-chip compression and agile computation.

References in corpus (5)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Deep Learning for Hyperspectral Image Classification: An Overview
• Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array
• Restormer: Efficient Transformer for High-Resolution Image Restoration
• Uformer: A General U-Shaped Transformer for Image Restoration

Cited by in corpus (4)

• Medical Image Classification with KAN-Integrated Transformers and Dilated Neighborhood Attention
• Deep learning-based single-shot computational spectrometer using multilayer thin films
• Infrared Single-Pixel Hyperspectral Imaging via Spatial-Temporal Multiplexing
• Compact Low-Cost Hyperspectral Imaging via Angular-to-Spectral Diversity Conversion