Enhanced Astronomical Source Classification with Integration of Attention Mechanisms and Vision Transformers
arXiv:2408.13634 · doi:10.1007/s10509-024-04357-9
Abstract
Accurate classification of celestial objects is essential for advancing our understanding of the universe. MargNet is a recently developed deep learning-based classifier applied to SDSS DR16 dataset to segregate stars, quasars, and compact galaxies using photometric data. MargNet utilizes a stacked architecture, combining a Convolutional Neural Network (CNN) for image modelling and an Artificial Neural Network (ANN) for modelling photometric parameters. In this study, we propose enhancing MargNet's performance by incorporating attention mechanisms and Vision Transformer (ViT)-based models for processing image data. The attention mechanism allows the model to focus on relevant features and capture intricate patterns within images, effectively distinguishing between different classes of celestial objects. Additionally, we leverage ViTs, a transformer-based deep learning architecture renowned for exceptional performance in image classification tasks. We enhance the model's understanding of complex astronomical images by utilizing ViT's ability to capture global dependencies and contextual information. Our approach uses a curated dataset comprising 240,000 compact and 150,000 faint objects. The models learn classification directly from the data, minimizing human intervention. Furthermore, we explore ViT as a hybrid architecture that uses photometric features and images together as input to predict astronomical objects. Our results demonstrate that the proposed attention mechanism augmented CNN in MargNet marginally outperforms the traditional MargNet and the proposed ViT-based MargNet models. Additionally, the ViT-based hybrid model emerges as the most lightweight and easy-to-train model with classification accuracy similar to that of the best-performing attention-enhanced MargNet.
arXiv comment33 pages, 11 figures. Accepted for publication in APSS
References in corpus (42)
- XGBoost: A Scalable Tree Boosting System
- SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
- Array Programming with NumPy
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Astropy: A Community Python Package for Astronomy
- The Sloan Digital Sky Survey: Technical Summary
- LSST: from Science Drivers to Reference Design and Anticipated Data Products
- Attention Mechanisms in Computer Vision: A Survey
- The Sixteenth Data Release of the Sloan Digital Sky Surveys: First Release from the APOGEE-2 Southern Survey and Full Release of eBOSS Spectra
- The Dark Energy Survey: more than dark energy - an overview
- Subaru Prime Focus Camera -- Suprime-Cam --
- Multimodal Fusion Transformer for Remote Sensing Image Classification
- Data Mining and Machine Learning in Astronomy
- Deep Learning for Real-time Gravitational Wave Detection and Parameter Estimation: Results with Advanced LIGO Data
- The Blanco Cosmology Survey: Data Acquisition, Processing, Calibration, Quality Diagnostics and Data Release
- Star-galaxy Classification Using Deep Convolutional Neural Networks
- Photometric Supernova Classification With Machine Learning
- Photometric redshifts from SDSS images using a Convolutional Neural Network
- SuperNNova: an open-source framework for Bayesian, Neural Network based supernova classification
- Machine Learning for the Zwicky Transient Facility
- Decision Tree Classifiers for Star/Galaxy Separation
- Machine and Deep Learning Applied to Galaxy Morphology -- A Comparative Study
- Application of Convolutional Neural Networks for Stellar Spectral Classification
- Identifying Strong Lenses with Unsupervised Machine Learning using Convolutional Autoencoder
- Star/galaxy separation at faint magnitudes: Application to a simulated Dark Energy Survey
- Separation of pulsar signals from noise with supervised machine learning algorithms
- Star-Galaxy Classification in Multi-Band Optical Imaging
- Detection of Bars in Galaxies using a Deep Convolutional Neural Network
- J-PLUS: Morphological star/galaxy classification by PDF analysis
- Exploring Plain Vision Transformer Backbones for Object Detection
- On the discovery of stars, quasars, and galaxies in the Southern Hemisphere with S-PLUS DR2
- deepCR: Cosmic Ray Rejection with Deep Learning
- Stellar classification from single-band imaging using machine learning
- Galaxy Morphology Classification using Neural Ordinary Differential Equations
- The B band luminosities of QSO host galaxies
- The PAU Survey: star-galaxy classification with multi narrow-band data
- J-PLUS: Support Vector Machine Applied to STAR-GALAXY-QSOClassification
- Photometric identification of compact galaxies, stars and quasars using multiple neural networks
- Morphological Star-Galaxy Separation
- Stellar Spectral Interpolation using Machine Learning
- A Machine Learning Approach to the Detection of Ghosting and Scattered Light Artifacts in Dark Energy Survey Images
- Cosmic Ray Rejection with Attention Augmented Deep Learning