Finding Strong Gravitational Lenses Through Self-Attention

Abstract

The upcoming large scale surveys like LSST are expected to find approximately $10^5$ strong gravitational lenses by analysing data of many orders of magnitude larger than those in contemporary astronomical surveys. In this case, non-automated techniques will be highly challenging and time-consuming, even if they are possible at all. We propose a new automated architecture based on the principle of self-attention to find strong gravitational lenses. The advantages of self-attention-based encoder models over convolution neural networks are investigated, and ways to optimise the outcome of encoder models are analysed. We constructed and trained 21 self-attention based encoder models and five convolution neural networks to identify gravitational lenses from the Bologna Lens Challenge. Each model was trained separately using 18,000 simulated images, cross-validated using 2,000 images, and then applied to a test set with 100,000 images. We used four different metrics for evaluation: classification accuracy, area under the receiver operating characteristic curve (AUROC), the TPR${}_0$ score and the TPR${}_{10}$ score. The performances of self-attention-based encoder models and CNNs participating in the challenge are compared. They were able to surpass the CNN models that participated in the Bologna Lens Challenge by a high margin for the TPR${}_0$ and TPR_$10$. Self-Attention based models have clear advantages compared to simpler CNNs. They have highly competing performance in comparison to the currently used residual neural networks. Compared to CNNs, self-attention based models can identify highly confident lensing candidates and will be able to filter out potential candidates from real data. Moreover, introducing the encoder layers can also tackle the over-fitting problem present in the CNNs by acting as effective filters.

18 Pages, 4 tables and 19 Figures

References in corpus (12)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• H0LiCOW V. New COSMOGRAIL time delays of HE0435-1223: $H_0$ to 3.8% precision from strong lensing in a flat $\Lambda$CDM model
• The Sloan Lens ACS Survey. V. The Full ACS Strong-Lens Sample
• The fourth data release of the Kilo-Degree Survey: ugri imaging and nine-band optical-IR photometry over 1000 square degrees
• The Dark Energy Survey Data Release 2
• The CFHTLS Strong Lensing Legacy Survey: I. Survey overview and T0002 release sample
• Deep-HiTS: Rotation Invariant Convolutional Neural Network for Transient Detection
• LinKS: Discovering galaxy-scale strong lenses in the Kilo-Degree Survey using Convolutional Neural Networks
• Using Convolutional Neural Networks to identify Gravitational Lenses in Astronomical images
• The use of convolutional neural networks for modelling large optically-selected strong galaxy-lens samples
• Deep Learning for Strong Lensing Search: Tests of the Convolutional Neural Networks and New Candidates from KiDS DR3
• A Comparative Study of Convolutional Neural Networks for the Detection of Strong Gravitational Lensing

Cited by in corpus (1)

• Transformers as Strong Lens Detectors- From Simulation to Surveys