SKATR: A Self-Supervised Summary Transformer for SKA
arXiv:2410.18899 · doi:10.21468/SciPostPhys.18.5.155
Abstract
The Square Kilometer Array will initiate a new era of radio astronomy by allowing 3D imaging of the Universe during Cosmic Dawn and Reionization. Modern machine learning is crucial to analyse the highly structured and complex signal. However, accurate training data is expensive to simulate, and supervised learning may not generalize. We introduce a self-supervised vision transformer, SKATR, whose learned encoding can be cheaply adapted for downstream tasks on 21cm maps. Focusing on regression and generative inference of astrophysical and cosmological parameters, we demonstrate that SKATR representations are maximally informative and that SKATR generalises out-of-domain to differently-simulated, noised, and higher-resolution datasets.
v2: Match published version
References in corpus (40)
- 21cmFAST: A Fast, Semi-Numerical Simulation of the High-Redshift 21-cm Signal
- Efficient Simulations of Early Structure Formation and Reionization
- What Next-Generation 21 cm Power Spectrum Measurements Can Teach Us About the Epoch of Reionization
- Four Fundamental Foreground Power Spectrum Shapes for 21 cm Cosmology Observations
- The Epoch of Reionization Window: II. Statistical Methods for Foreground Wedge Reduction
- The Baryon Acoustic Oscillation Broadband and Broad-beam Array: Design Overview and Sensitivity Forecasts
- 21cmFAST v3: A Python-integrated C code forgenerating 3D realizations of the cosmic 21cm signal
- Reionization and galaxy inference from the high-redshift Lyα forest
- Astraeus I: The interplay between galaxy formation and reionization
- Quantifying the non-Gaussianity in the EoR 21-cm signal through bispectrum
- New Constraints on Warm Dark Matter from the Lyman- Forest Power Spectrum
- Unveiling Dark Matter free-streaming at the smallest scales with high redshift Lyman-alpha forest
- Unveiling the nature of dark matter with high redshift 21 cm line experiments
- AstroCLIP: A Cross-Modal Foundation Model for Galaxies
- Photometry of high-redshift blended galaxies using deep learning
- Simulating the 21-cm signal from reionisation including non-linear ionisations and inhomogeneous recombinations
- Constraining the EoR model parameters with the 21cm bispectrum
- Fuzzy Dark Matter and the 21cm Power Spectrum
- Symmetries, Safety, and Self-Supervision
- Cosmology with one galaxy?
- Modern Machine Learning for LHC Physicists
- Machine Learning for Observational Cosmology
- OmniJet-: The first cross-task foundation model for particle physics
- Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models
- CaloDREAM -- Detector Response Emulation via Attentive flow Matching
- Inferring Astrophysics and Dark Matter Properties from 21cm Tomography using Deep Learning
- SKA Science Data Challenge 2: analysis and results
- The bispectrum and 21cm foregrounds during the Epoch of Reionization
- General Modified Gravity With 21cm Intensity Mapping: Simulations and Forecast
- Data Compression and Inference in Cosmology with Self-Supervised Machine Learning
- Constraining coupled quintessence with the 21cm signal
- Machine Learning and Cosmology
- Cornering Extended Starobinsky Inflation with CMB and SKA
- Optimal, fast, and robust inference of reionization-era cosmology with the 21cmPIE-INN
- Synergistic tests of inflation
- Lux ex tenebris: The imprint of annihilating dark matter on the intergalactic medium during Cosmic Dawn
- Is Tokenization Needed for Masked Particle Modelling?
- Hybrid Summary Statistics
- Deep Learning 21cm Lightcones in 3D
- Bridging the Gap: Examining Vision Foundation Models for Optical and Radio Astronomy Applications
Cited by in corpus (6)
- The 21cm-galaxy cross-correlation: Realistic forecast for 21cm signal detection and reionisation constraints
- From ANN to BNN: Inferring Reionization Parameters using Uncertainty-aware Emulators of 21-cm Summaries
- CosmoUiT: A Vision Transformer-UNet Hybrid for Fast and Accurate Emulation of 21-cm Maps from the Epoch of Reionization
- Large Language Models -- the Future of Fundamental Physics?
- Direct reconstruction of the Reionization history from 21cm 2D Power Spectra
- Beyond the Power Spectrum: A New Framework for Non-Stationary Fields with Applications to Light-Cone Effects in Line Intensity Mapping