SKYNET: an efficient and robust neural network training tool for machine learning in astronomy

Abstract

We present the first public release of our generic neural network training algorithm, called SkyNet. This efficient and robust machine learning tool is able to train large and deep feed-forward neural networks, including autoencoders, for use in a wide range of supervised and unsupervised learning applications, such as regression, classification, density estimation, clustering and dimensionality reduction. SkyNet uses a `pre-training' method to obtain a set of network parameters that has empirically been shown to be close to a good solution, followed by further optimisation using a regularised variant of Newton's method, where the level of regularisation is determined and adjusted automatically; the latter uses second-order derivative information to improve convergence, but without the need to evaluate or store the full Hessian matrix, by using a fast approximate method to calculate Hessian-vector products. This combination of methods allows for the training of complicated networks that are difficult to optimise using standard backpropagation techniques. SkyNet employs convergence criteria that naturally prevent overfitting, and also includes a fast algorithm for estimating the accuracy of network outputs. The utility and flexibility of SkyNet are demonstrated by application to a number of toy problems, and to astronomical problems focusing on the recovery of structure from blurred and noisy images, the identification of gamma-ray bursters, and the compression and denoising of galaxy images. The SkyNet software, which is implemented in standard ANSI C and fully parallelised using MPI, is available at http://www.mrao.cam.ac.uk/software/skynet/.

19 pages, 21 figures, 7 tables; this version is re-submission to MNRAS in response to referee comments; software available at http://www.mrao.cam.ac.uk/software/skynet/

References in corpus (8)

• Importance Nested Sampling and the MultiNest Algorithm
• Probing the Cosmic Gamma-Ray Burst Rate with Trigger Simulations of the Swift Burst Alert Telescope
• Fast cosmological parameter estimation using neural networks
• Using neural networks to estimate redshift distributions. An application to CFHTLenS
• {\sc CosmoNet}: fast cosmological parameter estimation in non-flat models using neural networks
• A Coverage Study of the CMSSM Based on ATLAS Sensitivity Using Fast Neural Networks Techniques
• Cluster detection in weak lensing surveys
• Pico: Parameters for the Impatient Cosmologist

Cited by in corpus (60)

• Robust parameter estimation for compact binaries with ground-based gravitational-wave observations using the LALInference software library
• redMaGiC: Selecting Luminous Red Galaxies from the DES Science Verification Data
• The First Two Years of Electromagnetic Follow-Up with Advanced LIGO and Virgo
• Bayesian analysis of cosmic-ray propagation: evidence against homogeneous diffusion
• GREAT3 results I: systematic errors in shear estimation and the impact of real galaxy morphology
• Cosmology constraints from shear peak statistics in Dark Energy Survey Science Verification data
• Parameter estimation for binary neutron-star coalescences with realistic noise during the Advanced LIGO era
• Bulk Lorentz factors of Gamma-Ray Bursts
• Weak-lensing mass calibration of redMaPPer galaxy clusters in Dark Energy Survey Science Verification data
• Neural Networks for Modeling and Control of Particle Accelerators
• Cosmic Shear Measurements with DES Science Verification Data
• Galaxy clustering in the DESI Legacy Survey and its imprint on the CMB
• Evaluation of probabilistic photometric redshift estimation approaches for The Rubin Observatory Legacy Survey of Space and Time (LSST)
• Using neural networks to estimate redshift distributions. An application to CFHTLenS
• Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy
• Cosmology from large scale galaxy clustering and galaxy-galaxy lensing with Dark Energy Survey Science Verification data
• Photometric Redshift Estimation with a Convolutional Neural Network: NetZ
• Galaxy-Galaxy Lensing in the DES Science Verification Data
• Joint Measurement of Lensing-Galaxy Correlations Using SPT and DES SV Data
• Machine Learning Classification of SDSS Transient Survey Images
• Radio Galaxy Zoo: Unsupervised Clustering of Convolutionally Auto-encoded Radio-astronomical Images
• Machine Learning and Cosmological Simulations I: Semi-Analytical Models
• 2dFLenS and KiDS: Determining source redshift distributions with cross-correlations
• Convolutional Neural Networks on the HEALPix sphere: a pixel-based algorithm and its application to CMB data analysis
• Conditional Density Estimation Tools in Python and R with Applications to Photometric Redshifts and Likelihood-Free Cosmological Inference
• A review of unsupervised learning in astronomy
• On parametrised cold dense matter equation of state inference
• Stellar classification from single-band imaging using machine learning
• Cross-Correlation Redshift Calibration Without Spectroscopic Calibration Samples in DES Science Verification Data
• Weak lensing magnification in the Dark Energy Survey Science Verification Data
• Weak-lensing shear measurement with machine learning: teaching artificial neural networks about feature noise
• Classifying Kepler light curves for 12,000 A and F stars using supervised feature-based machine learning
• Weak Lensing Analysis of SPT selected Galaxy Clusters using Dark Energy Survey Science Verification Data
• Bars formed in galaxy merging and their classification with deep learning
• Fast Bayesian inference for slow-roll inflation
• Galaxy bias from the Dark Energy Survey Science Verification data: combining galaxy density maps and weak lensing maps
• DES Science Portal: Computing Photometric Redshifts
• Statistical analysis of probability density functions for photometric redshifts through the KiDS-ESO-DR3 galaxies
• Optical Transient Object Classification in Wide Field Small Aperture Telescopes with Neural Networks
• Transit Clairvoyance: Enhancing TESS follow-up using artificial neural networks
• Galaxy bias from galaxy-galaxy lensing in the DES Science Verification Data
• Bayesian sparse reconstruction: a brute-force approach to astronomical imaging and machine learning
• The 2-degree Field Lensing Survey: photometric redshifts from a large new training sample to r<19.5
• Machine learning synthetic spectra for probabilistic redshift estimation: SYTH-Z
• Survey2Survey: A deep learning generative model approach for cross-survey image mapping
• Neural network-based emulation of interstellar medium models
• Galaxy Spectra neural Network (GaSNet). II. Using Deep Learning for Spectral Classification and Redshift Predictions
• Machine Learning Model of the Swift/BAT Trigger Algorithm for Long GRB Population Studies
• Deep Learning and genetic algorithms for cosmological Bayesian inference speed-up
• Learn-As-You-Go Acceleration of Cosmological Parameter Estimates
• How machine learning conquers the unitary limit
• Rejection criteria based on outliers in the KiDS photometric redshifts and PDF distributions derived by machine learning
• Artificial Neural Network for Constructing Type Ia Supernovae Spectrum Evolution Model
• A Neural Network Perturbation Theory Based on the Born Series
• An Algorithm for the Visualization of Relevant Patterns in Astronomical Light Curves
• Towards a constructive multilayer perceptron for regression task using non-parametric clustering. A case study of Photo-Z redshift reconstruction
• Bayesian analysis of multiple direct detection experiments
• Investigation on deep learning-based galaxy image translation models
• Optical Follow-up of Planck Cluster Candidates with Small Instruments
• Artificial Neural Network Spectral Light Curve Template for Type Ia Supernovae and its Cosmological Constraints