Proper image subtraction - optimal transient detection, photometry and hypothesis testing
arXiv:1601.02655 · doi:10.3847/0004-637X/830/1/27
Abstract
Transient detection and flux measurement via image subtraction stand at the base of time domain astronomy. Due to the varying seeing conditions, the image subtraction process is non-trivial, and existing solutions suffer from a variety of problems. Starting from basic statistical principles, we develop the optimal statistic for transient detection, flux measurement and any image-difference hypothesis testing. We derive a closed-form statistic that: (i) Is mathematically proven to be the optimal transient detection statistic in the limit of background-dominated noise; (ii) Is numerically stable; (iii) For accurately registered, adequately sampled images, does not leave subtraction or deconvolution artifacts; (iv) Allows automatic transient detection to the theoretical sensitivity limit by providing credible detection significance; (v) Has uncorrelated white noise; (vi) Is a sufficient statistic for any further statistical test on the difference image, and in particular, allows to distinguish particle hits and other image artifacts from real transients; (vii) Is symmetric to the exchange of the new and reference images; (viii) Is at least an order of magnitude faster to compute than some popular methods; and (ix) Is straightforward to implement. Furthermore, we present extensions of this method that make it resilient to registration errors, color-refraction errors, and any noise source that can be modelled. In addition, we show that the optimal way to prepare a reference image is the proper image coaddition presented in Zackay \& Ofek (2015b). We demonstrate this method on simulated data and real observations from the Palomar Transient Factory data release 2. We provide an implementation of this algorithm in MATLAB and Python.
Comments are welcome. Authors name corrected
References in corpus (8)
- A Wolf-Rayet-like progenitor of supernova SN 2013cu from spectral observations of a wind
- A New Algorithm For Difference Image Analysis
- Machine learning for transient discovery in Pan-STARRS1 difference imaging
- iPTF14yb: The First Discovery of a GRB Afterglow Independent of a High-Energy Trigger
- IPAC Image Processing and Data Archiving for the Palomar Transient Factory
- Supernovae in Low-Redshift Galaxy Clusters: Observations by the Wise Observatory Optical Transient Search (WOOTS)
- Astronomical Image Subtraction by Cross-Convolution
- The Brighter-Fatter and other Sensor Effects in CCD Simulations for Precision Astronomy
Cited by in corpus (163)
- The Zwicky Transient Facility: System Overview, Performance, and First Results
- The Zwicky Transient Facility: Data Processing, Products, and Archive
- The Zwicky Transient Facility: Science Objectives
- Design and operation of the ATLAS Transient Science Server
- The Zwicky Transient Facility: Observing System
- Real-bogus classification for the Zwicky Transient Facility using deep learning
- A large fraction of hydrogen-rich supernova progenitors experience elevated mass loss shortly prior to explosion
- Discovery and confirmation of the shortest gamma ray burst from a collapsar
- Machine Learning for the Zwicky Transient Facility
- The Koala: A Fast Blue Optical Transient with Luminous Radio Emission from a Starburst Dwarf Galaxy at
- Bright, months-long stellar outbursts announce the explosion of interaction-powered supernovae
- The first tidal disruption flare in ZTF: from photometric selection to multi-wavelength characterization
- Analysis of broad-lined Type Ic supernovae from the (intermediate) Palomar Transient Factory
- The Zwicky Transient Facility Census of the Local Universe I: Systematic search for Calcium rich gap transients reveal three related spectroscopic sub-classes
- Spitzer Mid-Infrared Detections of Neutron Star Merger GW170817 Suggests Synthesis of the Heaviest Elements
- A WC/WO star exploding within an expanding carbon-oxygen-neon nebula
- ZTF Early Observations of Type Ia Supernovae I: Properties of the 2018 Sample
- Kilonova Luminosity Function Constraints based on Zwicky Transient Facility Searches for 13 Neutron Star Mergers
- A Search for Extragalactic Fast Blue Optical Transients in ZTF and the Rate of AT2018cow-like Transients
- Short-Lived Circumstellar Interaction in the Low-Luminosity Type IIP SN 2016bkv
- Follow-up of the Neutron Star Bearing Gravitational Wave Candidate Events S190425z and S190426c with MMT and SOAR
- The IPAC Image Subtraction and Discovery Pipeline for the intermediate Palomar Transient Factory
- From Discovery to the First Month of the Type II Supernova 2023ixf: High and Variable Mass Loss in the Final Year before Explosion
- Fast-transient Searches in Real Time with ZTFReST: Identification of Three Optically-discovered Gamma-ray Burst Afterglows and New Constraints on the Kilonova Rate
- The Large Synoptic Survey Telescope as a Near-Earth Object Discovery Machine
- Palomar Gattini-IR: Survey overview, data processing system, on-sky performance and first results
- ZTF Early Observations of Type Ia Supernovae II: First Light, the Initial Rise, and Time to Reach Maximum Brightness
- Detection and Classification of Astronomical Targets with Deep Neural Networks in Wide Field Small Aperture Telescopes
- A Shock Flash Breaking Out of a Dusty Red Supergiant
- Luminous Type II Short-Plateau Supernovae 2006Y, 2006ai, and 2016egz: A Transitional Class from Stripped Massive Red Supergiants
- SN 2019ehk: A Double-Peaked Ca-rich Transient with Luminous X-ray Emission and Shock-Ionized Spectral Features
- Searches After Gravitational-waves Using ARizona Observatories (SAGUARO): System Overview and First Results from Advanced LIGO/Virgo's Third Observing Run
- SN2019dge: a Helium-rich Ultra-Stripped Envelope Supernova
- The Automated Photometry Of Transients (AutoPhOT) pipeline
- Image Subtraction in Fourier Space
- Optimal and Efficient Streak Detection in Astronomical Images
- SN2020bvc: a Broad-lined Type Ic Supernova with a Double-peaked Optical Light Curve and a Luminous X-ray and Radio Counterpart
- Cosmological Results from the RAISIN Survey: Using Type Ia Supernovae in the Near Infrared as a Novel Path to Measure the Dark Energy Equation of State
- The Hydrogen-Poor Superluminous Supernovae from the Zwicky Transient Facility Phase-I Survey: I. Light Curves and Measurements
- Circumstellar Interaction Powers the Light Curves of Luminous Rapidly Evolving Optical Transients
- DeepStreaks: identifying fast-moving objects in the Zwicky Transient Facility data with deep learning
- How to coadd images? I. Optimal source detection and photometry using ensembles of images
- A transient search using combined human and machine classifications
- The Spectacular Ultraviolet Flash From the Peculiar Type Ia Supernova 2019yvq
- A population of heavily reddened, optically missed novae from Palomar Gattini-IR: Constraints on the Galactic nova rate
- Constraining the Kilonova Rate with Zwicky Transient Facility Searches Independent of Gravitational Wave and Short Gamma-ray Burst Triggers
- Spitzer Space Telescope Infrared Observations of the Binary Neutron Star Merger GW170817
- ZTF20aajnksq (AT2020blt): A Fast Optical Transient at With No Detected Gamma-Ray Burst Counterpart
- Effective Image Differencing with ConvNets for Real-time Transient Hunting
- A radio-detected Type Ia supernova with helium-rich circumstellar material
- Transient-optimised real-bogus classification with Bayesian Convolutional Neural Networks -- sifting the GOTO candidate stream
- The Gravity Collective: A Search for the Electromagnetic Counterpart to the Neutron Star-Black Hole Merger GW190814
- The Large Array Survey Telescope -- System Overview and Performances
- Cosmological Fast Optical Transients with the Zwicky Transient Facility: A Search for Dirty Fireballs
- 2900 square degree search for the optical counterpart of short gamma-ray burst GRB 180523B with the Zwicky Transient Facility
- A Systematic Study of Ia-CSM Supernovae from the ZTF Bright Transient Survey
- Multi-frequency observations of SGR J1935+2154
- Searches after Gravitational Waves Using ARizona Observatories (SAGUARO): Observations and Analysis from Advanced LIGO/Virgo's Third Observing Run
- How to coadd images? II. A coaddition image that is optimal for any purpose in the background dominated noise limit
- Orbital Foregrounds for Ultra-Short Duration Transients
- A non-equipartition shockwave traveling in a dense circumstellar environment around SN2020oi
- The BlackGEM telescope array I: Overview
- Four (Super)luminous Supernovae from the First Months of the ZTF Survey
- Challenging the AGN scenario for JWST/NIRSpec LRD and non-LRD broad H emitters in light of non-detection of NIRCam photometric variability and X-ray
- Discovery of an intermediate-luminosity red transient in M51 and its likely dust-obscured, infrared-variable progenitor
- The luminous and rapidly evolving SN 2018bcc: Clues toward the origin of Type Ibn SNe from the Zwicky Transient Facility
- A fast X-ray transient from a weak relativistic jet associated with a type Ic-BL supernova
- Less than 1% of Core-Collapse Supernovae in the local universe occur in elliptical galaxies
- GW190814 follow-up with the optical telescope MeerLICHT
- Collapsars as Sites of r-process Nucleosynthesis: Systematic Near-Infrared Follow-up of Type Ic-BL Supernovae
- Minutes-duration Optical Flares with Supernova Luminosities
- The Zwicky Transient Facility phase I sample of hydrogen-rich superluminous supernovae without strong narrow emission lines
- A high-rate foreground of sub-second flares from geosynchronous satellites
- Optimizing Serendipitous Detections of Kilonovae: Cadence and Filter Selection
- Optical Transient Object Classification in Wide Field Small Aperture Telescopes with Neural Networks
- Zwicky Transient Facility constraints on the optical emission from the nearby repeating FRB 180916.J0158+65
- R-band light-curve properties of Type Ia supernovae from the (intermediate) Palomar Transient Factory
- Machine Learning on Difference Image Analysis: A comparison of methods for transient detection
- Gaia transients in galactic nuclei
- A code for robust astrometric solution of astronomical images
- A luminous dust-obscured Tidal Disruption Event candidate in a star forming galaxy at 42 Mpc
- Multi-resolution filtering: an empirical method for isolating faint, extended emission in Dragonfly data and other low resolution images
- SN 2020bqj: a Type Ibn supernova with a long lasting peak plateau
- SN2020qlb: A hydrogen-poor superluminous supernova with well-characterized light curve undulations
- MeerCRAB: MeerLICHT Classification of Real and Bogus Transients using Deep Learning
- SN 2020sck: deflagration in a carbon-oxygen white dwarf
- Mary, a pipeline to aid discovery of optical transients
- Multiple Peaks and a Long Precursor in the Type IIn Supernova 2021qqp: An Energetic Explosion in a Complex Circumstellar Environment
- An Exceptional Dimming Event for a Massive, Cool Supergiant in M51
- Optimal matched filter in the low-number count Poisson noise regime and implications for X-ray source detection
- Limits on Simultaneous and Delayed Optical Emission from Well-localized Fast Radio Bursts
- The Exotic Type Ic Broad-Lined Supernova SN 2018gep: Blurring the Line Between Supernovae and Fast Optical Transients
- RINGO3 polarimetry of very young ZTF supernovae
- SN 2019zrk, a bright SN 2009ip analog with a precursor
- The Broad-lined Ic Supernova ZTF18aaqjovh (SN 2018bvw): An Optically-discovered Engine-driven Supernova Candidate with Luminous Radio Emission
- The Palomar Transient Factory Sky2Night programme
- Supernova Siblings and their Parent Galaxies in the Zwicky Transient Facility Bright Transient Surve
- Cataclysmic Variables in the Second Year of the Zwicky Transient Facility
- A classification algorithm for time-domain novelties in preparation for LSST alerts: Application to variable stars and transients detected with DECam in the Galactic Bulge
- Outbursting Young Stellar Object PGIR 20dci in the Perseus Arm
- Constraining the X-ray - Infrared spectral index of second-timescale flares from SGR1935+2154 with Palomar Gattini-IR
- Sample of hydrogen-rich superluminous supernovae from the Zwicky Transient Facility
- SN 2017cfd: A Normal Type Ia Supernova Discovered Very Young
- Sifting Through the Static: Moving Object Detection in Difference Images
- Early Ultra-Violet observations of type IIn supernovae constrain the asphericity of their circumstellar material
- Continuum reverberation mapping of the quasar PG 2130+099
- Automated Detection of Satellite Trails in Ground-Based Observations Using U-Net and Hough Transform
- Supernova 2020wnt: An Atypical Superluminous Supernova with a Hidden Central Engine
- Tails: Chasing Comets with the Zwicky Transient Facility and Deep Learning
- Inferring CSM Properties of Type II SNe Using a Magnitude-Limited ZTF Sample
- Deep Drilling in the Time Domain with DECam: Survey Characterization
- AT2018lqh and the nature of the emerging population of day-scale duration optical transients
- GROWTH on S190426c II: GROWTH-India Telescope search for an optical counterpart with a custom image reduction and candidate vetting pipeline
- SN 2020udy: a SN Iax with strict limits on interaction consistent with a helium-star companion
- Synthetic tracking using ZTF Long Dwell Datasets
- Rapidly evolving Galactic plane outbursts in NEOWISE: Revisiting the Galactic nova rate with the first all-sky search in the mid-infrared
- Retrospective Search for Strongly Lensed Supernovae in the DESI Legacy Imaging Surveys
- In search of short gamma-ray burst optical counterpart with the Zwicky Transient Facility
- WTP10aaauow: Discovery of a new FU Ori outburst towards the RCW49 star-forming region in NEOWISE data
- A possible advantage of telescopes with a non-circular pupil
- SAGUARO: Time-domain Infrastructure for the Fourth Gravitational-wave Observing Run and Beyond
- The Evryscope Fast Transient Engine: Real-Time Detection for Rapidly Evolving Transients
- Searching for late-time interaction signatures in Type Ia supernovae from the Zwicky Transient Facility
- The Hubble Catalog of Variables (HCV)
- Gravitational microlensing events from the first year of the northern Galactic plane survey by the Zwicky Transient Facility
- Near-Infrared and Optical Observations of Type Ic SN 2021krf: Luminous Late-time Emission and Dust Formation
- SN 2017hpa: A Nearby Carbon-Rich Type Ia Supernova with a Large Velocity Gradient
- A novel method for transient detection in high-cadence optical surveys: Its application for a systematic search for novae in M31
- SN 2019nyk: A rapidly declining Type II supernova with early interaction signatures
- SN 2022joj: A Peculiar Type Ia Supernova Possibly Driven by an Asymmetric Helium-shell Double Detonation
- Contemporaneous optical-radio observations of a fast radio burst in a close galaxy pair
- A diverse, overlooked population of Type Ia supernovae exhibiting mid-infrared signatures of delayed circumstellar interaction
- Detecting optical transients using artificial neural networks and reference images from different surveys
- Automated Transient Detection with Shapelet Analysis in Image-subtracted Data
- TOROS Optical follow-up of the Advanced LIGO-VIRGO O2 second observational campaign
- PyTorchDIA: A flexible, GPU-accelerated numerical approach to Difference Image Analysis
- The luminous red nova AT 2018bwo in NGC 45 and its binary yellow supergiant progenitor
- Disappearance of a massive star in the Andromeda Galaxy due to formation of a black hole
- Life Beyond PTF
- A spectroscopic, photometric, polarimetric and radio study of the eclipsing polar UZ Fornacis: the first simultaneous SALT and MeerKAT observations
- Variability and transient search in the SUDARE-VOICE field: a new method to extract the light curves
- WINTER on S250206dm: A near-infrared search for an electromagnetic counterpart to a gravitational-wave event
- Searching for Sub-Second Stellar Variability with Wide-Field Star Trails and Deep Learning
- The Thresher: Lucky Imaging without the Waste
- Differencing and Coadding JWST Images with Matched Point Spread Function
- Image plane detection of FRB121102 with the MeerKAT radio telescope
- Textual interpretation of transient image classifications from large language models
- The Gravitational-wave Optical Transient Observer (GOTO): prototype performance and prospects for transient science
- State Space Modelling for detecting and characterising Gravitational Waves afterglows
- A six year image-subtraction light curve of SN 2010jl
- PGIR 20eid (SN2020qmp): A Type IIP Supernova at 15.6 Mpc discovered by the Palomar Gattini-IR survey
- Optimizing Convolution Direction and Template Selection for Difference Image Analysis
- Search for the Optical Counterpart of Einstein Probe Discovered Fast X-ray Transients from Lulin Observatory
- Ca2+ transient detection and segmentation with the Astronomically motivated algorithm for Background Estimation And Transient Segmentation (Astro-BEATS)
- India's first robotic eye for time domain astrophysics: the GROWTH-India telescope
- Gaia-GIC-1: An Evolving Catastrophic Planetesimal Collision Candidate
- Unveiling SN 2022eyw: A Bright Member of the Type Iax Supernova Subclass
- The Large Array Survey Telescope-Pipeline. II. Image Subtraction and Transient Detection
- SN 2021tsz: A luminous, short photospheric phase Type II supernova in a low-metallicity host
- Faintest of them all : ZTF 21aaoryiz/SN 2021fcg -- Discovery of an extremely low luminosity type Iax supernova
- Constraining the Time of Gravitational Wave Emission from Core-Collapse Supernovae
- Precision speckle pattern reconstruction for high contrast imaging
- Variable Stars in M31 Stellar Clusters from the Panchromatic Hubble Andromeda Treasury