High-level python abstractions for optimal checkpointing in inversion problems

Abstract

Inversion and PDE-constrained optimization problems often rely on solving the adjoint problem to calculate the gradient of the objec- tive function. This requires storing large amounts of intermediate data, setting a limit to the largest problem that might be solved with a given amount of memory available. Checkpointing is an approach that can reduce the amount of memory required by redoing parts of the computation instead of storing intermediate results. The Revolve checkpointing algorithm o ers an optimal schedule that trades computational cost for smaller memory footprints. Integrat- ing Revolve into a modern python HPC code and combining it with code generation is not straightforward. We present an API that makes checkpointing accessible from a DSL-based code generation environment along with some initial performance gures with a focus on seismic applications.

Cited by in corpus (4)

• Devito (v3.1.0): an embedded domain-specific language for finite differences and geophysical exploration
• Training on the Edge: The why and the how
• Combining Checkpointing and Data Compression to Accelerate Adjoint-Based Optimization Problems
• Backpropagation for long sequences: beyond memory constraints with constant overheads