Depth separation for reduced deep networks in nonlinear model reduction: Distilling shock waves in nonlinear hyperbolic problems

Abstract

Classical reduced models are low-rank approximations using a fixed basis designed to achieve dimensionality reduction of large-scale systems. In this work, we introduce reduced deep networks, a generalization of classical reduced models formulated as deep neural networks. We prove depth separation results showing that reduced deep networks approximate solutions of parametrized hyperbolic partial differential equations with approximation error $\epsilon$ with $\mathcal{O}(|\log (\epsilon )|)$ degrees of freedom, even in the nonlinear setting where solutions exhibit shock waves. We also show that classical reduced models achieve exponentially worse approximation rates by establishing lower bounds on the relevant Kolmogorov $N$-widths.

References in corpus (12)

• Distilling the Knowledge in a Neural Network
• Compressing Neural Networks with the Hashing Trick
• A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
• Adaptive $h$-refinement for reduced-order models
• The method of freezing as a new tool for nonlinear reduced basis approximation of parameterized evolution equations
• Nonlinear Approximation and (Deep) ReLU Networks
• Generating Random Earthquake Events for PTHA
• Approximated Lax Pairs for the Reduced Order Integration of Nonlinear Evolution Equations
• Projection-Based Model Reduction with Dynamically Transformed Modes
• Lagrangian basis method for dimensionality reduction of convection dominated nonlinear flows
• Manifold Approximations via Transported Subspaces: Model reduction for transport-dominated problems
• Dimensional splitting of hyperbolic partial differential equations using the Radon transform

Cited by in corpus (6)

• A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder
• Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problems
• Reduced order models for Lagrangian hydrodynamics
• Machine learning moment closure models for the radiative transfer equation I: directly learning a gradient based closure
• Manifold Approximations via Transported Subspaces: Model reduction for transport-dominated problems
• Depth separation beyond radial functions