Towards out of distribution generalization for problems in mechanics
arXiv:2206.14917 · doi:10.1016/j.cma.2022.115569
Abstract
There has been a massive increase in research interest towards applying data driven methods to problems in mechanics. While traditional machine learning (ML) methods have enabled many breakthroughs, they rely on the assumption that the training (observed) data and testing (unseen) data are independent and identically distributed (i.i.d). Thus, traditional ML approaches often break down when applied to real world mechanics problems with unknown test environments and data distribution shifts. In contrast, out-of-distribution (OOD) generalization assumes that the test data may shift (i.e., violate the i.i.d. assumption). To date, multiple methods have been proposed to improve the OOD generalization of ML methods. However, because of the lack of benchmark datasets for OOD regression problems, the efficiency of these OOD methods on regression problems, which dominate the mechanics field, remains unknown. To address this, we investigate the performance of OOD generalization methods for regression problems in mechanics. Specifically, we identify three OOD problems: covariate shift, mechanism shift, and sampling bias. For each problem, we create two benchmark examples that extend the Mechanical MNIST dataset collection, and we investigate the performance of popular OOD generalization methods on these mechanics-specific regression problems. Our numerical experiments show that in most cases, while the OOD generalization algorithms perform better compared to traditional ML methods on these OOD problems, there is a compelling need to develop more robust OOD generalization methods that are effective across multiple OOD scenarios. Overall, we expect that this study, as well as the associated open access benchmark datasets, will enable further development of OOD generalization methods for mechanics specific regression problems.
References in corpus (38)
- Explaining and Harnessing Adversarial Examples
- Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
- Deep Visual Domain Adaptation: A Survey
- An Introduction to Deep Reinforcement Learning
- Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition
- Data-driven discovery of coordinates and governing equations
- DOLFIN: Automated Finite Element Computing
- Integrating Machine Learning and Multiscale Modeling: Perspectives, Challenges, and Opportunities in the Biological, Biomedical, and Behavioral Sciences
- Opportunities and Challenges for Machine Learning in Materials Science
- Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
- Learning Confidence for Out-of-Distribution Detection in Neural Networks
- Generalized Out-of-Distribution Detection: A Survey
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
- Towards Out-Of-Distribution Generalization: A Survey
- Sparse Identification of Nonlinear Dynamics for Rapid Model Recovery
- Meta-modeling game for deriving theoretical-consistent, micro-structural-based traction-separation laws via deep reinforcement learning
- An Investigation of Why Overparameterization Exacerbates Spurious Correlations
- Machine learning materials physics: Multi-resolution neural networks learn the free energy and nonlinear elastic response of evolving microstructures
- Domain Generalization using Causal Matching
- Forward and Inverse Design of Kirigami via Supervised Autoencoder
- Efficient Domain Generalization via Common-Specific Low-Rank Decomposition
- A cooperative game for automated learning of elasto-plasticity knowledge graphs and models with AI-guided experimentation
- Understanding the Failure Modes of Out-of-Distribution Generalization
- An Ensemble of Simple Convolutional Neural Network Models for MNIST Digit Recognition
- Causality in cognitive neuroscience: concepts, challenges, and distributional robustness
- Finding Symmetry Breaking Order Parameters with Euclidean Neural Networks
- Towards a Theoretical Framework of Out-of-Distribution Generalization
- A fourth-order phase-field fracture model: Formulation and numerical solution using a continuous/discontinuous Galerkin method
- Invariant Rationalization
- Learning Models with Uniform Performance via Distributionally Robust Optimization
- Enhancing Mechanical Metamodels with a Generative Model-Based Augmented Training Dataset
- Estimate and Replace: A Novel Approach to Integrating Deep Neural Networks with Existing Applications
- Kernelized Heterogeneous Risk Minimization
- On the Impact of Spurious Correlation for Out-of-distribution Detection
- Understanding and Testing Generalization of Deep Networks on Out-of-Distribution Data
- Bayesian OOD detection with aleatoric uncertainty and outlier exposure
- Predicting Mechanically Driven Full-Field Quantities of Interest with Deep Learning-Based Metamodels