Synergistic Learning with Multi-Task DeepONet for Efficient PDE Problem Solving

Abstract

Multi-task learning (MTL) is an inductive transfer mechanism designed to leverage useful information from multiple tasks to improve generalization performance compared to single-task learning. It has been extensively explored in traditional machine learning to address issues such as data sparsity and overfitting in neural networks. In this work, we apply MTL to problems in science and engineering governed by partial differential equations (PDEs). However, implementing MTL in this context is complex, as it requires task-specific modifications to accommodate various scenarios representing different physical processes. To this end, we present a multi-task deep operator network (MT-DeepONet) to learn solutions across various functional forms of source terms in a PDE and multiple geometries in a single concurrent training session. We introduce modifications in the branch network of the vanilla DeepONet to account for various functional forms of a parameterized coefficient in a PDE. Additionally, we handle parameterized geometries by introducing a binary mask in the branch network and incorporating it into the loss term to improve convergence and generalization to new geometry tasks. Our approach is demonstrated on three benchmark problems: (1) learning different functional forms of the source term in the Fisher equation; (2) learning multiple geometries in a 2D Darcy Flow problem and showcasing better transfer learning capabilities to new geometries; and (3) learning 3D parameterized geometries for a heat transfer problem and demonstrate the ability to predict on new but similar geometries. Our MT-DeepONet framework offers a novel approach to solving PDE problems in engineering and science under a unified umbrella based on synergistic learning that reduces the overall training cost for neural operators.

References in corpus (10)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• A physics-informed variational DeepONet for predicting the crack path in brittle materials
• Deep transfer operator learning for partial differential equations under conditional shift
• Geom-DeepONet: A Point-cloud-based Deep Operator Network for Field Predictions on 3D Parameterized Geometries
• In-Context Operator Learning with Data Prompts for Differential Equation Problems
• On the influence of over-parameterization in manifold based surrogates and deep neural operators
• Deep Multi-Task Learning with Shared Memory
• DeepONet prediction of linear instability waves in high-speed boundary layers
• Blending Neural Operators and Relaxation Methods in PDE Numerical Solvers
• Poseidon: Efficient Foundation Models for PDEs

Cited by in corpus (1)

• Learning Nonlinear Responses in PET Bottle Buckling with a Hybrid DeepONet-Transolver Framework