282 citations · 989 across the 11 of their papers we have counts for
12 papers
S-OPT: A Points Selection Algorithm for Hyper-Reduction in Reduced Order Models
Jessica T. Lauzon, Siu Wun Cheung, Yeonjong Shin +3
While projection-based reduced order models can reduce the dimension of full order solutions, the resulting reduced models may still contain terms that scale with the full order di…
GFINNs: GENERIC Formalism Informed Neural Networks for Deterministic and Stochastic Dynamical Systems
Zhen Zhang, Yeonjong Shin, George Em Karniadakis
We propose the GENERIC formalism informed neural networks (GFINNs) that obey the symmetric degeneracy conditions of the GENERIC formalism. GFINNs comprise two modules, each of whic…
Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions
Ameya D. Jagtap, Yeonjong Shin, Kenji Kawaguchi +1
We propose a new type of neural networks, Kronecker neural networks (KNNs), that form a general framework for neural networks with adaptive activation functions. KNNs employ the Kr…
A Caputo fractional derivative-based algorithm for optimization
Yeonjong Shin, Jérôme Darbon, George Em Karniadakis
We propose a novel Caputo fractional derivative-based optimization algorithm. Upon defining the Caputo fractional gradient with respect to the Cartesian coordinate, we present a ge…
Convergence rate of DeepONets for learning operators arising from advection-diffusion equations
Beichuan Deng, Yeonjong Shin, Lu Lu +2
We present convergence analysis of operator learning in [Chen and Chen 1995] and [Lu et al. 2020], where continuous operators are approximated by a sum of products of branch and tr…
Error estimates of residual minimization using neural networks for linear PDEs
Yeonjong Shin, Zhongqiang Zhang, George Em Karniadakis
We propose an abstract framework for analyzing the convergence of least-squares methods based on residual minimization when feasible solutions are neural networks. With the norm re…
Plateau Phenomenon in Gradient Descent Training of ReLU networks: Explanation, Quantification and Avoidance
Mark Ainsworth, Yeonjong Shin
The ability of neural networks to provide `best in class' approximation across a wide range of applications is well-documented. Nevertheless, the powerful expressivity of neural ne…
On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs
Yeonjong Shin, Jerome Darbon, George Em Karniadakis
Physics informed neural networks (PINNs) are deep learning based techniques for solving partial differential equations (PDEs) encounted in computational science and engineering. Gu…
Effects of Depth, Width, and Initialization: A Convergence Analysis of Layer-wise Training for Deep Linear Neural Networks
Yeonjong Shin
Deep neural networks have been used in various machine learning applications and achieved tremendous empirical successes. However, training deep neural networks is a challenging ta…
Trainability of ReLU networks and Data-dependent Initialization
Yeonjong Shin, George Em Karniadakis
In this paper, we study the trainability of rectified linear unit (ReLU) networks. A ReLU neuron is said to be dead if it only outputs a constant for any input. Two death states of…
Dying ReLU and Initialization: Theory and Numerical Examples
Lu Lu, Yeonjong Shin, Yanhui Su +1
The dying ReLU refers to the problem when ReLU neurons become inactive and only output 0 for any input. There are many empirical and heuristic explanations of why ReLU neurons die.…
Collapse of Deep and Narrow Neural Nets
Lu Lu, Yanhui Su, George Em Karniadakis
Recent theoretical work has demonstrated that deep neural networks have superior performance over shallow networks, but their training is more difficult, e.g., they suffer from the…