Non-Euclidean Monotone Operator Theory with Applications to Recurrent Neural Networks

Abstract

We provide a novel transcription of monotone operator theory to the non-Euclidean finite-dimensional spaces ${\ell }_1$ and ${\ell }_{\infty }$. We first establish properties of mappings which are monotone with respect to the non-Euclidean norms ${\ell }_1$ or ${\ell }_{\infty }$. In analogy with their Euclidean counterparts, mappings which are monotone with respect to a non-Euclidean norm are amenable to numerous algorithms for computing their zeros. We demonstrate that several classic iterative methods for computing zeros of monotone operators are directly applicable in the non-Euclidean framework. We present a case-study in the equilibrium computation of recurrent neural networks and demonstrate that casting the computation as a suitable operator splitting problem improves convergence rates.

References in corpus (1)

• Non-Euclidean Contraction Theory for Robust Nonlinear Stability