Non-Euclidean Monotone Operator Theory with Applications to Recurrent Neural Networks

Abstract

We provide a novel transcription of monotone operator theory to the non-Euclidean finite-dimensional spaces ${\ell }_1$ and ${\ell }_{\infty }$. We first establish properties of mappings which are monotone with respect to the non-Euclidean norms ${\ell }_1$ or ${\ell }_{\infty }$. In analogy with their Euclidean counterparts, mappings which are monotone with respect to a non-Euclidean norm are amenable to numerous algorithms for computing their zeros. We demonstrate that several classic iterative methods for computing zeros of monotone operators are directly applicable in the non-Euclidean framework. We present a case-study in the equilibrium computation of recurrent neural networks and demonstrate that casting the computation as a suitable operator splitting problem improves convergence rates.

References in corpus (4)

• Distributed GNE seeking under partial-decision information over networks via a doubly-augmented operator splitting approach
• Online Primal-Dual Methods with Measurement Feedback for Time-Varying Convex Optimization
• Non-Euclidean Contraction Theory for Robust Nonlinear Stability
• On the rate of convergence of Krasnoselski-Mann iterations and their connection with sums of Bernoullis