LSTM Neural Networks: Input to State Stability and Probabilistic Safety Verification

Abstract

The goal of this paper is to analyze Long Short Term Memory (LSTM) neural networks from a dynamical system perspective. The classical recursive equations describing the evolution of LSTM can be recast in state space form, resulting in a time-invariant nonlinear dynamical system. A sufficient condition guaranteeing the Input-to-State (ISS) stability property of this class of systems is provided. The ISS property entails the boundedness of the output reachable set of the LSTM. In light of this result, a novel approach for the safety verification of the network, based on the Scenario Approach, is devised. The proposed method is eventually tested on a pH neutralization process.

arXiv commentAccepted for Learning for dynamics & control (L4DC) 2020

References in corpus (4)

• LSTM: A Search Space Odyssey
• On the difficulty of training Recurrent Neural Networks
• Stable Recurrent Models
• Scenario optimization for optimal training of Echo State Networks

Cited by in corpus (6)

• On Recurrent Neural Networks for learning-based control: recent results and ideas for future developments
• On the stability properties of Gated Recurrent Units neural networks
• Stability of discrete-time feed-forward neural networks in NARX configuration
• Recurrent Neural Network-based Internal Model Control design for stable nonlinear systems
• Learning Event-triggered Control from Data through Joint Optimization
• Towards lifelong learning of Recurrent Neural Networks for control design