Path classification by stochastic linear recurrent neural networks

Abstract

We investigate the functioning of a classifying biological neural network from the perspective of statistical learning theory, modelled, in a simplified setting, as a continuous-time stochastic recurrent neural network (RNN) with identity activation function. In the purely stochastic (robust) regime, we give a generalisation error bound that holds with high probability, thus showing that the empirical risk minimiser is the best-in-class hypothesis. We show that RNNs retain a partial signature of the paths they are fed as the unique information exploited for training and classification tasks. We argue that these RNNs are easy to train and robust and back these observations with numerical experiments on both synthetic and real data. We also exhibit a trade-off phenomenon between accuracy and robustness.

References in corpus (5)

• Poisoning Attacks against Support Vector Machines
• full-FORCE: A Target-Based Method for Training Recurrent Networks
• Understanding and Mitigating the Tradeoff Between Robustness and Accuracy
• Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series
• Probabilistic supervised learning