Nonparametric Online Learning Using Lipschitz Regularized Deep Neural Networks

Abstract

Deep neural networks are considered to be state of the art models in many offline machine learning tasks. However, their performance and generalization abilities in online learning tasks are much less understood. Therefore, we focus on online learning and tackle the challenging problem where the underlying process is stationary and ergodic and thus removing the i.i.d. assumption and allowing observations to depend on each other arbitrarily. We prove the generalization abilities of Lipschitz regularized deep neural networks and show that by using those networks, a convergence to the best possible prediction strategy is guaranteed.

References in corpus (11)

• Spectral Normalization for Generative Adversarial Networks
• Understanding deep learning requires rethinking generalization
• Stochastic Pooling for Regularization of Deep Convolutional Neural Networks
• On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
• Deep and Confident Prediction for Time Series at Uber
• A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem
• Improved robustness to adversarial examples using Lipschitz regularization of the loss
• A Surprising Linear Relationship Predicts Test Performance in Deep Networks
• Lipschitz Networks and Distributional Robustness
• Growth-Optimal Portfolio Selection under CVaR Constraints
• Generalisation in fully-connected neural networks for time series forecasting