Shallow Representation is Deep: Learning Uncertainty-aware and Worst-case Random Feature Dynamics

Abstract

Random features is a powerful universal function approximator that inherits the theoretical rigor of kernel methods and can scale up to modern learning tasks. This paper views uncertain system models as unknown or uncertain smooth functions in universal reproducing kernel Hilbert spaces. By directly approximating the one-step dynamics function using random features with uncertain parameters, which are equivalent to a shallow Bayesian neural network, we then view the whole dynamical system as a multi-layer neural network. Exploiting the structure of Hamiltonian dynamics, we show that finding worst-case dynamics realizations using Pontryagin's minimum principle is equivalent to performing the Frank-Wolfe algorithm on the deep net. Various numerical experiments on dynamics learning showcase the capacity of our modeling methodology.

arXiv comment9 pages, 5 figures

References in corpus (8)

• The generalization error of random features regression: Precise asymptotics and double descent curve
• Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
• Generalization Properties of Learning with Random Features
• Randomized Nonlinear Component Analysis
• Learning with SGD and Random Features
• Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond
• On the Approximation Properties of Random ReLU Features
• Meta Learning MPC using Finite-Dimensional Gaussian Process Approximations