Active Learning for Nonlinear System Identification with Guarantees

Abstract

While the identification of nonlinear dynamical systems is a fundamental building block of model-based reinforcement learning and feedback control, its sample complexity is only understood for systems that either have discrete states and actions or for systems that can be identified from data generated by i.i.d. random inputs. Nonetheless, many interesting dynamical systems have continuous states and actions and can only be identified through a judicious choice of inputs. Motivated by practical settings, we study a class of nonlinear dynamical systems whose state transitions depend linearly on a known feature embedding of state-action pairs. To estimate such systems in finite time identification methods must explore all directions in feature space. We propose an active learning approach that achieves this by repeating three steps: trajectory planning, trajectory tracking, and re-estimation of the system from all available data. We show that our method estimates nonlinear dynamical systems at a parametric rate, similar to the statistical rate of standard linear regression.

29 pages

References in corpus (9)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Online Least Squares Estimation with Self-Normalized Processes: An Application to Bandit Problems
• Learning Linear Dynamical Systems with Semi-Parametric Least Squares
• Reward-Free Exploration for Reinforcement Learning
• Learning nonlinear dynamical systems from a single trajectory
• Spectral Filtering for General Linear Dynamical Systems
• Non-asymptotic and Accurate Learning of Nonlinear Dynamical Systems
• Sample Complexity of Kalman Filtering for Unknown Systems
• Convex Programming for Estimation in Nonlinear Recurrent Models

Cited by in corpus (18)

• Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control
• Information Theoretic Regret Bounds for Online Nonlinear Control
• Non-asymptotic and Accurate Learning of Nonlinear Dynamical Systems
• Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systems
• Instrumental Variable Value Iteration for Causal Offline Reinforcement Learning
• Representation Learning for Online and Offline RL in Low-rank MDPs
• PAC-Bayes Generalisation Bounds for Dynamical Systems Including Stable RNNs
• Certainty Equivalent Perception-Based Control
• MobILE: Model-Based Imitation Learning From Observation Alone
• Online Robust Control of Nonlinear Systems with Large Uncertainty
• Task-Optimal Exploration in Linear Dynamical Systems
• Towards a Dimension-Free Understanding of Adaptive Linear Control
• A Free Lunch from the Noise: Provable and Practical Exploration for Representation Learning
• Near-optimal Offline and Streaming Algorithms for Learning Non-Linear Dynamical Systems
• Sparsity in Partially Controllable Linear Systems
• Koopman Spectrum Nonlinear Regulators and Efficient Online Learning
• Regret Bounds for Adaptive Nonlinear Control
• Robust Online Control with Model Misspecification