How Entropic Regression Beats the Outliers Problem in Nonlinear System Identification

Abstract

In this work, we developed a nonlinear System Identification (SID) method that we called Entropic Regression. Our method adopts an information-theoretic measure for the data-driven discovery of the underlying dynamics. Our method shows robustness toward noise and outliers and it outperforms many of the current state-of-the-art methods. Moreover, the method of Entropic Regression overcomes many of the major limitations of the current methods such as sloppy parameters, diverse scale, and SID in high dimensional systems such as complex networks. The use of information-theoretic measures in entropic regression poses unique advantages, due to the Asymptotic Equipartition Property (AEP) of probability distributions, that outliers and other low-occurrence events are conveniently and intrinsically de-emphasized as not-typical, by definition. We provide a numerical comparison with the current state-of-the-art methods in sparse regression, and we apply the methods to different chaotic systems such as the Lorenz System, the Kuramoto-Sivashinsky equations, and the Double Well Potential.

References in corpus (13)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Measuring Information Transfer
• Estimating Mutual Information
• Predicting catastrophes in nonlinear dynamical systems by compressive sensing
• Data Based Identification and Prediction of Nonlinear and Complex Dynamical Systems
• Causal Network Inference by Optimal Causation Entropy
• Causation Entropy Identifies Indirect Influences, Dominance of Neighbors and Anticipatory Couplings
• Hopf's last hope: spatiotemporal chaos in terms of unstable recurrent patterns
• Synchronized clusters in coupled map networks: Self-organized and driven phase synchronization
• Unstable recurrent patterns in Kuramoto-Sivashinsky dynamics
• Identifying Coupling Structure in Complex Systems through the Optimal Causation Entropy Principle
• Universal structural estimator and dynamics approximator for complex networks
• Anomalous synchronization threshold in coupled logistic maps

Cited by in corpus (18)

• Next Generation Reservoir Computing
• Knowledge-Based Learning of Nonlinear Dynamics and Chaos
• Finding nonlinear system equations and complex network structures from data: a sparse optimization approach
• Model selection of chaotic systems from data with hidden variables using sparse data assimilation
• Sparse Methods for Automatic Relevance Determination
• Symbolic Regression via Neural Networks
• Entropic Causal Inference for Neurological Applications
• CEBoosting: Online Sparse Identification of Dynamical Systems with Regime Switching by Causation Entropy Boosting
• Data-Driven Statistical Reduced-Order Modeling and Quantification of Polycrystal Mechanics Leading to Porosity-Based Ductile Damage
• CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation
• CGKN: A Deep Learning Framework for Modeling Complex Dynamical Systems and Efficient Data Assimilation
• Bridging the Gap: Machine Learning to Resolve Improperly Modeled Dynamics
• SubTSBR to tackle high noise and outliers for data-driven discovery of differential equations
• On Geometry of Information Flow for Causal Inference
• ERFit: Entropic Regression Fit Matlab Package, for Data-Driven System Identification of Underlying Dynamic Equations
• Entropic Regression DMD (ERDMD) Discovers Informative Sparse and Nonuniformly Time Delayed Models
• A Martingale-Free Introduction to Conditional Gaussian Nonlinear Systems
• Inference of Stochastic Dynamical Systems from Cross-Sectional Population Data