3 papers
cs.LG2024
Autocorrelation Matters: Understanding the Role of Initialization Schemes for State Space Models
Fusheng Liu, Qianxiao Li
Current methods for initializing state space model (SSM) parameters primarily rely on the HiPPO framework \citep{gu2023how}, which is based on online function approximation with th…
cs.LG2024
From Generalization Analysis to Optimization Designs for State Space Models
Fusheng Liu, Qianxiao Li
A State Space Model (SSM) is a foundation model in time series analysis, which has recently been shown as an alternative to transformers in sequence modeling. In this paper, we the…
stat.ML2022
From Optimization Dynamics to Generalization Bounds via Łojasiewicz Gradient Inequality
Fusheng Liu, Haizhao Yang, Soufiane Hayou +1
Optimization and generalization are two essential aspects of statistical machine learning. In this paper, we propose a framework to connect optimization with generalization by anal…