12 papers
Duality Theory for Filter Stability in Hidden Markov Models
Prashant G. Mehta
This article is concerned with the problem of nonlinear (stochastic) filter stability for a discrete time hidden Markov model (HMM) with finite state and observation spaces. For th…
Transformer-like Inference from Optimal Control
Aditya Kudre, Heng-Sheng Chang, Prashant G. Mehta
Decoder-only transformers compute the conditional probability of the next token from a sequence of past observations. This paper derives, from first principles, inference architect…
Duality Theory for Non-Markovian Linear Gaussian Models
Aditya Kudre, Heng-Sheng Chang, Prashant G. Mehta
This work develops a duality theory for partially observed linear Gaussian models in discrete time. The state process evolves according to a causal but non-Markovian (or higher-ord…
Differentiable Filtering for Learning Hidden Markov Models
Reginald Zhiyan Chen, Heng-Sheng Chang, Prashant G. Mehta
Hidden Markov Models (HMMs) are fundamental for modeling sequential data, yet learning their parameters from observations remains challenging. Classical methods like the Baum-Welch…
What can we learn from signals and systems in a transformer? Insights for probabilistic modeling and inference architecture
Heng-Sheng Chang, Prashant G. Mehta
In the 1940s, Wiener introduced a linear predictor, where the future prediction is computed by linearly combining the past data. A transformer generalizes this idea: it is a nonlin…
Dual Filter: A Transformer-like Inference Architecture for Hidden Markov Models
Heng-Sheng Chang, Prashant G. Mehta
This paper presents a mathematical framework for causal nonlinear prediction in settings where observations are generated from an underlying hidden Markov model (HMM). Both the pro…
Error Analysis of Sampling Algorithms for Approximating Stochastic Optimal Control
Anant A. Joshi, Amirhossein Taghvaei, Prashant G. Mehta
This paper is concerned with the error analysis of two types of sampling algorithms, namely model predictive path integral (MPPI) and an interacting particle system (\IPS) algorith…
How to implement the Bayes' formula in the age of ML?
Amirhossein Taghvaei, Prashant G. Mehta
This chapter contains a self-contained introduction to the significance of Bayes' formula in the context of nonlinear filtering problems. Both discrete-time and continuous-time set…
Interacting Particle Systems for Fast Linear Quadratic RL
Anant A Joshi, Heng-Sheng Chang, Amirhossein Taghvaei +2
This paper is concerned with the design of algorithms based on systems of interacting particles to represent, approximate, and learn the optimal control law for reinforcement learn…
Arrow of Time in Estimation and Control: Duality Theory Beyond the Linear Gaussian Model
Jin Won Kim, Prashant G. Mehta
Duality between estimation and control is a foundational concept in Control Theory. Most students learn about the elementary duality -- between observability and controllability --…
Backward Map for Filter Stability Analysis
Jin Won Kim, Anant A. Joshi, Prashant G. Mehta
In this paper, a backward map is introduced for the purposes of analysis of the nonlinear (stochastic) filter stability. The backward map is important because the filter-stability…
Duality for nonlinear filtering
Jin Won Kim
This thesis is concerned with the stochastic filtering problem for a hidden Markov model (HMM) with the white noise observation model. For this filtering problem, we make three typ…