4 papers · 1 filter
Transformer-like Inference from Optimal Control
Aditya Kudre, Heng-Sheng Chang, Prashant G. Mehta
Decoder-only transformers compute the conditional probability of the next token from a sequence of past observations. This paper derives, from first principles, inference architect…
Differentiable Filtering for Learning Hidden Markov Models
Reginald Zhiyan Chen, Heng-Sheng Chang, Prashant G. Mehta
Hidden Markov Models (HMMs) are fundamental for modeling sequential data, yet learning their parameters from observations remains challenging. Classical methods like the Baum-Welch…
Dual Filter: A Transformer-like Inference Architecture for Hidden Markov Models
Heng-Sheng Chang, Prashant G. Mehta
This paper presents a mathematical framework for causal nonlinear prediction in settings where observations are generated from an underlying hidden Markov model (HMM). Both the pro…
What can we learn from signals and systems in a transformer? Insights for probabilistic modeling and inference architecture
Heng-Sheng Chang, Prashant G. Mehta
In the 1940s, Wiener introduced a linear predictor, where the future prediction is computed by linearly combining the past data. A transformer generalizes this idea: it is a nonlin…