36 citations · 96 across the 15 of their papers we have counts for
4 papers · 1 filter
Transformer-like Inference from Optimal Control
Aditya Kudre, Heng-Sheng Chang, Prashant G. Mehta
Decoder-only transformers compute the conditional probability of the next token from a sequence of past observations. This paper derives, from first principles, inference architect…
Differentiable Filtering for Learning Hidden Markov Models
Reginald Zhiyan Chen, Heng-Sheng Chang, Prashant G. Mehta
Hidden Markov Models (HMMs) are fundamental for modeling sequential data, yet learning their parameters from observations remains challenging. Classical methods like the Baum-Welch…
What can we learn from signals and systems in a transformer? Insights for probabilistic modeling and inference architecture
Heng-Sheng Chang, Prashant G. Mehta
In the 1940s, Wiener introduced a linear predictor, where the future prediction is computed by linearly combining the past data. A transformer generalizes this idea: it is a nonlin…
Dual Filter: A Transformer-like Inference Architecture for Hidden Markov Models
Heng-Sheng Chang, Prashant G. Mehta
This paper presents a mathematical framework for causal nonlinear prediction in settings where observations are generated from an underlying hidden Markov model (HMM). Both the pro…