Can a Transformer Represent a Kalman Filter?

Gautam Goel, Peter Bartlett

Transformers are a class of autoregressive deep learning architectures which have recently achieved state-of-the-art performance in various vision, language, and robotics tasks. We…