On Limitation of Transformer for Learning HMMs

Jiachen Hu, Qinghua Liu, Chi Jin

Despite the remarkable success of Transformer-based architectures in various sequential modeling tasks, such as natural language processing, computer vision, and robotics, their ab…