Markovian Transformers for Informative Language Modeling

Scott Viteri, Max Lamparth, Peter Chatain +1

Chain-of-Thought (CoT) reasoning often fails to faithfully reflect a language model's underlying decision process. We address this by introducing a Markovian language model framewo…