4 papers
The Topological Trouble With Transformers
Michael C. Mozer, Shoaib Ahmed Siddiqui, Rosanne Liu
Transformers encode structure in sequences via an expanding contextual history. However, their purely feedforward architecture fundamentally limits dynamic state tracking. State tr…
Do Real-World Datasets Contain Natural Experiments? An Empirical Study Using Causal Feature Selection
Gautam Gare, John Galeotti, Michael Mozer +2
In nature, events that affect some individuals or groups but not others constitute an implicit intervention and are known as natural experiments. For example, the COVID-19 pandemic…
Analysis of Optimality of Large Language Models on Planning Problems
Bernd Bohnet, Michael C. Mozer, Kevin Swersky +4
Classic AI planning problems have been revisited in the Large Language Model (LLM) era, with a focus of recent benchmarks on success rates rather than plan efficiency. We examine t…
Can foundation models actively gather information in interactive environments to test hypotheses?
Danny P. Sawyer, Nan Rosemary Ke, Hubert Soyer +9
Foundation models excel at single-turn reasoning but struggle with multi-turn exploration in dynamic environments, a requirement for many real-world challenges. We evaluated these…