134 citations · 184 across the 21 of their papers we have counts for
6 papers · 1 filter
Disentangling Fact from Grid Cell Fiction in Trained Deep Path Integrators
Rylan Schaeffer, Mikail Khona, Sanmi Koyejo +1
Work on deep learning-based models of grid cells suggests that grid cells generically and robustly arise from optimizing networks to path integrate, i.e., track one's spatial posit…
Testing Assumptions Underlying a Unified Theory for the Origin of Grid Cells
Rylan Schaeffer, Mikail Khona, Adrian Bertagnoli +2
Representing and reasoning about physical space is fundamental to animal survival, and the mammalian lineage expresses a wealth of specialized neural representations that encode sp…
Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cells
Rylan Schaeffer, Mikail Khona, Tzuhsuan Ma +3
To solve the spatial problems of mapping, localization and navigation, the mammalian lineage has developed striking spatial representations. One important spatial representation is…
Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks
Rahul Ramesh, Ekdeep Singh Lubana, Mikail Khona +2
Transformers trained on huge text corpora exhibit a remarkable set of capabilities, e.g., performing basic arithmetic. Given the inherent compositional nature of language, one can…
Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networks
Ziming Liu, Mikail Khona, Ila R. Fiete +1
Recurrent neural networks (RNNs) trained on compositional tasks can exhibit functional modularity, in which neurons can be clustered by activity similarity and participation in sha…
Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle
Rylan Schaeffer, Mikail Khona, Zachary Robertson +5
Double descent is a surprising phenomenon in machine learning, in which as the number of model parameters grows relative to the number of data, test error drops as models grow ever…