133 citations · 170 across the 5 of their papers we have counts for
5 papers
Testing Assumptions Underlying a Unified Theory for the Origin of Grid Cells
Rylan Schaeffer, Mikail Khona, Adrian Bertagnoli +2
Representing and reasoning about physical space is fundamental to animal survival, and the mammalian lineage expresses a wealth of specialized neural representations that encode sp…
Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cells
Rylan Schaeffer, Mikail Khona, Tzuhsuan Ma +3
To solve the spatial problems of mapping, localization and navigation, the mammalian lineage has developed striking spatial representations. One important spatial representation is…
Pretraining on the Test Set Is All You Need
Rylan Schaeffer
Inspired by recent work demonstrating the promise of smaller Transformer-based language models pretrained on carefully curated data, we supercharge such approaches by investing hea…
Are Emergent Abilities of Large Language Models a Mirage?
Rylan Schaeffer, Brando Miranda, Sanmi Koyejo
Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent…
Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle
Rylan Schaeffer, Mikail Khona, Zachary Robertson +5
Double descent is a surprising phenomenon in machine learning, in which as the number of model parameters grows relative to the number of data, test error drops as models grow ever…