85 citations · 273 across the 13 of their papers we have counts for
4 papers · 1 filter
NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime
Gengze Zhou, Yicong Hong, Jiazhao Zhang +9
Language models trained with long-horizon agentic reinforcement learning can generalize knowledge through reasoning, express precise actions, and pursue goals over many steps, rais…
Lasting Effects of Abstract Pretraining Beyond Perplexity
Zachary Shinnick, Hemanth Saratchandran, Damien Teney +1
Language models are typically pretrained from random initialization. Recent work challenges this convention, showing that a brief warm-up on abstract, algorithmically generated dat…
Procedural Core: A Compact Recurrent Initialization for Vision Transformers
Zachary Shinnick, Christian Internò, Hemanth Saratchandran +2
Transformers are typically trained from random initialization, requiring all their capabilities to emerge from large-scale optimization. Recent work showed that a small amount of a…
Back from the Future: Key-Value Cache Management by Counter-Causal Surprise
Stephen Gould, Anton van den Hengel
Key-value (KV) cache management through compression and eviction strategies has emerged as an important research direction in recent years. Computational demands of large language…