2 citations · 2 across the 4 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.CL2024
How Does Code Pretraining Affect Language Model Task Performance?
Jackson Petty, Sjoerd van Steenkiste, Tal Linzen
Large language models are increasingly trained on corpora containing both natural language and non-linguistic data like source code. Aside from aiding programming-related tasks, an…
cs.LG2024★ 2 cited
The Illusion of State in State-Space Models
William Merrill, Jackson Petty, Ashish Sabharwal
State-space models (SSMs) have emerged as a potential alternative architecture for building large language models (LLMs) compared to the previously ubiquitous transformer architect…