7 papers
Pretraining Recurrent Networks without Recurrence
Akarsh Kumar, Phillip Isola
Training recurrent neural networks (RNNs) requires assigning credit across long sequences of computations. Standard backpropagation through time (BPTT) addresses this problem poorl…
In Search of the Ingredients of Open-Endedness: Replicating Picbreeder with Large Vision-Language Models
Sam Earle, Kai Arulkumaran, Andrew Dai +3
We are in the midst of large-scale industrial and academic efforts to automate the processes of scientific, technological and creative production through AI-driven assistants. Hist…
Vector Policy Optimization: Training for Diversity Improves Test-Time Search
Ryan Bahlous-Boldi, Isha Puri, Idan Shenfeld +6
Language models must now generalize out of the box to novel environments and work inside inference-scaling search procedures, such as AlphaEvolve, that select rollouts with a varie…
Training Language Models via Neural Cellular Automata
Dan Lee, Seungwook Han, Akarsh Kumar +1
Pre-training is crucial for large language models (LLMs), as it is when most representations and capabilities are acquired. However, natural language pre-training has problems: hig…
Digital Red Queen: Adversarial Program Evolution in Core War with LLMs
Akarsh Kumar, Ryan Bahlous-Boldi, Prafull Sharma +4
Large language models (LLMs) are increasingly being used to evolve solutions to problems in many domains, in a process inspired by biological evolution. However, unlike biological…
Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis
Akarsh Kumar, Jeff Clune, Joel Lehman +1
Much of the excitement in modern AI is driven by the observation that scaling up existing systems leads to better performance. But does better performance necessarily imply better…