4 papers · 1 filter
Log-Depth Recurrent Language Modeling
Yiqin Wang, Nuri Cingillioglu, Charles Pert
Language modeling using Transformers has become commonplace despite their fixed computational depth and quadratic runtime with respect to input tokens. Recurrent models on the othe…
Length Generalization with Log-Depth Recurrent Units
Charles Pert, Dalal Alrajeh, Alessandra Russo
Length generalization remains a persistent challenge for neural networks: recurrent models tend to suffer from positional biases, while transformers are constrained by fixed comput…
Beyond Fixed Tasks: Unsupervised Environment Design for Task-Level Pairs
Daniel Furelos-Blanco, Charles Pert, Frederik Kelbel +3
Training general agents to follow complex instructions (tasks) in intricate environments (levels) remains a core challenge in reinforcement learning. Random sampling of task-level…
RNN Generalization to Omega-Regular Languages
Charles Pert, Dalal Alrajeh, Alessandra Russo
Büchi automata (BAs) recognize -regular languages defined by formal specifications like linear temporal logic (LTL) and are commonly used in the verification of reactive systems…