4 papers
PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning
Alexis Fox, Junlin Wang, Paul Rosu +1
Long-horizon tasks require sustained perception, reasoning, and exploration, and are a persistent challenge for large language model (LLM) agents. This gap is reflected in their li…
Don't Retrain, Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment
Fred Zhangzhi Peng, Alexis Fox, Anru R. Zhang +1
Diffusion language models (DLMs) have recently demonstrated capabilities that complement standard autoregressive (AR) models, particularly in non-sequential generation and bidirect…
Interpretable-by-Design Transformers via Architectural Stream Independence
Clayton Kerce, Alexis Fox
While transformers achieve strong performance, their internal decision-making processes remain opaque. We investigate whether architectural constraints can enforce interpretability…
The Dual-Stream Transformer: Channelized Architecture for Interpretable Language Modeling
J. Clayton Kerce, Alexis Fox
Standard transformers entangle all computation in a single residual stream, obscuring which components perform which functions. We introduce the Dual-Stream Transformer, which deco…