12 citations · 12 across the 8 of their papers we have counts for
8 papers
TANGO: Treating Tokens as Operators
Joshua Nunley
Transformers separate cross-token mixing in self-attention from token-wise transformation in feed-forward networks. We ask whether combining these operations can lower predictive l…
Kuramoto Attention: Synchronizing Self-Attention on the Torus
Joshua Nunley
Transformer models are increasingly used as computational models of cognition and neural representation, so the mechanism implemented by self-attention is of interest beyond engine…
Attention as Frustrated Synchronization
Joshua Nunley
A network of oscillators that synchronizes perfectly computes nothing further, so an attention architecture built from synchronization must locate its computation in structured dep…
Self-Regulation through Communication in Evolved Neural Agents
Joshua Nunley
Communication is typically understood as indication: signals that transfer information from sender to receiver. We present a minimal predator avoidance task in which pairs of evolv…
Democracy on Rugged Landscapes: Phase Transitions in Optimal Voting Rules
Joshua Nunley
Laws and institutions shape individual outcomes through complex interactions with citizens' diverse circumstances, yet how different voting methods navigate this coupled landscape…
Subgroups of Induce Natural RNN and Transformer Architectures
Joshua Nunley
This paper presents a direct framework for sequence models with hidden states on closed subgroups of U(d). We use a minimal axiomatic setup and derive recurrent and transformer tem…
Neural Fields as World Models
Joshua Nunley
Humans rehearse possible futures offline, as in mental practice and perhaps dreaming, suggesting that world models may support task learning away from the environment. Standard mac…
Universal wavenumber selection laws in apical growth
Ryan Goh, Rajendra Beekie, Daniel Matthias +2
We study pattern-forming dissipative systems in growing domains. We characterize classes of boundary conditions that allow for defect-free growth and derive universal scaling laws…