3 papers
q-bio.NC2025
A Network of Biologically Inspired Rectified Spectral Units (ReSUs) Learns Hierarchical Features Without Error Backpropagation
Shanshan Qin, Joshua L. Pughe-Sanford, Alexander Genkin +4
We introduce a biologically inspired, multilayer neural architecture composed of Rectified Spectral Units (ReSUs). Each ReSU projects a recent window of its input history onto a ca…
q-bio.NC2025
Neurons as Detectors of Coherent Sets in Sensory Dynamics
Joshua L. Pughe-Sanford, Xuehao Ding, Jason J. Moore +4
We model sensory streams as observations from high-dimensional stochastic dynamical systems and conceptualize sensory neurons as self-supervised learners of compact representations…
cs.LG2025
In-context denoising with one-layer transformers: connections between attention and associative memory retrieval
Matthew Smart, Alberto Bietti, Anirvan M. Sengupta
We introduce in-context denoising, a task that refines the connection between attention-based architectures and dense associative memory (DAM) networks, also known as modern Hopfie…