3 papers
q-bio.NC2025
A Network of Biologically Inspired Rectified Spectral Units (ReSUs) Learns Hierarchical Features Without Error Backpropagation
Shanshan Qin, Joshua L. Pughe-Sanford, Alexander Genkin +4
We introduce a biologically inspired, multilayer neural architecture composed of Rectified Spectral Units (ReSUs). Each ReSU projects a recent window of its input history onto a ca…
q-bio.NC2025
Neurons as Detectors of Coherent Sets in Sensory Dynamics
Joshua L. Pughe-Sanford, Xuehao Ding, Jason J. Moore +4
We model sensory streams as observations from high-dimensional stochastic dynamical systems and conceptualize sensory neurons as self-supervised learners of compact representations…
cs.LG2024
LoRA Learns Less and Forgets Less
Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz +9
Low-Rank Adaptation (LoRA) is a widely-used parameter-efficient finetuning method for large language models. LoRA saves memory by training only low rank perturbations to selected w…