6 citations · 21 across the 13 of their papers we have counts for
14 papers
Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment
Mustafa Uzun, Mete Erdogan, Cengiz Pehlevan +1
We introduce Score Broadcast and Decorrelation (SBD), a principled framework for broadcast-based credit assignment for general families of differentiable losses. Error broadcast is…
Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
Clarissa Lauditi, Cengiz Pehlevan, Blake Bordelon
We study the evolution of hidden-weight spectra in wide neural networks trained by (stochastic) gradient descent. We develop a two-level dynamical mean-field theory (DMFT) that joi…
Demystifying LLM-as-a-Judge: Analytically Tractable Model for Inference-Time Scaling
Indranil Halder, Cengiz Pehlevan
Recent developments in large language models have shown advantages in reallocating a notable share of computational resource from training time to inference time. However, the prin…
Error Broadcast and Decorrelation as a Potential Artificial and Natural Learning Mechanism
Mete Erdogan, Cengiz Pehlevan, Alper T. Erdogan
We introduce Error Broadcast and Decorrelation (EBD), a novel learning framework for neural networks that addresses credit assignment by directly broadcasting output errors to indi…
Pixel-Based Similarities as an Alternative to Neural Data for Improving Convolutional Neural Network Adversarial Robustness
Elie Attias, Cengiz Pehlevan, Dina Obeid
Convolutional Neural Networks (CNNs) excel in many visual tasks but remain susceptible to adversarial attacks-imperceptible perturbations that degrade performance. Prior research r…
Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry
Bariscan Bozkurt, Cengiz Pehlevan, Alper T Erdogan
The backpropagation algorithm has experienced remarkable success in training large-scale artificial neural networks; however, its biological plausibility has been strongly criticiz…
Correlative Information Maximization Based Biologically Plausible Neural Networks for Correlated Source Separation
Bariscan Bozkurt, Ates Isfendiyaroglu, Cengiz Pehlevan +1
The brain effortlessly extracts latent causes of stimuli, but how it does this at the network level remains unknown. Most prior attempts at this problem proposed neural networks th…
Biologically-Plausible Determinant Maximization Neural Networks for Blind Separation of Correlated Sources
Bariscan Bozkurt, Cengiz Pehlevan, Alper T. Erdogan
Extraction of latent sources of complex stimuli is critical for making sense of the world. While the brain solves this blind source separation (BSS) problem continuously, its algor…
Attention Approximates Sparse Distributed Memory
Trenton Bricken, Cengiz Pehlevan
While Attention has come to be an important mechanism in deep learning, there remains limited intuition for why it works so well. Here, we show that Transformer Attention can be cl…
Associative Memory in Iterated Overparameterized Sigmoid Autoencoders
Yibo Jiang, Cengiz Pehlevan
Recent work showed that overparameterized autoencoders can be trained to implement associative memory via iterative maps, when the trained input-output Jacobian of the network has…
Minimax Dynamics of Optimally Balanced Spiking Networks of Excitatory and Inhibitory Neurons
Qianyi Li, Cengiz Pehlevan
Excitation-inhibition (E-I) balance is ubiquitously observed in the cortex. Recent studies suggest an intriguing link between balance on fast timescales, tight balance, and efficie…
Blind Bounded Source Separation Using Neural Networks with Local Learning Rules
Alper T. Erdogan, Cengiz Pehlevan
An important problem encountered by both natural and engineered signal processing systems is blind source separation. In many instances of the problem, the sources are bounded by t…
Structured and Deep Similarity Matching via Structured and Deep Hebbian Networks
Dina Obeid, Hugo Ramambason, Cengiz Pehlevan
Synaptic plasticity is widely accepted to be the mechanism behind learning in the brain's neural networks. A central question is how synapses, with access to only local information…
A Spiking Neural Network with Local Learning Rules Derived From Nonnegative Similarity Matching
Cengiz Pehlevan
The design and analysis of spiking neural network algorithms will be accelerated by the advent of new theoretical approaches. In an attempt at such approach, we provide a principle…