6 papers
Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment
Mustafa Uzun, Mete Erdogan, Cengiz Pehlevan +1
We introduce Score Broadcast and Decorrelation (SBD), a principled framework for broadcast-based credit assignment for general families of differentiable losses. Error broadcast is…
Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
Clarissa Lauditi, Cengiz Pehlevan, Blake Bordelon
We study the evolution of hidden-weight spectra in wide neural networks trained by (stochastic) gradient descent. We develop a two-level dynamical mean-field theory (DMFT) that joi…
Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry
Bariscan Bozkurt, Cengiz Pehlevan, Alper T Erdogan
The backpropagation algorithm has experienced remarkable success in training large-scale artificial neural networks; however, its biological plausibility has been strongly criticiz…
Pixel-Based Similarities as an Alternative to Neural Data for Improving Convolutional Neural Network Adversarial Robustness
Elie Attias, Cengiz Pehlevan, Dina Obeid
Convolutional Neural Networks (CNNs) excel in many visual tasks but remain susceptible to adversarial attacks-imperceptible perturbations that degrade performance. Prior research r…
Demystifying LLM-as-a-Judge: Analytically Tractable Model for Inference-Time Scaling
Indranil Halder, Cengiz Pehlevan
Recent developments in large language models have shown advantages in reallocating a notable share of computational resource from training time to inference time. However, the prin…
Error Broadcast and Decorrelation as a Potential Artificial and Natural Learning Mechanism
Mete Erdogan, Cengiz Pehlevan, Alper T. Erdogan
We introduce Error Broadcast and Decorrelation (EBD), a novel learning framework for neural networks that addresses credit assignment by directly broadcasting output errors to indi…