5 papers · 1 filter
Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment
Mustafa Uzun, Mete Erdogan, Cengiz Pehlevan +1
We introduce Score Broadcast and Decorrelation (SBD), a principled framework for broadcast-based credit assignment for general families of differentiable losses. Error broadcast is…
Generalized Gradient Norm Clipping & Non-Euclidean -Smoothness
Thomas Pethick, Wanyun Xie, Mete Erdogan +3
This work introduces a hybrid non-Euclidean optimization method which generalizes gradient norm clipping by combining steepest descent and conditional gradient approaches. The meth…
Tangent Space Fine-Tuning for Directional Preference Alignment in Large Language Models
Mete Erdogan
Our goal is to enable large language models (LLMs) to balance multiple human preference dimensions; such as helpfulness, safety, and verbosity, through principled and controllable…
Error Broadcast and Decorrelation as a Potential Artificial and Natural Learning Mechanism
Mete Erdogan, Cengiz Pehlevan, Alper T. Erdogan
We introduce Error Broadcast and Decorrelation (EBD), a novel learning framework for neural networks that addresses credit assignment by directly broadcasting output errors to indi…
Efficient Large Language Model Inference with Neural Block Linearization
Mete Erdogan, Francesco Tonin, Volkan Cevher
The high inference demands of transformer-based Large Language Models (LLMs) pose substantial challenges in their deployment. To this end, we introduce Neural Block Linearization (…