7 papers
Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment
Mustafa Uzun, Mete Erdogan, Cengiz Pehlevan +1
We introduce Score Broadcast and Decorrelation (SBD), a principled framework for broadcast-based credit assignment for general families of differentiable losses. Error broadcast is…
Generalized Gradient Norm Clipping & Non-Euclidean -Smoothness
Thomas Pethick, Wanyun Xie, Mete Erdogan +3
This work introduces a hybrid non-Euclidean optimization method which generalizes gradient norm clipping by combining steepest descent and conditional gradient approaches. The meth…
Tangent Space Fine-Tuning for Directional Preference Alignment in Large Language Models
Mete Erdogan
Our goal is to enable large language models (LLMs) to balance multiple human preference dimensions; such as helpfulness, safety, and verbosity, through principled and controllable…
An Information-Theoretic Perspective on LLM Tokenizers
Mete Erdogan, Abhiram Gorle, Shubham Chandak +2
Large language model (LLM) tokenizers act as structured compressors: by mapping text to discrete token sequences, they determine token count (and thus compute and context usage) an…
Error Broadcast and Decorrelation as a Potential Artificial and Natural Learning Mechanism
Mete Erdogan, Cengiz Pehlevan, Alper T. Erdogan
We introduce Error Broadcast and Decorrelation (EBD), a novel learning framework for neural networks that addresses credit assignment by directly broadcasting output errors to indi…
Efficient Large Language Model Inference with Neural Block Linearization
Mete Erdogan, Francesco Tonin, Volkan Cevher
The high inference demands of transformer-based Large Language Models (LLMs) pose substantial challenges in their deployment. To this end, we introduce Neural Block Linearization (…