2 papers
cs.LG2026
How Language Models Differ in Redistributing Attention-Head Activity Under Serial Demand
Johnny Jingze Li, Abdulla Kuleib, Kalyan Basu +1
The way a model distributes activity over each layer's attention heads offers a coarse view of how it routes information through depth; how this changes with the task is part of wh…
cs.LG2024
Quantifying Emergence in Neural Networks: Insights from Pruning and Training Dynamics
Faisal AlShinaifi, Zeyad Almoaigel, Johnny Jingze Li +2
Emergence, where complex behaviors develop from the interactions of simpler components within a network, plays a crucial role in enhancing neural network capabilities. We introduce…