Probing Information Distribution in Transformer Architectures through Entropy Analysis

Amedeo Buonanno, Alessandro Rivetti, Francesco A. N. Palmieri +2

This work explores entropy analysis as a tool for probing information distribution within Transformer-based architectures. By quantifying token-level uncertainty and examining entr…