The Garden of Forking Paths: Observing Dynamic Parameters Distribution in Large Language Models

Carlo Nicolini, Jacopo Staiano, Bruno Lepri +1

A substantial gap persists in understanding the reasons behind the exceptional performance of the Transformer architecture in NLP. A particularly unexplored area involves the mecha…