Law of Neural Interaction: Depth-Width Shape, Interaction Efficiency, and Generalization

Wenjie Sun, Jinning Yang, Shuai Zhang +1

The guidance of scaling laws has increased the resource demands of modern large language models (LLMs), yet it remains questionable whether these models utilize resources effective…