Binary Hypothesis Testing for Softmax Models and Leverage Score Models

Yuzhou Gu, Zhao Song, Junze Yin

Softmax distributions are widely used in machine learning, including Large Language Models (LLMs), where the attention unit uses softmax distributions. We abstract the attention un…