Deriving the Scaled-Dot-Function via Maximum Likelihood Estimation and Maximum Entropy Approach

Jiyong Ma

In this paper, we present a maximum likelihood estimation approach to determine the value vector in transformer models. We model the sequence of value vectors, key vectors, and the…