2 papers
cs.LG2026
Screening Is Enough
Ken M. Nakanishi
A core limitation of standard softmax attention is that it does not provide an independently interpretable measure of query--key relevance: attention scores are unbounded, while at…
cs.CL2025
Scalable-Softmax Is Superior for Attention
Ken M. Nakanishi
The maximum element of the vector output by the Softmax function approaches zero as the input vector size increases. Transformer-based language models rely on Softmax to compute at…