4 papers
Beyond Classical Attention: Quantum Attention for Scalable Computation
Xuyang Guo, Zhao Song, Xin Yang +1
As large language models (LLMs) demonstrate outstanding performance across various tasks, attention-driven models have profoundly transformed the field of machine learning. Since a…
Binary Hypothesis Testing for Softmax Models and Leverage Score Models
Yuzhou Gu, Zhao Song, Junze Yin
Softmax distributions are widely used in machine learning, including Large Language Models (LLMs), where the attention unit uses softmax distributions. We abstract the attention un…
An Iterative Algorithm for Rescaled Hyperbolic Functions Regression
Yeqi Gao, Zhao Song, Junze Yin
Large language models (LLMs) have numerous real-life applications across various domains, such as natural language translation, sentiment analysis, language modeling, chatbots and…
Differentially Private Attention Computation
Yeqi Gao, Zhao Song, Xin Yang +1
Large language models (LLMs), especially those based on the Transformer architecture, have had a profound impact on various aspects of daily life, such as natural language processi…