7 citations · 37 across the 14 of their papers we have counts for
14 papers
Binary Hypothesis Testing for Softmax Models and Leverage Score Models
Yuzhou Gu, Zhao Song, Junze Yin
Softmax distributions are widely used in machine learning, including Large Language Models (LLMs), where the attention unit uses softmax distributions. We abstract the attention un…
Quantum Speedup for Spectral Approximation of Kronecker Products
Yeqi Gao, Zhao Song, Ruizhe Zhang
Given its widespread application in machine learning and optimization, the Kronecker product emerges as a pivotal linear algebra operator. However, its computational demands render…
A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time
Yeqi Gao, Zhao Song, Weixin Wang +1
Large language models (LLMs) have played a pivotal role in revolutionizing various facets of our daily existence. Solving attention regression is a fundamental task in optimizing L…
GradientCoin: A Peer-to-Peer Decentralized Large Language Models
Yeqi Gao, Zhao Song, Junze Yin
Since 2008, after the proposal of a Bitcoin electronic cash system, Bitcoin has fundamentally changed the economic system over the last decade. Since 2022, large language models (L…
In-Context Learning for Attention Scheme: from Single Softmax Regression to Multiple Softmax Regression via a Tensor Trick
Yeqi Gao, Zhao Song, Shenghao Xie
Large language models (LLMs) have brought significant and transformative changes in human society. These models have demonstrated remarkable capabilities in natural language unders…
Beyond Classical Attention: Quantum Attention for Scalable Computation
Xuyang Guo, Zhao Song, Xin Yang +1
As large language models (LLMs) demonstrate outstanding performance across various tasks, attention-driven models have profoundly transformed the field of machine learning. Since a…
Differentially Private Attention Computation
Yeqi Gao, Zhao Song, Xin Yang +1
Large language models (LLMs), especially those based on the Transformer architecture, have had a profound impact on various aspects of daily life, such as natural language processi…
An Iterative Algorithm for Rescaled Hyperbolic Functions Regression
Yeqi Gao, Zhao Song, Junze Yin
Large language models (LLMs) have numerous real-life applications across various domains, such as natural language translation, sentiment analysis, language modeling, chatbots and…
Solving Tensor Low Cycle Rank Approximation
Yichuan Deng, Yeqi Gao, Zhao Song
Large language models have become ubiquitous in modern life, finding applications in various domains such as natural language processing, language translation, and speech recogniti…
An Over-parameterized Exponential Regression
Yeqi Gao, Sridhar Mahadevan, Zhao Song
Over the past few years, there has been a significant amount of research focused on studying the ReLU activation function, with the aim of achieving neural network convergence thro…
An Time Fourier Set Query Algorithm
Yeqi Gao, Zhao Song, Baocheng Sun
Fourier transformation is an extensively studied problem in many research fields. It has many applications in machine learning, signal processing, compressed sensing, and so on. In…
A Sublinear Adversarial Training Algorithm
Yeqi Gao, Lianke Qin, Zhao Song +1
Adversarial training is a widely used strategy for making neural networks resistant to adversarial perturbations. For a neural network of width , input training data in …
Improved Reconstruction for Fourier-Sparse Signals
Yeqi Gao, Zhao Song, Baocheng Sun +2
We revisit the classical problem of Fourier-sparse signal reconstruction -- a variant of the \emph{Set Query} problem -- which asks to efficiently reconstruct (a subset of) a -d…
Optimizing Memory Efficiency of Graph Neural Networks on Edge Computing Platforms
Ao Zhou, Jianlei Yang, Yeqi Gao +7
Graph neural networks (GNN) have achieved state-of-the-art performance on various industrial tasks. However, the poor efficiency of GNN inference and frequent Out-Of-Memory (OOM) p…