10 citations · 39 across the 22 of their papers we have counts for
16 papers · 1 filter
Inverting the Leverage Score Gradient: An Efficient Approximate Newton Method
Chenyang Li, Zhao Song, Zhaoxing Xu +1
Leverage scores have become essential in statistics and machine learning, aiding regression analysis, randomized matrix computations, and various other tasks. This paper delves int…
Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformers
Yingyu Liang, Heshan Liu, Zhenmei Shi +3
The self-attention mechanism is the key to the success of transformers in recent Large Language Models (LLMs). However, the quadratic computational cost in the input seque…
How to Inverting the Leverage Score Distribution?
Zhihang Li, Zhao Song, Weixin Wang +2
Leverage score is a fundamental problem in machine learning and theoretical computer science. It has extensive applications in regression analysis, randomized algorithms, and neura…
Local Convergence of Approximate Newton Method for Two Layer Nonlinear Regression
Zhihang Li, Zhao Song, Zifan Wang +1
There have been significant advancements made by large language models (LLMs) in various aspects of our daily lives. LLMs serve as a transformative force in natural language proces…
The Expressibility of Polynomial based Attention Scheme
Zhao Song, Guangyi Xu, Junze Yin
Large language models (LLMs) have significantly improved various aspects of our daily lives. These models have impacted numerous domains, from healthcare to education, enhancing pr…
A Unified Scheme of ResNet and Softmax
Zhao Song, Weixin Wang, Junze Yin
Large language models (LLMs) have brought significant changes to human society. Softmax regression and residual neural networks (ResNet) are two important techniques in deep learni…
GradientCoin: A Peer-to-Peer Decentralized Large Language Models
Yeqi Gao, Zhao Song, Junze Yin
Since 2008, after the proposal of a Bitcoin electronic cash system, Bitcoin has fundamentally changed the economic system over the last decade. Since 2022, large language models (L…
Solving Attention Kernel Regression Problem via Pre-conditioner
Zhao Song, Junze Yin, Lichen Zhang
The attention mechanism is the key to large language models, and the attention matrix serves as an algorithmic and computational bottleneck for such a scheme. In this paper, we def…
Query Complexity of Active Learning for Function Family With Nearly Orthogonal Basis
Xiang Chen, Zhao Song, Baocheng Sun +2
Many machine learning algorithms require large numbers of labeled data to deliver state-of-the-art results. In applications such as medical diagnosis and fraud detection, though th…
Efficient Alternating Minimization with Applications to Weighted Low Rank Approximation
Zhao Song, Mingquan Ye, Junze Yin +1
Weighted low rank approximation is a fundamental problem in numerical linear algebra, and it has many applications in machine learning. Given a matrix $M \in \mathbb{R}^{n \times n…
Faster Robust Tensor Power Method for Arbitrary Order
Yichuan Deng, Zhao Song, Junze Yin
Tensor decomposition is a fundamental method used in various areas to deal with high-dimensional data. \emph{Tensor power method} (TPM) is one of the widely-used techniques in the…
Federated Empirical Risk Minimization via Second-Order Method
Song Bian, Zhao Song, Junze Yin
Many convex optimization problems with important applications in machine learning are formulated as empirical risk minimization (ERM). There are several examples: linear and logist…
An Iterative Algorithm for Rescaled Hyperbolic Functions Regression
Yeqi Gao, Zhao Song, Junze Yin
Large language models (LLMs) have numerous real-life applications across various domains, such as natural language translation, sentiment analysis, language modeling, chatbots and…
Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time
Yuzhou Gu, Zhao Song, Junze Yin +1
Given a matrix , the low rank matrix completion problem asks us to find a rank- approximation of as for …
Dynamic Maintenance of Kernel Density Estimation Data Structure: From Practice to Theory
Jiehao Liang, Zhao Song, Zhaozhuo Xu +2
Kernel density estimation (KDE) stands out as a challenging task in machine learning. The problem is defined in the following way: given a kernel function and a set of poi…
InstaHide's Sample Complexity When Mixing Two Private Images
Baihe Huang, Zhao Song, Runzhou Tao +3
Training neural networks usually require large numbers of sensitive training data, and how to protect the privacy of training data has thus become a critical topic in deep learning…