10 citations · 39 across the 22 of their papers we have counts for
16 papers · 1 filter
Local Convergence of Approximate Newton Method for Two Layer Nonlinear Regression
Zhihang Li, Zhao Song, Zifan Wang +1
There have been significant advancements made by large language models (LLMs) in various aspects of our daily lives. LLMs serve as a transformative force in natural language proces…
Revisiting Quantum Algorithms for Linear Regressions: Quadratic Speedups without Data-Dependent Parameters
Zhao Song, Junze Yin, Ruizhe Zhang
Linear regression is one of the most fundamental linear algebra problems. Given a dense matrix and a vector , the goal is to find such that…
The Expressibility of Polynomial based Attention Scheme
Zhao Song, Guangyi Xu, Junze Yin
Large language models (LLMs) have significantly improved various aspects of our daily lives. These models have impacted numerous domains, from healthcare to education, enhancing pr…
A Unified Scheme of ResNet and Softmax
Zhao Song, Weixin Wang, Junze Yin
Large language models (LLMs) have brought significant changes to human society. Softmax regression and residual neural networks (ResNet) are two important techniques in deep learni…
A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time
Yeqi Gao, Zhao Song, Weixin Wang +1
Large language models (LLMs) have played a pivotal role in revolutionizing various facets of our daily existence. Solving attention regression is a fundamental task in optimizing L…
GradientCoin: A Peer-to-Peer Decentralized Large Language Models
Yeqi Gao, Zhao Song, Junze Yin
Since 2008, after the proposal of a Bitcoin electronic cash system, Bitcoin has fundamentally changed the economic system over the last decade. Since 2022, large language models (L…
Solving Attention Kernel Regression Problem via Pre-conditioner
Zhao Song, Junze Yin, Lichen Zhang
The attention mechanism is the key to large language models, and the attention matrix serves as an algorithmic and computational bottleneck for such a scheme. In this paper, we def…
Dynamical Fractal: Theory and Case Study
Junze Yin
Urbanization is a phenomenon of concern for planning and public health: projections are difficult because of policy changes and natural events, and indicators are multiple. There a…
Query Complexity of Active Learning for Function Family With Nearly Orthogonal Basis
Xiang Chen, Zhao Song, Baocheng Sun +2
Many machine learning algorithms require large numbers of labeled data to deliver state-of-the-art results. In applications such as medical diagnosis and fraud detection, though th…
Efficient Alternating Minimization with Applications to Weighted Low Rank Approximation
Zhao Song, Mingquan Ye, Junze Yin +1
Weighted low rank approximation is a fundamental problem in numerical linear algebra, and it has many applications in machine learning. Given a matrix $M \in \mathbb{R}^{n \times n…
Faster Robust Tensor Power Method for Arbitrary Order
Yichuan Deng, Zhao Song, Junze Yin
Tensor decomposition is a fundamental method used in various areas to deal with high-dimensional data. \emph{Tensor power method} (TPM) is one of the widely-used techniques in the…
Federated Empirical Risk Minimization via Second-Order Method
Song Bian, Zhao Song, Junze Yin
Many convex optimization problems with important applications in machine learning are formulated as empirical risk minimization (ERM). There are several examples: linear and logist…
Fast and Efficient Matching Algorithm with Deadline Instances
Zhao Song, Weixin Wang, Chenbo Yin +1
The online weighted matching problem is a fundamental problem in machine learning due to its numerous applications. Despite many efforts in this area, existing algorithms are eithe…
An Iterative Algorithm for Rescaled Hyperbolic Functions Regression
Yeqi Gao, Zhao Song, Junze Yin
Large language models (LLMs) have numerous real-life applications across various domains, such as natural language translation, sentiment analysis, language modeling, chatbots and…
Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time
Yuzhou Gu, Zhao Song, Junze Yin +1
Given a matrix , the low rank matrix completion problem asks us to find a rank- approximation of as for …
A Nearly-Optimal Bound for Fast Regression with Guarantee
Zhao Song, Mingquan Ye, Junze Yin +1
Given a matrix and a vector , we consider the regression problem with guarantees: finding a vector $x'\in \mathbb{R}^…