7 papers
Fast John Ellipsoid Computation with Differential Privacy Optimization
Xiaoyu Li, Yingyu Liang, Zhenmei Shi +2
Determining the John ellipsoid - the largest volume ellipsoid contained within a convex polytope - is a fundamental problem with applications in machine learning, optimization, and…
Provable Differentially Private Computation of the Cross-Attention Mechanism
Yekun Ke, Yingyu Liang, Zhenmei Shi +2
Cross-attention has emerged as a cornerstone module in modern artificial intelligence, underpinning critical applications such as retrieval-augmented generation (RAG), system promp…
Differential Privacy Mechanisms in Neural Tangent Kernel Regression
Jiuxiang Gu, Yingyu Liang, Zhizhou Sha +2
Training data privacy is a fundamental problem in modern Artificial Intelligence (AI) applications, such as face recognition, recommendation systems, language generation, and many…
Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspective
Yingyu Liang, Zhenmei Shi, Zhao Song +1
Diffusion models have made rapid progress in generating high-quality samples across various domains. However, a theoretical understanding of the Lipschitz continuity and second mom…
Training Tensor Attention Efficiently: From Cubic to Almost Linear Time
Yang Cao, Yingyu Liang, Zhenmei Shi +1
Tensor Attention, a multi-view attention that is able to capture high-order correlations among multiple modalities, can overcome the representational limitations of classical matri…
Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformers
Yingyu Liang, Heshan Liu, Zhenmei Shi +3
The self-attention mechanism is the key to the success of transformers in recent Large Language Models (LLMs). However, the quadratic computational cost in the input seque…