32 citations · 80 across the 15 of their papers we have counts for
15 papers
Formal guarantees for heuristic optimization algorithms used in machine learning
Xiaoyu Li
Recently, Stochastic Gradient Descent (SGD) and its variants have become the dominant methods in the large-scale optimization of machine learning (ML) problems. A variety of strate…
On Distributed Adaptive Optimization with Gradient Compression
Xiaoyun Li, Belhal Karimi, Ping Li
We study COMP-AMS, a distributed optimization framework based on gradient averaging and adaptive AMSGrad algorithm. Gradient compression with error feedback is applied to reduce th…
C-OPH: Improving the Accuracy of One Permutation Hashing (OPH) with Circulant Permutations
Xiaoyun Li, Ping Li
Minwise hashing (MinHash) is a classical method for efficiently estimating the Jaccrad similarity in massive binary (0/1) data. To generate hash values for each data vector, th…
Toward Communication Efficient Adaptive Gradient Method
Xiangyi Chen, Xiaoyun Li, Ping Li
In recent years, distributed optimization is proven to be an effective approach to accelerate training of large scale machine learning models such as deep neural networks. With the…
C-MinHash: Practically Reducing Two Permutations to Just One
Xiaoyun Li, Ping Li
Traditional minwise hashing (MinHash) requires applying independent permutations to estimate the Jaccard similarity in massive binary (0/1) data, where can be (e.g.,) 1024…
C-MinHash: Rigorously Reducing Permutations to Two
Xiaoyun Li, Ping Li
Minwise hashing (MinHash) is an important and practical algorithm for generating random hashes to approximate the Jaccard (resemblance) similarity in massive binary (0/1) data. The…
Quantization Algorithms for Random Fourier Features
Xiaoyun Li, Ping Li
The method of random projection (RP) is the standard technique in machine learning and many other areas, for dimensionality reduction, approximate near neighbor search, compressed…
On the Last Iterate Convergence of Momentum Methods
Xiaoyu Li, Mingrui Liu, Francesco Orabona
SGD with Momentum (SGDM) is a widely used family of algorithms for large-scale optimization of machine learning problems. Yet, when optimizing generic convex functions, no advantag…
FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching
Farzin Haddadpour, Belhal Karimi, Ping Li +1
Communication complexity and privacy are the two key challenges in Federated Learning where the goal is to perform a distributed learning through a large volume of devices. In this…
A High Probability Analysis of Adaptive SGD with Momentum
Xiaoyu Li, Francesco Orabona
Stochastic Gradient Descent (SGD) and its variants are the most used algorithms in machine learning applications. In particular, SGD with adaptive learning rates and momentum is th…
IVFS: Simple and Efficient Feature Selection for High Dimensional Topology Preservation
Xiaoyun Li, Chengxi Wu, Ping Li
Feature selection is an important tool to deal with high dimensional data. In unsupervised case, many popular algorithms aim at maintaining the structure of the original data. In t…
Randomized Kernel Multi-view Discriminant Analysis
Xiaoyun Li, Jie Gui, Ping Li
In many artificial intelligence and computer vision systems, the same object can be observed at distinct viewpoints or by diverse sensors, which raises the challenges for recognizi…
A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance
Xiaoyu Li, Zhenxun Zhuang, Francesco Orabona
Stochastic Gradient Descent (SGD) is a popular tool in training large-scale machine learning models. Its performance, however, is highly variable, depending crucially on the choice…
An Optimistic Acceleration of AMSGrad for Nonconvex Optimization
Jun-Kun Wang, Xiaoyun Li, Belhal Karimi +1
We propose a new variant of AMSGrad, a popular adaptive gradient based optimization algorithm widely used for training deep neural networks. Our algorithm adds prior knowledge abou…
On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes
Xiaoyu Li, Francesco Orabona
Stochastic gradient descent is the method of choice for large scale optimization of machine learning objective functions. Yet, its performance is greatly variable and heavily depen…