42 citations · 182 across the 23 of their papers we have counts for
23 papers
Temporal Contrastive Learning for Spiking Neural Networks
Haonan Qiu, Zeyin Song, Yanqi Chen +6
Biologically inspired spiking neural networks (SNNs) have garnered considerable attention due to their low-energy consumption and spatio-temporal information processing capabilitie…
Efficient Uncertainty Estimation in Spiking Neural Networks via MC-dropout
Tao Sun, Bojian Yin, Sander Bohte
Spiking neural networks (SNNs) have gained attention as models of sparse and event-driven communication of biological neurons, and as such have shown increasing promise for energy-…
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient
Xiaoge Deng, Dongsheng Li, Tao Sun +1
Gradient-based optimization methods implemented on distributed computing architectures are increasingly used to tackle large-scale machine learning applications. A key bottleneck i…
Decentralized Federated Averaging
Tao Sun, Dongsheng Li, Bao Wang
Federated averaging (FedAvg) is a communication efficient algorithm for the distributed training with an enormous number of clients. In FedAvg, clients keep their data locally for…
Inertial Proximal Deep Learning Alternating Minimization for Efficient Neutral Network Training
Linbo Qiao, Tao Sun, Hengyue Pan +1
In recent years, the Deep Learning Alternating Minimization (DLAM), which is actually the alternating minimization applied to the penalty form of the deep neutral networks training…
Adaptive Temporal Difference Learning with Linear Function Approximation
Tao Sun, Han Shen, Tianyi Chen +1
This paper revisits the temporal difference (TD) learning algorithm for the policy evaluation tasks in reinforcement learning. Typically, the performance of TD(0) and TD() is ve…
General Proximal Incremental Aggregated Gradient Algorithms: Better and Novel Results under General Scheme
Tao Sun, Yuejiao Sun, Dongsheng Li +1
The incremental aggregated gradient algorithm is popular in network optimization and machine learning research. However, the current convergence results require the objective funct…
Decentralized Markov Chain Gradient Descent
Tao Sun, Dongsheng Li
Decentralized stochastic gradient method emerges as a promising solution for solving large-scale machine learning problems. This paper studies the decentralized Markov chain gradie…
Inertial nonconvex alternating minimizations for the image deblurring
Tao Sun, Roberto Barrio, Marcos Rodriguez +1
In image processing, Total Variation (TV) regularization models are commonly used to recover blurred images. One of the most efficient and popular methods to solve the convex TV pr…
Heavy-ball Algorithms Always Escape Saddle Points
Tao Sun, Dongsheng Li, Zhe Quan +3
Nonconvex optimization algorithms with random initialization have attracted increasing attention recently. It has been showed that many first-order methods always avoid saddle poin…
Iteratively reweighted penalty alternating minimization methods with continuation for image deblurring
Tao Sun, Dongsheng Li, Hao Jiang +1
In this paper, we consider a class of nonconvex problems with linear constraints appearing frequently in the area of image processing. We solve this problem by the penalty method a…
Markov Chain Block Coordinate Descent
Tao Sun, Yuejiao Sun, Yangyang Xu +1
The method of block coordinate gradient descent (BCD) has been a powerful method for large-scale optimization. This paper considers the BCD method that successively updates a serie…
Non-ergodic Convergence Analysis of Heavy-Ball Algorithms
Tao Sun, Penghang Yin, Dongsheng Li +3
In this paper, we revisit the convergence of the Heavy-ball method, and present improved convergence complexity results in the convex setting. We provide the first non-ergodic O(1/…
On Markov Chain Gradient Descent
Tao Sun, Yuejiao Sun, Wotao Yin
Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and enginee…
An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines
Lei Guan, Linbo Qiao, Dongsheng Li +3
Support vector machines (SVMs) with sparsity-inducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable sel…
LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning
Tianyi Chen, Georgios B. Giannakis, Tao Sun +1
This paper presents a new class of gradient methods for distributed machine learning that adaptively skip the gradient calculations to learn with reduced communication and computat…
Non-ergodic Complexity of Convex Proximal Inertial Gradient Descents
Tao Sun, Linbo Qiao, Dongsheng Li
The proximal inertial gradient descent is efficient for the composite minimization and applicable for broad of machine learning problems. In this paper, we revisit the computationa…
A convergence framework for inexact nonconvex and nonsmooth algorithms and its applications to several iterations
Tao Sun, Hao Jiang, Lizhi Cheng +1
In this paper, we consider the convergence of an abstract inexact nonconvex and nonsmooth algorithm. We promise a pseudo sufficient descent condition and a pseudo relative error co…
Iteratively Linearized Reweighted Alternating Direction Method of Multipliers for a Class of Nonconvex Problems
Tao Sun, Hao Jiang, Lizhi Cheng +1
In this paper, we consider solving a class of nonconvex and nonsmooth problems frequently appearing in signal processing and machine learning research. The traditional alternating…
Asynchronous Coordinate Descent under More Realistic Assumptions
Tao Sun, Robert Hannah, Wotao Yin
Asynchronous-parallel algorithms have the potential to vastly speed up algorithms by eliminating costly synchronization. However, our understanding to these algorithms is limited b…
A note on alternating minimization algorithms: Bregman frame
Tao Sun, Lizhi Cheng
In this paper, we propose a Bregman frame for several classical alternating minimization algorithms. In the frame, these algorithms have uniform mathematical formulation. We also p…
A note on the convergence of nonconvex line search
Tao Sun, Lizhi Chenga, Hao Jiang
In this note, we consider the line search for a class of abstract nonconvex algorithm which have been deeply studied in the Kurdyka-Lojasiewicz theory. We provide a weak convergenc…
Proximal linearized iteratively reweighted least squares for a class of nonconvex and nonsmooth problems
Hui Zhang, Tao Sun, Lizhi Cheng
For solving a wide class of nonconvex and nonsmooth problems, we propose a proximal linearized iteratively reweighted least squares (PL-IRLS) algorithm. We first approximate the or…