20 citations · 47 across the 10 of their papers we have counts for
10 papers
DIFF2: Differential Private Optimization via Gradient Differences for Nonconvex Distributed Learning
Tomoya Murata, Taiji Suzuki
Differential private optimization for nonconvex smooth objective is considered. In the previous work, the best known utility bound is $\widetilde O(\sqrt{d}/(n\varepsilon_\mathrm{D…
Versatile Single-Loop Method for Gradient Estimator: First and Second Order Optimality, and its Application to Federated Learning
Kazusato Oko, Shunta Akiyama, Tomoya Murata +1
While variance reduction methods have shown great success in solving large scale optimization problems, many of them suffer from accumulated errors and, therefore, should periodica…
Escaping Saddle Points with Bias-Variance Reduced Local Perturbed SGD for Communication Efficient Nonconvex Distributed Learning
Tomoya Murata, Taiji Suzuki
In recent centralized nonconvex distributed learning and federated learning, local methods are one of the promising approaches to reduce communication time. However, existing work…
Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning
Tomoya Murata, Taiji Suzuki
Recently, local SGD has got much attention and been extensively studied in the distributed learning community to overcome the communication bottleneck problem. However, the superio…
Gradient Descent in RKHS with Importance Labeling
Tomoya Murata, Taiji Suzuki
Labeling cost is often expensive and is a fundamental limitation of supervised learning. In this paper, we study importance labeling problem, in which we are given many unlabeled d…
Accelerated Sparsified SGD with Error Feedback
Tomoya Murata, Taiji Suzuki
A stochastic gradient method for synchronous distributed optimization is studied. For reducing communication cost, we particularly focus on utilization of compression of communicat…
Sample Efficient Stochastic Gradient Iterative Hard Thresholding Method for Stochastic Sparse Linear Regression with Limited Attribute Observation
Tomoya Murata, Taiji Suzuki
We develop new stochastic gradient methods for efficiently solving sparse linear regression in a partial attribute observation setting, where learners are only allowed to observe a…
Spectral Pruning: Compressing Deep Neural Networks via Spectral Analysis and its Generalization Error
Taiji Suzuki, Hiroshi Abe, Tomoya Murata +6
Compression techniques for deep neural network models are becoming very important for the efficient execution of high-performance deep learning systems on edge-computing devices. T…
Doubly Accelerated Stochastic Variance Reduced Dual Averaging Method for Regularized Empirical Risk Minimization
Tomoya Murata, Taiji Suzuki
In this paper, we develop a new accelerated stochastic gradient method for efficiently solving the convex regularized empirical risk minimization problem in mini-batch settings. Th…
Stochastic dual averaging methods using variance reduction techniques for regularized empirical risk minimization problems
Tomoya Murata, Taiji Suzuki
We consider a composite convex minimization problem associated with regularized empirical risk minimization, which often arises in machine learning. We propose two new stochastic g…