72 citations · 98 across the 13 of their papers we have counts for
13 papers
High Quality Diffusion Distillation on a Single GPU with Relative and Absolute Position Matching
Guoqiang Zhang, Kenta Niwa, J. P. Lewis +2
We introduce relative and absolute position matching (RAPM), a diffusion distillation method resulting in high quality generation that can be trained efficiently on a single GPU. R…
Beyond Exponential Graph: Communication-Efficient Topologies for Decentralized Learning via Finite-time Convergence
Yuki Takezawa, Ryoma Sato, Han Bao +2
Decentralized learning has recently been attracting increasing attention for its applications in parallel computation and privacy preservation. Many recent studies stated that the…
Momentum Tracking: Momentum Acceleration for Decentralized Deep Learning on Heterogeneous Data
Yuki Takezawa, Han Bao, Kenta Niwa +2
SGD with momentum is one of the key components for improving the performance of neural networks. For decentralized learning, a straightforward approach using momentum is Distribute…
Communication Compression for Decentralized Learning with Operator Splitting Methods
Yuki Takezawa, Kenta Niwa, Makoto Yamada
In decentralized learning, operator splitting methods using a primal-dual formulation (e.g., the Edge-Consensus Learning (ECL)) has been shown to be robust to heterogeneous data an…
Theoretical Analysis of Primal-Dual Algorithm for Non-Convex Stochastic Decentralized Optimization
Yuki Takezawa, Kenta Niwa, Makoto Yamada
In recent years, decentralized learning has emerged as a powerful tool not only for large-scale machine learning, but also for preserving privacy. One of the key challenges in dece…
A DNN Optimizer that Improves over AdaBelief by Suppression of the Adaptive Stepsize Range
Guoqiang Zhang, Kenta Niwa, W. Bastiaan Kleijn
We make contributions towards improving adaptive-optimizer performance. Our improvements are based on suppression of the range of adaptive stepsizes in the AdaBelief optimizer. Fir…
Revisiting the Primal-Dual Method of Multipliers for Optimisation over Centralised Networks
Guoqiang Zhang, Kenta Niwa, W. Bastiaan Kleijn
The primal-dual method of multipliers (PDMM) was originally designed for solving a decomposable optimisation problem over a general network. In this paper, we revisit PDMM for opti…
Approximated Orthonormal Normalisation in Training Neural Networks
Guoqiang Zhang, Kenta Niwa, W. B. Kleijn
Generalisation of a deep neural network (DNN) is one major concern when employing the deep learning approach for solving practical problems. In this paper we propose a new techniqu…
Rapidly Adapting Moment Estimation
Guoqiang Zhang, Kenta Niwa, W. Bastiaan Kleijn
Adaptive gradient methods such as Adam have been shown to be very effective for training deep neural networks (DNNs) by tracking the second moment of gradients to compute the indiv…
DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score
Yuma Koizumi, Kenta Niwa, Yusuke Hioka +2
We propose a training method for deep neural network (DNN)-based source enhancement to increase objective sound quality assessment (OSQA) scores such as the perceptual evaluation o…
Bregman Monotone Operator Splitting
Kenta Niwa, W. Bastiaan Kleijn
Monotone operator splitting is a powerful paradigm that facilitates parallel processing for optimization problems where the cost function can be split into two convex functions. We…
Software Defined Media: Virtualization of Audio-Visual Services
Manabu Tsukada, Keiko Ogawa, Masahiro Ikeda +6
Internet-native audio-visual services are witnessing rapid development. Among these services, object-based audio-visual services are gaining importance. In 2014, we established the…
PSD estimation in Beamspace for Estimating Direct-to-Reverberant Ratio from A Reverberant Speech Signal
Yusuke Hioka, Kenta Niwa
A method for estimation of direct-to-reverberant ratio (DRR) using a microphone array is proposed. The proposed method estimates the power spectral density (PSD) of the direct soun…