12 citations · 25 across the 15 of their papers we have counts for
8 papers · 1 filter
Lookahead Drifting Model
Guoqiang Zhang, Kenta Niwa, W. Bastiaan Kleijn
Recently, a new paradigm named \emph{drifting model} has been proposed for mapping distributions, which achieves the SOTA image generation performance over ImageNet via one-step ne…
On Accelerating Diffusion-Based Sampling Process via Improved Integration Approximation
Guoqiang Zhang, Niwa Kenta, W. Bastiaan Kleijn
A popular approach to sample a diffusion-based generative model is to solve an ordinary differential equation (ODE). In existing samplers, the coefficients of the ODE solvers are p…
A DNN Optimizer that Improves over AdaBelief by Suppression of the Adaptive Stepsize Range
Guoqiang Zhang, Kenta Niwa, W. Bastiaan Kleijn
We make contributions towards improving adaptive-optimizer performance. Our improvements are based on suppression of the range of adaptive stepsizes in the AdaBelief optimizer. Fir…
Extending AdamW by Leveraging Its Second Moment and Magnitude
Guoqiang Zhang, Niwa Kenta, W. Bastiaan Kleijn
Recent work [4] analyses the local convergence of Adam in a neighbourhood of an optimal solution for a twice-differentiable function. It is found that the learning rate has to be s…
DS-UI: Dual-Supervised Mixture of Gaussian Mixture Models for Uncertainty Inference
Jiyang Xie, Zhanyu Ma, Jing-Hao Xue +2
This paper proposes a dual-supervised uncertainty inference (DS-UI) framework for improving Bayesian estimation-based uncertainty inference (UI) in deep neural network (DNN)-based…
Approximated Orthonormal Normalisation in Training Neural Networks
Guoqiang Zhang, Kenta Niwa, W. B. Kleijn
Generalisation of a deep neural network (DNN) is one major concern when employing the deep learning approach for solving practical problems. In this paper we propose a new techniqu…
Rapidly Adapting Moment Estimation
Guoqiang Zhang, Kenta Niwa, W. Bastiaan Kleijn
Adaptive gradient methods such as Adam have been shown to be very effective for training deep neural networks (DNNs) by tracking the second moment of gradients to compute the indiv…
Training Deep Neural Networks via Optimization Over Graphs
Guoqiang Zhang, W. Bastiaan Kleijn
In this work, we propose to train a deep neural network by distributed optimization over a graph. Two nonlinear functions are considered: the rectified linear unit (ReLU) and a lin…