8 citations · 20 across the 13 of their papers we have counts for
6 papers · 1 filter
Macro-block dropout for improved regularization in training end-to-end speech recognition models
Chanwoo Kim, Sathish Indurti, Jinhwan Park +1
This paper proposes a new regularization algorithm referred to as macro-block dropout. The overfitting issue has been a difficult problem in training large neural network models. T…
Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks
Yoonho Boo, Sungho Shin, Jungwook Choi +1
The quantization of deep neural networks (QDNNs) has been actively studied for deployment in edge devices. Recent studies employ the knowledge distillation (KD) method to improve t…
S-SGD: Symmetrical Stochastic Gradient Descent with Weight Noise Injection for Reaching Flat Minima
Wonyong Sung, Iksoo Choi, Jinhwan Park +2
The stochastic gradient descent (SGD) method is most widely used for deep neural network (DNN) training. However, the method does not always converge to a flat minimum of the loss…
Quantized Neural Networks: Characterization and Holistic Optimization
Yoonho Boo, Sungho Shin, Wonyong Sung
Quantized deep neural networks (QDNNs) are necessary for low-power, high throughput, and embedded applications. Previous studies mostly focused on developing optimization methods f…
SQWA: Stochastic Quantized Weight Averaging for Improving the Generalization Capability of Low-Precision Deep Neural Networks
Sungho Shin, Yoonho Boo, Wonyong Sung
Designing a deep neural network (DNN) with good generalization capability is a complex process especially when the weights are severely quantized. Model averaging is a promising ap…
Generative Knowledge Transfer for Neural Language Models
Sungho Shin, Kyuyeon Hwang, Wonyong Sung
In this paper, we propose a generative knowledge transfer technique that trains an RNN based language model (student network) using text and output probabilities generated from a p…