143 citations · 429 across the 41 of their papers we have counts for
11 papers · 1 filter
Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks
Yoonho Boo, Sungho Shin, Jungwook Choi +1
The quantization of deep neural networks (QDNNs) has been actively studied for deployment in edge devices. Recent studies employ the knowledge distillation (KD) method to improve t…
Multiple Classification with Split Learning
Jongwon Kim, Sungho Shin, Yeonguk Yu +2
Privacy issues were raised in the process of training deep learning in medical, mobility, and other fields. To solve this problem, we present privacy-preserving distributed deep le…
S-SGD: Symmetrical Stochastic Gradient Descent with Weight Noise Injection for Reaching Flat Minima
Wonyong Sung, Iksoo Choi, Jinhwan Park +2
The stochastic gradient descent (SGD) method is most widely used for deep neural network (DNN) training. However, the method does not always converge to a flat minimum of the loss…
Quantized Neural Networks: Characterization and Holistic Optimization
Yoonho Boo, Sungho Shin, Wonyong Sung
Quantized deep neural networks (QDNNs) are necessary for low-power, high throughput, and embedded applications. Previous studies mostly focused on developing optimization methods f…
SQWA: Stochastic Quantized Weight Averaging for Improving the Generalization Capability of Low-Precision Deep Neural Networks
Sungho Shin, Yoonho Boo, Wonyong Sung
Designing a deep neural network (DNN) with good generalization capability is a complex process especially when the weights are severely quantized. Model averaging is a promising ap…
Knowledge distillation for optimization of quantized deep neural networks
Sungho Shin, Yoonho Boo, Wonyong Sung
Knowledge distillation (KD) is a very popular method for model size reduction. Recently, the technique is exploited for quantized deep neural networks (QDNNs) training as a way to…
Fixed-point optimization of deep neural networks with adaptive step size retraining
Sungho Shin, Yoonho Boo, Wonyong Sung
Fixed-point optimization of deep neural networks plays an important role in hardware based design and low-power implementations. Many deep neural networks show fairly good performa…
Quantized neural network design under weight capacity constraint
Sungho Shin, Kyuyeon Hwang, Wonyong Sung
The complexity of deep neural network algorithms for hardware implementation can be lowered either by scaling the number of units or reducing the word-length of weights. Both appro…
Generative Knowledge Transfer for Neural Language Models
Sungho Shin, Kyuyeon Hwang, Wonyong Sung
In this paper, we propose a generative knowledge transfer technique that trains an RNN based language model (student network) using text and output probabilities generated from a p…
Fixed-Point Performance Analysis of Recurrent Neural Networks
Sungho Shin, Kyuyeon Hwang, Wonyong Sung
Recurrent neural networks have shown excellent performance in many applications, however they require increased complexity in hardware or software based implementations. The hardwa…
Resiliency of Deep Neural Networks under Quantization
Wonyong Sung, Sungho Shin, Kyuyeon Hwang
The complexity of deep neural network algorithms for hardware implementation can be much lowered by optimizing the word-length of weights and signals. Direct quantization of floati…