143 citations · 390 across the 17 of their papers we have counts for
8 papers · 1 filter
Knowledge distillation for optimization of quantized deep neural networks
Sungho Shin, Yoonho Boo, Wonyong Sung
Knowledge distillation (KD) is a very popular method for model size reduction. Recently, the technique is exploited for quantized deep neural networks (QDNNs) training as a way to…
Fixed-point optimization of deep neural networks with adaptive step size retraining
Sungho Shin, Yoonho Boo, Wonyong Sung
Fixed-point optimization of deep neural networks plays an important role in hardware based design and low-power implementations. Many deep neural networks show fairly good performa…
Quantized neural network design under weight capacity constraint
Sungho Shin, Kyuyeon Hwang, Wonyong Sung
The complexity of deep neural network algorithms for hardware implementation can be lowered either by scaling the number of units or reducing the word-length of weights. Both appro…
Compact Deep Convolutional Neural Networks With Coarse Pruning
Sajid Anwar, Wonyong Sung
The learning capability of a neural network improves with increasing depth at higher computational costs. Wider layers with dense kernel connectivity patterns furhter increase this…
Character-Level Language Modeling with Hierarchical Recurrent Neural Networks
Kyuyeon Hwang, Wonyong Sung
Recurrent neural network (RNN) based character-level language models (CLMs) are extremely useful for modeling out-of-vocabulary words by nature. However, their performance is gener…
Fixed-Point Performance Analysis of Recurrent Neural Networks
Sungho Shin, Kyuyeon Hwang, Wonyong Sung
Recurrent neural networks have shown excellent performance in many applications, however they require increased complexity in hardware or software based implementations. The hardwa…
Online Sequence Training of Recurrent Neural Networks with Connectionist Temporal Classification
Kyuyeon Hwang, Wonyong Sung
Connectionist temporal classification (CTC) based supervised sequence training of recurrent neural networks (RNNs) has shown great success in many machine learning areas including…
Resiliency of Deep Neural Networks under Quantization
Wonyong Sung, Sungho Shin, Kyuyeon Hwang
The complexity of deep neural network algorithms for hardware implementation can be much lowered by optimizing the word-length of weights and signals. Direct quantization of floati…