143 citations · 390 across the 17 of their papers we have counts for
18 papers
Knowledge distillation for optimization of quantized deep neural networks
Sungho Shin, Yoonho Boo, Wonyong Sung
Knowledge distillation (KD) is a very popular method for model size reduction. Recently, the technique is exploited for quantized deep neural networks (QDNNs) training as a way to…
Workload-aware Automatic Parallelization for Multi-GPU DNN Training
Sungho Shin, Youngmin Jo, Jungwook Choi +3
Deep neural networks (DNNs) have emerged as successful solutions for variety of artificial intelligence applications, but their very large and deep models impose high computational…
Single Stream Parallelization of Recurrent Neural Networks for Low Power and Fast Inference
Wonyong Sung, Jinhwan Park
As neural network algorithms show high performance in many applications, their efficient inference on mobile and embedded systems are of great interests. When a single stream recur…
Structured Sparse Ternary Weight Coding of Deep Neural Networks for Efficient Hardware Implementations
Yoonho Boo, Wonyong Sung
Deep neural networks (DNNs) usually demand a large amount of operations for real-time inference. Especially, fully-connected layers contain a large number of weights, thus they usu…
Fixed-point optimization of deep neural networks with adaptive step size retraining
Sungho Shin, Yoonho Boo, Wonyong Sung
Fixed-point optimization of deep neural networks plays an important role in hardware based design and low-power implementations. Many deep neural networks show fairly good performa…
Quantized neural network design under weight capacity constraint
Sungho Shin, Kyuyeon Hwang, Wonyong Sung
The complexity of deep neural network algorithms for hardware implementation can be lowered either by scaling the number of units or reducing the word-length of weights. Both appro…
Compact Deep Convolutional Neural Networks With Coarse Pruning
Sajid Anwar, Wonyong Sung
The learning capability of a neural network improves with increasing depth at higher computational costs. Wider layers with dense kernel connectivity patterns furhter increase this…
FPGA-Based Low-Power Speech Recognition with Recurrent Neural Networks
Minjae Lee, Kyuyeon Hwang, Jinhwan Park +3
In this paper, a neural network based real-time speech recognition (SR) system is developed using an FPGA for very low-power operation. The implemented system employs two recurrent…
Character-Level Language Modeling with Hierarchical Recurrent Neural Networks
Kyuyeon Hwang, Wonyong Sung
Recurrent neural network (RNN) based character-level language models (CLMs) are extremely useful for modeling out-of-vocabulary words by nature. However, their performance is gener…
Dynamic Hand Gesture Recognition for Wearable Devices with Low Complexity Recurrent Neural Networks
Sungho Shin, Wonyong Sung
Gesture recognition is a very essential technology for many wearable devices. While previous algorithms are mostly based on statistical methods including the hidden Markov model, w…
FPGA Based Implementation of Deep Neural Networks Using On-chip Memory Only
Jinhwan Park, Wonyong Sung
Deep neural networks (DNNs) demand a very large amount of computation and weight storage, and thus efficient implementation using special purpose hardware is highly desired. In thi…
Character-Level Incremental Speech Recognition with Recurrent Neural Networks
Kyuyeon Hwang, Wonyong Sung
In real-time speech recognition applications, the latency is an important issue. We have developed a character-level incremental speech recognition (ISR) system that responds quick…
Online Keyword Spotting with a Character-Level Recurrent Neural Network
Kyuyeon Hwang, Minjae Lee, Wonyong Sung
In this paper, we propose a context-aware keyword spotting model employing a character-level recurrent neural network (RNN) for spoken term detection in continuous speech. The RNN…
Structured Pruning of Deep Convolutional Neural Networks
Sajid Anwar, Kyuyeon Hwang, Wonyong Sung
Real time application of deep learning algorithms is often hindered by high computational complexity and frequent memory accesses. Network pruning is a promising technique to solve…
Fixed-Point Performance Analysis of Recurrent Neural Networks
Sungho Shin, Kyuyeon Hwang, Wonyong Sung
Recurrent neural networks have shown excellent performance in many applications, however they require increased complexity in hardware or software based implementations. The hardwa…
Online Sequence Training of Recurrent Neural Networks with Connectionist Temporal Classification
Kyuyeon Hwang, Wonyong Sung
Connectionist temporal classification (CTC) based supervised sequence training of recurrent neural networks (RNNs) has shown great success in many machine learning areas including…
Resiliency of Deep Neural Networks under Quantization
Wonyong Sung, Sungho Shin, Kyuyeon Hwang
The complexity of deep neural network algorithms for hardware implementation can be much lowered by optimizing the word-length of weights and signals. Direct quantization of floati…
Single stream parallelization of generalized LSTM-like RNNs on a GPU
Kyuyeon Hwang, Wonyong Sung
Recurrent neural networks (RNNs) have shown outstanding performance on processing sequence data. However, they suffer from long training time, which demands parallel implementation…