26 citations · 97 across the 29 of their papers we have counts for
7 papers · 1 filter
Demystifying and Generalizing BinaryConnect
Tim Dockhorn, Yaoliang Yu, Eyyüb Sari +2
BinaryConnect (BC) and its many variations have become the de facto standard for neural network quantization. However, our understanding of the inner workings of BC is still quite…
Kronecker Decomposition for GPT Compression
Ali Edalati, Marzieh Tahaei, Ahmad Rashid +3
GPT is an auto-regressive Transformer-based pre-trained language model which has attracted a lot of attention in the natural language processing (NLP) domain due to its state-of-th…
KroneckerBERT: Learning Kronecker Decomposition for Pre-trained Language Models via Knowledge Distillation
Marzieh S. Tahaei, Ella Charlaix, Vahid Partovi Nia +2
The development of over-parameterized pre-trained language models has made a significant contribution toward the success of natural language processing. While over-parameterization…
Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition
Marawan Gamal Abdel Hameed, Marzieh S. Tahaei, Ali Mosleh +1
Modern Convolutional Neural Network (CNN) architectures, despite their superiority in solving various problems, are generally too large to be deployed on resource constrained edge…
iRNN: Integer-only Recurrent Neural Network
Eyyüb Sari, Vanessa Courville, Vahid Partovi Nia
Recurrent neural networks (RNN) are used in many real-world text and speech applications. They include complex modules such as recurrence, exponential-based activation, gate intera…
: Sign-Sparse-Shift Reparametrization for Effective Training of Low-bit Shift Networks
Xinlin Li, Bang Liu, Yaoliang Yu +3
Shift neural networks reduce computation complexity by removing expensive multiplication operations and quantizing continuous weights into low-bit discrete values, which are fast a…
A Twin Neural Model for Uplift
Mouloud Belbahri, Olivier Gandouet, Alejandro Murua +1
Uplift is a particular case of conditional treatment effect modeling. Such models deal with cause-and-effect inference for a specific factor, such as a marketing intervention or a…