26 citations · 97 across the 29 of their papers we have counts for
17 papers · 1 filter
Mathematical Challenges in Deep Learning
Vahid Partovi Nia, Guojun Zhang, Ivan Kobyzev +8
Deep models are dominating the artificial intelligence (AI) industry since the ImageNet challenge in 2012. The size of deep models is increasing ever since, which brings new challe…
On the Convergence of Stochastic Gradient Descent in Low-precision Number Formats
Matteo Cacciola, Antonio Frangioni, Masoud Asgharian +2
Deep learning models are dominating almost all artificial intelligence tasks such as vision, text, and speech processing. Stochastic Gradient Descent (SGD) is the main tool for tra…
EuclidNets: An Alternative Operation for Efficient Inference of Deep Learning Models
Xinlin Li, Mariana Parazeres, Adam Oberman +3
With the advent of deep learning application on edge devices, researchers actively try to optimize their deployments on low-power and restricted memory devices. There are establish…
Training Integer-Only Deep Recurrent Neural Networks
Vahid Partovi Nia, Eyyüb Sari, Vanessa Courville +1
Recurrent neural networks (RNN) are the backbone of many text and speech applications. These architectures are typically made up of several computationally complex components such…
Is Integer Arithmetic Enough for Deep Learning Training?
Alireza Ghaffari, Marzieh S. Tahaei, Mohammadreza Tayaranian +2
The ever-increasing computational complexity of deep learning models makes their training and deployment difficult on various cloud and edge platforms. Replacing floating-point ari…
Rethinking Pareto Frontier for Performance Evaluation of Deep Neural Networks
Vahid Partovi Nia, Alireza Ghaffari, Mahdi Zolnouri +1
Performance optimization of deep learning models is conducted either manually or through automatic architecture search, or a combination of both. On the other hand, their performan…
Demystifying and Generalizing BinaryConnect
Tim Dockhorn, Yaoliang Yu, Eyyüb Sari +2
BinaryConnect (BC) and its many variations have become the de facto standard for neural network quantization. However, our understanding of the inner workings of BC is still quite…
iRNN: Integer-only Recurrent Neural Network
Eyyüb Sari, Vanessa Courville, Vahid Partovi Nia
Recurrent neural networks (RNN) are used in many real-world text and speech applications. They include complex modules such as recurrence, exponential-based activation, gate intera…
Tensor train decompositions on recurrent networks
Alejandro Murua, Ramchalam Ramakrishnan, Xinlin Li +2
Recurrent neural networks (RNN) such as long-short-term memory (LSTM) networks are essential in a multitude of daily live tasks such as speech, language, video, and multimodal lear…
Batch Normalization in Quantized Networks
Eyyüb Sari, Vahid Partovi Nia
Implementation of quantized neural networks on computing hardware leads to considerable speed up and memory saving. However, quantized deep networks are difficult to train and batc…
Random Bias Initialization Improves Quantized Training
Xinlin Li, Vahid Partovi Nia
Binary neural networks improve computationally efficiency of deep models with a large margin. However, there is still a performance gap between a successful full-precision training…
Adaptive Binary-Ternary Quantization
Ryan Razani, Grégoire Morin, Vahid Partovi Nia +1
Neural network models are resource hungry. It is difficult to deploy such deep networks on devices with limited resources, like smart wearables, cellphones, drones, and autonomous…
How Does Batch Normalization Help Binary Training?
Eyyüb Sari, Mouloud Belbahri, Vahid Partovi Nia
Binary Neural Networks (BNNs) are difficult to train, and suffer from drop of accuracy. It appears in practice that BNNs fail to train in the absence of Batch Normalization (BatchN…
Differentiable Mask for Pruning Convolutional and Recurrent Networks
Ramchalam Kinattinkara Ramakrishnan, Eyyüb Sari, Vahid Partovi Nia
Pruning is one of the most effective model reduction techniques. Deep networks require massive computation and such models need to be compressed to bring them on edge devices. Most…
Active Learning for High-Dimensional Binary Features
Ali Vahdat, Mouloud Belbahri, Vahid Partovi Nia
Erbium-doped fiber amplifier (EDFA) is an optical amplifier/repeater device used to boost the intensity of optical signals being carried through a fiber optic communication system.…
Activation Adaptation in Neural Networks
Farnoush Farhadi, Vahid Partovi Nia, Andrea Lodi
Many neural network architectures rely on the choice of the activation function for each hidden layer. Given the activation function, the neural network is trained over the bias an…
Regularized Binary Network Training
Sajad Darabi, Mouloud Belbahri, Matthieu Courbariaux +1
There is a significant performance gap between Binary Neural Networks (BNNs) and floating point Deep Neural Networks (DNNs). We propose to improve the binary training method, by in…