11 citations · 16 across the 13 of their papers we have counts for
13 papers
Understanding Neural Network Binarization with Forward and Backward Proximal Quantizers
Yiwei Lu, Yaoliang Yu, Xinlin Li +1
In neural network binarization, BinaryConnect (BC) and its variants are considered the standard. These methods apply the sign function in their forward pass and their respective gr…
BinaryViT: Pushing Binary Vision Transformers Towards Convolutional Models
Phuoc-Hoan Charles Le, Xinlin Li
With the increasing popularity and the increasing size of vision transformers (ViTs), there has been an increasing interest in making them more efficient and less computationally c…
Common Information Dimension
Osama Hanna, Xinlin Li, Suhas Diggavi +1
The exact common information between a set of random variables is defined as the minimum entropy of a shared random variable that allows for the exact distributive si…
Mathematical Challenges in Deep Learning
Vahid Partovi Nia, Guojun Zhang, Ivan Kobyzev +8
Deep models are dominating the artificial intelligence (AI) industry since the ImageNet challenge in 2012. The size of deep models is increasing ever since, which brings new challe…
EuclidNets: An Alternative Operation for Efficient Inference of Deep Learning Models
Xinlin Li, Mariana Parazeres, Adam Oberman +3
With the advent of deep learning application on edge devices, researchers actively try to optimize their deployments on low-power and restricted memory devices. There are establish…
DenseShift: Towards Accurate and Efficient Low-Bit Power-of-Two Quantization
Xinlin Li, Bang Liu, Rui Heng Yang +3
Efficiently deploying deep neural networks on low-resource edge devices is challenging due to their ever-increasing resource requirements. To address this issue, researchers have p…
Low-bit Shift Network for End-to-End Spoken Language Understanding
Anderson R. Avila, Khalil Bibi, Rui Heng Yang +3
Deep neural networks (DNN) have achieved impressive success in multiple domains. Over the years, the accuracy of these models has increased with the proliferation of deeper and mor…
Deep Neural Networks pruning via the Structured Perspective Regularization
Matteo Cacciola, Antonio Frangioni, Xinlin Li +1
In Machine Learning, Artificial Neural Networks (ANNs) are a very powerful tool, broadly used in many applications. Often, the selected (deep) architectures include many layers, an…
: Sign-Sparse-Shift Reparametrization for Effective Training of Low-bit Shift Networks
Xinlin Li, Bang Liu, Yaoliang Yu +3
Shift neural networks reduce computation complexity by removing expensive multiplication operations and quantizing continuous weights into low-bit discrete values, which are fast a…
Tensor train decompositions on recurrent networks
Alejandro Murua, Ramchalam Ramakrishnan, Xinlin Li +2
Recurrent neural networks (RNN) such as long-short-term memory (LSTM) networks are essential in a multitude of daily live tasks such as speech, language, video, and multimodal lear…
A Causal Direction Test for Heterogeneous Populations
Vahid Partovi Nia, Xinlin Li, Masoud Asgharian +3
A probabilistic expert system emulates the decision-making ability of a human expert through a directional graphical model. The first step in building such systems is to understand…
Importance of Data Loading Pipeline in Training Deep Neural Networks
Mahdi Zolnouri, Xinlin Li, Vahid Partovi Nia
Training large-scale deep neural networks is a long, time-consuming operation, often requiring many GPUs to accelerate. In large models, the time spent loading data takes a signifi…
Random Bias Initialization Improves Quantized Training
Xinlin Li, Vahid Partovi Nia
Binary neural networks improve computationally efficiency of deep models with a large margin. However, there is still a performance gap between a successful full-precision training…