60 citations · 74 across the 8 of their papers we have counts for
8 papers
VideoSEMA: a scalable and efficient Mamba-like attention for video understanding
Nhat Thanh Tran, Fanghui Xue, Shuai Zhang +4
We present for video understanding (classification) a split space-time attention model, VideoSEMA, consisting of a scalable and efficient Mamba-like attention (SEMA) block in space…
SEMA: a Scalable and Efficient Mamba like Attention via Token Localization and Averaging
Nhat Thanh Tran, Fanghui Xue, Shuai Zhang +4
Attention is the critical component of a transformer. Yet the quadratic computational complexity of vanilla full attention in the input size and the inability of its linear attenti…
AFIDAF: Alternating Fourier and Image Domain Adaptive Filters as an Efficient Alternative to Attention in ViTs
Yunling Zheng, Zeyi Xu, Fanghui Xue +5
We propose and demonstrate an alternating Fourier and image domain filtering approach for feature extraction as an efficient alternative to build a vision backbone without using th…
Improving Network Slimming with Nonconvex Regularization
Kevin Bui, Fredrick Park, Shuai Zhang +2
Convolutional neural networks (CNNs) have developed to become powerful models for various computer vision tasks ranging from object detection to semantic segmentation. However, mos…
TRP: Trained Rank Pruning for Efficient Deep Neural Networks
Yuhui Xu, Yuxi Li, Shuai Zhang +6
To enable DNNs on edge devices like mobile phones, low-rank approximation has been widely adopted because of its solid theoretical rationale and efficient implementations. Several…
Regularized Structured Sparsity Convolutional Neural Networks
Kevin Bui, Fredrick Park, Shuai Zhang +2
Deepening and widening convolutional neural networks (CNNs) significantly increases the number of trainable weight parameters by adding more convolutional layers and feature maps p…
Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets
Penghang Yin, Jiancheng Lyu, Shuai Zhang +3
Training activation quantized neural networks involves minimizing a piecewise constant function whose gradient vanishes almost everywhere, which is undesirable for the standard bac…
BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights
Penghang Yin, Shuai Zhang, Jiancheng Lyu +3
We propose BinaryRelax, a simple two-phase algorithm, for training deep neural networks with quantized weights. The set constraint that characterizes the quantization of weights is…