211 citations · 847 across the 45 of their papers we have counts for
24 papers · 1 filter
Advancing time series completion via RFAMoE and MDFF
Ci Zhang, Huayu Li, Changdi Yang +6
Recent studies show that using diffusion models for time series signal reconstruction holds great promise. However, such approaches remain largely unexplored in the domain of medic…
Dynamic Sparsity Is Channel-Level Sparsity Learner
Lu Yin, Gen Li, Meng Fang +7
Sparse training has received an upsurging interest in machine learning due to its tantalizing saving potential for the entire training process as well as inference. Dynamic sparse…
VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer
Mengshu Sun, Haoyu Ma, Guoliang Kang +5
The transformer architectures with attention mechanisms have obtained success in Nature Language Processing (NLP), and Vision Transformers (ViTs) have recently extended the applica…
Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets
Tianlong Chen, Xuxi Chen, Xiaolong Ma +2
The lottery ticket hypothesis (LTH) has shown that dense models contain highly sparse subnetworks (i.e., winning tickets) that can be trained in isolation to match full accuracy. D…
Load-balanced Gather-scatter Patterns for Sparse Deep Neural Networks
Fei Sun, Minghai Qin, Tianyun Zhang +6
Deep neural networks (DNNs) have been proven to be effective in solving many real-life problems, but its high computation cost prohibits those models from being deployed to edge de…
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
Geng Yuan, Xiaolong Ma, Wei Niu +13
Recently, a new trend of exploring sparsity for accelerating neural network training has emerged, embracing the paradigm of training on the edge. This paper proposes a novel Memory…
GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity
Wei Niu, Zhengang Li, Xiaolong Ma +6
It is appealing but challenging to achieve real-time deep neural network (DNN) inference on mobile devices because even the powerful modern mobile devices are considered as ``resou…
A Topological-Framework to Improve Analysis of Machine Learning Model Performance
Henry Kvinge, Colby Wight, Sarah Akers +7
As both machine learning models and the datasets on which they are evaluated have grown in size and complexity, the practice of using a few summary statistics to understand model p…
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
Xiaolong Ma, Geng Yuan, Xuan Shen +8
There have been long-standing controversies and inconsistencies over the experiment setup and criteria for identifying the "winning ticket" in literature. To reconcile such, we rev…
Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI
Geng Yuan, Zhiheng Liao, Xiaolong Ma +11
Recent research demonstrated the promise of using resistive random access memory (ReRAM) as an emerging technology to perform inherently parallel analog domain in-situ matrix-vecto…
Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not?
Ning Liu, Geng Yuan, Zhengping Che +7
In deep model compression, the recent finding "Lottery Ticket Hypothesis" (LTH) (Frankle & Carbin, 2018) pointed out that there could exist a winning ticket (i.e., a properly prune…
A Unified DNN Weight Compression Framework Using Reweighted Optimization Methods
Tianyun Zhang, Xiaolong Ma, Zheng Zhan +7
To address the large model size and intensive computation requirement of deep neural networks (DNNs), weight pruning techniques have been proposed and generally fall into two categ…
A Privacy-Preserving-Oriented DNN Pruning and Mobile Acceleration Framework
Yifan Gong, Zheng Zhan, Zhengang Li +8
Weight pruning of deep neural networks (DNNs) has been proposed to satisfy the limited storage and computing capability of mobile edge devices. However, previous pruning methods ma…
BLK-REW: A Unified Block-based DNN Pruning Framework using Reweighted Regularization Method
Xiaolong Ma, Zhengang Li, Yifan Gong +8
Accelerating DNN execution on various resource-limited computing platforms has been a long-standing problem. Prior works utilize l1-based group lasso or dynamic regularization such…
PCNN: Pattern-based Fine-Grained Regular Pruning towards Optimizing CNN Accelerators
Zhanhong Tan, Jiebo Song, Xiaolong Ma +8
Weight pruning is a powerful technique to realize model compression. We propose PCNN, a fine-grained regular 1D pruning method. A novel index format called Sparsity Pattern Mask (S…
SS-Auto: A Single-Shot, Automatic Structured Weight Pruning Framework of DNNs with Ultra-High Efficiency
Zhengang Li, Yifan Gong, Xiaolong Ma +6
Structured weight pruning is a representative model compression technique of DNNs for hardware efficiency and inference accelerations. Previous works in this area leave great space…
PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning
Wei Niu, Xiaolong Ma, Sheng Lin +5
With the emergence of a spectrum of high-end mobile devices, many applications that formerly required desktop-level computation capability are being transferred to these devices. H…
PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devices
Xiaolong Ma, Fu-Ming Guo, Wei Niu +5
Model compression techniques on Deep Neural Network (DNN) have been widely acknowledged as an effective way to achieve acceleration on a variety of platforms, and DNN weight prunin…
AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates
Ning Liu, Xiaolong Ma, Zhiyuan Xu +3
Structured weight pruning is a representative model compression technique of DNNs to reduce the storage and computation requirements and accelerate inference. An automatic hyperpar…
Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?
Xiaolong Ma, Sheng Lin, Shaokai Ye +10
Large deep neural network (DNN) models pose the key challenge to energy efficiency due to the significantly higher energy consumption of off-chip DRAM accesses than arithmetic or S…
Toward Extremely Low Bit and Lossless Accuracy in DNNs with Progressive ADMM
Sheng Lin, Xiaolong Ma, Shaokai Ye +3
Weight quantization is one of the most important techniques of Deep Neural Networks (DNNs) model compression method. A recent work using systematic framework of DNN weight quantiza…
26ms Inference Time for ResNet-50: Towards Real-Time Execution of all DNNs on Smartphone
Wei Niu, Xiaolong Ma, Yanzhi Wang +1
With the rapid emergence of a spectrum of high-end mobile devices, many applications that required desktop-level computation capability formerly can now run on these devices withou…
ResNet Can Be Pruned 60x: Introducing Network Purification and Unused Path Removal (P-RM) after Weight Pruning
Xiaolong Ma, Geng Yuan, Sheng Lin +3
The state-of-art DNN structures involve high computation and great demand for memory storage which pose intensive challenge on DNN framework resources. To mitigate the challenges,…
Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework
Yanzhi Wang, Caiwen Ding, Zhe Li +8
Hardware accelerations of deep learning systems have been extensively investigated in industry and academia. The aim of this paper is to achieve ultra-high energy efficiency and pe…