257 citations · 385 across the 22 of their papers we have counts for
9 papers · 1 filter
SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezing
Sheng Li, Geng Yuan, Yue Dai +3
There has been a proliferation of artificial intelligence applications, where model training is key to promising high-quality services for these applications. However, the model tr…
DualHSIC: HSIC-Bottleneck and Alignment for Continual Learning
Zifeng Wang, Zheng Zhan, Yifan Gong +4
Rehearsal-based approaches are a mainstay of continual learning (CL). They mitigate the catastrophic forgetting problem by maintaining a small fixed-size buffer with a subset of da…
All-in-One: A Highly Representative DNN Pruning Framework for Edge Devices with Dynamic Power Management
Yifan Gong, Zheng Zhan, Pu Zhao +6
During the deployment of deep neural networks (DNNs) on edge devices, many research efforts are devoted to the limited hardware resource. However, little attention is paid to the i…
Self-Ensemble Protection: Training Checkpoints Are Good Data Protectors
Sizhe Chen, Geng Yuan, Xinwen Cheng +4
As data becomes increasingly vital, a company would be very cautious about releasing data, because the competitors could use it to train high-performance models, thereby posing a t…
Pruning Adversarially Robust Neural Networks without Adversarial Examples
Tong Jian, Zifeng Wang, Yanzhi Wang +2
Adversarial pruning compresses models while preserving robustness. Current methods require access to adversarial examples during pruning. This significantly hampers training effici…
Advancing Model Pruning via Bi-level Optimization
Yihua Zhang, Yuguang Yao, Parikshit Ram +5
The deployment constraints in practical applications necessitate the pruning of large-scale deep learning models, i.e., promoting their weight sparsity. As illustrated by the Lotte…
Layer Freezing & Data Sieving: Missing Pieces of a Generic Framework for Sparse Training
Geng Yuan, Yanyu Li, Sheng Li +5
Recently, sparse training has emerged as a promising paradigm for efficient deep learning on edge devices. The current research mainly devotes efforts to reducing training costs by…
SparCL: Sparse Continual Learning on the Edge
Zifeng Wang, Zheng Zhan, Yifan Gong +7
Existing work in continual learning (CL) focuses on mitigating catastrophic forgetting, i.e., model performance deterioration on past tasks when learning a new task. However, the t…
PIM-QAT: Neural Network Quantization for Processing-In-Memory (PIM) Systems
Qing Jin, Zhiyu Chen, Jian Ren +3
Processing-in-memory (PIM), an increasingly studied neuromorphic hardware, promises orders of energy and throughput improvements for deep learning inference. Leveraging the massive…