257 citations · 385 across the 22 of their papers we have counts for
22 papers
SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezing
Sheng Li, Geng Yuan, Yue Dai +3
There has been a proliferation of artificial intelligence applications, where model training is key to promising high-quality services for these applications. However, the model tr…
SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Seconds
Yanyu Li, Huan Wang, Qing Jin +6
Text-to-image diffusion models can create stunning images from natural language descriptions that rival the work of professional artists and photographers. However, these models ar…
DualHSIC: HSIC-Bottleneck and Alignment for Continual Learning
Zifeng Wang, Zheng Zhan, Yifan Gong +4
Rehearsal-based approaches are a mainstay of continual learning (CL). They mitigate the catastrophic forgetting problem by maintaining a small fixed-size buffer with a subset of da…
Can Adversarial Examples Be Parsed to Reveal Victim Model Information?
Yuguang Yao, Jiancheng Liu, Yifan Gong +4
Numerous adversarial attack methods have been developed to generate imperceptible image perturbations that can cause erroneous predictions of state-of-the-art machine learning (ML)…
All-in-One: A Highly Representative DNN Pruning Framework for Edge Devices with Dynamic Power Management
Yifan Gong, Zheng Zhan, Pu Zhao +6
During the deployment of deep neural networks (DNNs) on edge devices, many research efforts are devoted to the limited hardware resource. However, little attention is paid to the i…
Rethinking Vision Transformers for MobileNet Size and Speed
Yanyu Li, Ju Hu, Yang Wen +5
With the success of Vision Transformers (ViTs) in computer vision tasks, recent arts try to optimize the performance and complexity of ViTs to enable efficient deployment on mobile…
Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Training
Zhenglun Kong, Haoyu Ma, Geng Yuan +12
Vision transformers (ViTs) have recently obtained success in many applications, but their intensive computation and heavy memory usage at both training and inference time limit the…
Self-Ensemble Protection: Training Checkpoints Are Good Data Protectors
Sizhe Chen, Geng Yuan, Xinwen Cheng +4
As data becomes increasingly vital, a company would be very cautious about releasing data, because the competitors could use it to train high-performance models, thereby posing a t…
You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model
Shengkun Tang, Yaqing Wang, Zhenglun Kong +6
Large-scale Transformer models bring significant improvements for various downstream vision language tasks with a unified architecture. The performance improvements come with incre…
Data Level Lottery Ticket Hypothesis for Vision Transformers
Xuan Shen, Zhenglun Kong, Minghai Qin +6
The conventional lottery ticket hypothesis (LTH) claims that there exists a sparse subnetwork within a dense neural network and a proper random initialization method called the win…
Pruning Adversarially Robust Neural Networks without Adversarial Examples
Tong Jian, Zifeng Wang, Yanzhi Wang +2
Adversarial pruning compresses models while preserving robustness. Current methods require access to adversarial examples during pruning. This significantly hampers training effici…
DeltaFS: Pursuing Zero Update Overhead via Metadata-Enabled Delta Compression for Log-structured File System on Mobile Devices
Chao Wu, Cheng Ji, Geng Yuan +5
Data compression has been widely adopted to release mobile devices from intensive write pressure. Delta compression is particularly promising for its high compression efficacy over…
Advancing Model Pruning via Bi-level Optimization
Yihua Zhang, Yuguang Yao, Parikshit Ram +5
The deployment constraints in practical applications necessitate the pruning of large-scale deep learning models, i.e., promoting their weight sparsity. As illustrated by the Lotte…
Layer Freezing & Data Sieving: Missing Pieces of a Generic Framework for Sparse Training
Geng Yuan, Yanyu Li, Sheng Li +5
Recently, sparse training has emerged as a promising paradigm for efficient deep learning on edge devices. The current research mainly devotes efforts to reducing training costs by…
SparCL: Sparse Continual Learning on the Edge
Zifeng Wang, Zheng Zhan, Yifan Gong +7
Existing work in continual learning (CL) focuses on mitigating catastrophic forgetting, i.e., model performance deterioration on past tasks when learning a new task. However, the t…
PIM-QAT: Neural Network Quantization for Processing-In-Memory (PIM) Systems
Qing Jin, Zhiyu Chen, Jian Ren +3
Processing-in-memory (PIM), an increasingly studied neuromorphic hardware, promises orders of energy and throughput improvements for deep learning inference. Leveraging the massive…
Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution
Yushu Wu, Yifan Gong, Pu Zhao +7
Deep learning-based super-resolution (SR) has gained tremendous popularity in recent years because of its high image quality performance and wide application scenarios. However, pr…
Real-Time Portrait Stylization on the Edge
Yanyu Li, Xuan Shen, Geng Yuan +5
In this work we demonstrate real-time portrait stylization, specifically, translating self-portrait into cartoon or anime style on mobile devices. We propose a latency-driven diffe…
Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization
Yanyu Li, Pu Zhao, Geng Yuan +3
Neural architecture search (NAS) and network pruning are widely studied efficient AI techniques, but not yet perfect. NAS performs exhaustive candidate architecture search, incurri…
EfficientFormer: Vision Transformers at MobileNet Speed
Yanyu Li, Geng Yuan, Yang Wen +5
Vision Transformers (ViT) have shown rapid progress in computer vision tasks, achieving promising results on various benchmarks. However, due to the massive number of parameters an…
Neural Network-based OFDM Receiver for Resource Constrained IoT Devices
Nasim Soltani, Hai Cheng, Mauro Belgiovine +10
Orthogonal Frequency Division Multiplexing (OFDM)-based waveforms are used for communication links in many current and emerging Internet of Things (IoT) applications, including the…
AirNN: Neural Networks with Over-the-Air Convolution via Reconfigurable Intelligent Surfaces
Sara Garcia Sanchez, Guillem Reus Muns, Carlos Bocanegra +6
Over-the-air analog computation allows offloading computation to the wireless environment through carefully constructed transmitted signals. In this paper, we design and implement…