211 citations · 474 across the 38 of their papers we have counts for
39 papers
BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration
Chence Yang, Ningxi Cheng, Arash Akbari +9
Speculative decoding accelerates autoregressive generation by using a lightweight draft to propose multiple tokens for parallel verification. However, existing methods often requir…
Optimizing 3D Gaussian Splattering for Mobile GPUs
Md Musfiqur Rahman Sanim, Zhihao Shu, Bahram Afsharmanesh +5
Image-based 3D scene reconstruction, which transforms multi-view images into a structured 3D representation of the surrounding environment, is a common task across many modern appl…
QuartDepth: Post-Training Quantization for Real-Time Depth Estimation on the Edge
Xuan Shen, Weize Ma, Jing Liu +9
Monocular Depth Estimation (MDE) has emerged as a pivotal task in computer vision, supporting numerous real-world applications. However, deploying accurate depth estimation models…
Open-Source Acceleration of Stable-Diffusion.cpp Deployable on All Devices
Jingxu Ng, Cheng Lv, Pu Zhao +5
Stable diffusion plays a crucial role in generating high-quality images. However, image generation is time-consuming and memory-intensive. To address this, stable-diffusion.cpp (Sd…
LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformers
Xuan Shen, Zhao Song, Yufa Zhou +12
Diffusion Transformers have emerged as the preeminent models for a wide array of generative tasks, demonstrating superior performance and efficacy across various applications. The…
Fast and Memory-Efficient Video Diffusion Using Streamlined Inference
Zheng Zhan, Yushu Wu, Yifan Gong +7
The rapid progress in artificial intelligence-generated content (AIGC), especially with diffusion models, has significantly advanced development of high-quality video generation. H…
Exploring Token Pruning in Vision State Space Models
Zheng Zhan, Zhenglun Kong, Yifan Gong +8
State Space Models (SSMs) have the advantage of keeping linear computational complexity compared to attention modules in transformers, and have been applied to vision tasks as a ne…
SoD: Statically Optimizing Dynamic Deep Neural Network
Wei Niu, Gagan Agrawal, Bin Ren
Though many compilation and runtime systems have been developed for DNNs in recent years, the focus has largely been on static DNNs. Dynamic DNNs, where tensor shapes and sizes and…
Squat: Quant Small Language Models on the Edge
Xuan Shen, Peiyan Dong, Zhenglun Kong +9
A growing trend has emerged in designing high-quality Small Language Models (SLMs) with a few million parameters. This trend is driven by the increasing concerns over cloud costs,…
Towards High-Quality and Efficient Video Super-Resolution via Spatial-Temporal Data Overfitting
Gen Li, Jie Ji, Minghai Qin +5
As deep convolutional neural networks (DNNs) are widely used in various fields of computer vision, leveraging the overfitting ability of the DNN to achieve video resolution upscali…
SparCL: Sparse Continual Learning on the Edge
Zifeng Wang, Zheng Zhan, Yifan Gong +7
Existing work in continual learning (CL) focuses on mitigating catastrophic forgetting, i.e., model performance deterioration on past tasks when learning a new task. However, the t…
Survey: Exploiting Data Redundancy for Optimization of Deep Learning
Jou-An Chen, Wei Niu, Bin Ren +2
Data redundancy is ubiquitous in the inputs and intermediate results of Deep Neural Networks (DNN). It offers many significant opportunities for improving DNN performance and effic…
Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution
Yushu Wu, Yifan Gong, Pu Zhao +7
Deep learning-based super-resolution (SR) has gained tremendous popularity in recent years because of its high image quality performance and wide application scenarios. However, pr…
Real-Time Portrait Stylization on the Edge
Yanyu Li, Xuan Shen, Geng Yuan +5
In this work we demonstrate real-time portrait stylization, specifically, translating self-portrait into cartoon or anime style on mobile devices. We propose a latency-driven diffe…
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning
Zhenglun Kong, Peiyan Dong, Xiaolong Ma +9
Recently, Vision Transformer (ViT) has continuously established new milestones in the computer vision field, while the high computation and memory cost makes its propagation in ind…
Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration
Yifan Gong, Geng Yuan, Zheng Zhan +9
Weight pruning is an effective model compression technique to tackle the challenges of achieving real-time deep neural network (DNN) inference on mobile devices. However, prior pru…
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
Geng Yuan, Xiaolong Ma, Wei Niu +13
Recently, a new trend of exploring sparsity for accelerating neural network training has emerged, embracing the paradigm of training on the edge. This paper proposes a novel Memory…
Enabling Level-4 Autonomous Driving on a Single $1k Off-the-Shelf Card
Hsin-Hsuan Sung, Yuanchao Xu, Jiexiong Guan +5
Autonomous driving is of great interest in both research and industry. The high cost has been one of the major roadblocks that slow down the development and adoption of autonomous…
GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity
Wei Niu, Zhengang Li, Xiaolong Ma +6
It is appealing but challenging to achieve real-time deep neural network (DNN) inference on mobile devices because even the powerful modern mobile devices are considered as ``resou…
DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion
Wei Niu, Jiexiong Guan, Yanzhi Wang +2
Deep Neural Networks (DNNs) have emerged as the core enabler of many major applications on mobile devices. To achieve high accuracy, DNN models have become increasingly deep with h…
Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search
Zheng Zhan, Yifan Gong, Pu Zhao +9
Though recent years have witnessed remarkable progress in single image super-resolution (SISR) tasks with the prosperous development of deep neural networks (DNNs), the deep learni…
Achieving Real-Time Object Detection on MobileDevices with Neural Pruning Search
Pu Zhao, Wei Niu, Geng Yuan +4
Object detection plays an important role in self-driving cars for security development. However, mobile systems on self-driving cars with limited computation resources lead to diff…
Towards Fast and Accurate Multi-Person Pose Estimation on Mobile Devices
Xuan Shen, Geng Yuan, Wei Niu +5
The rapid development of autonomous driving, abnormal behavior detection, and behavior recognition makes an increasing demand for multi-person pose estimation-based applications, e…
A Compression-Compilation Framework for On-mobile Real-time BERT Applications
Wei Niu, Zhenglun Kong, Geng Yuan +7
Transformer-based deep learning models have increasingly demonstrated high accuracy on many natural language processing (NLP) tasks. In this paper, we propose a compression-compila…