47 citations · 94 across the 13 of their papers we have counts for
13 papers
A 28nm 0.22μJ/token memory-compute-intensity-aware CNN-Transformer accelerator with hybrid-attention-based layer-fusion and cascaded pruning for semantic-segmentation
Pingcheng Dong, Yonghao Tan, Xuejiao Liu +14
This work presents a 28nm 13.93mm2 CNN-Transformer accelerator for semantic segmentation, achieving 3.86-to-10.91x energy reduction over previous designs. It features a hybrid atte…
APSQ: Additive Partial Sum Quantization with Algorithm-Hardware Co-Design
Yonghao Tan, Pingcheng Dong, Yongkun Wu +8
DNN accelerators, significantly advanced by model compression and specialized dataflow techniques, have marked considerable progress. However, the frequent access of high-precision…
SnapGen: Taming High-Resolution Text-to-Image Models for Mobile Devices with Efficient Architectures and Training
Dongting Hu, Jierun Chen, Xijie Huang +16
Existing text-to-image (T2I) diffusion models face several limitations, including large model sizes, slow runtime, and low-quality generation on mobile devices. This paper aims to…
RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization
Xijie Huang, Zechun Liu, Shih-Yang Liu +1
Low-Rank Adaptation (LoRA), as a representative Parameter-Efficient Fine-Tuning (PEFT)method, significantly enhances the training efficiency by updating only a small portion of the…
Genetic Quantization-Aware Approximation for Non-Linear Operations in Transformers
Pingcheng Dong, Yonghao Tan, Dong Zhang +11
Non-linear functions are prevalent in Transformers and their lightweight variants, incurring substantial and frequently underestimated hardware costs. Previous state-of-the-art wor…
Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning
Xijie Huang, Li Lyna Zhang, Kwang-Ting Cheng +2
Large Language Models (LLMs) have shown impressive capabilities, yet they still struggle with math reasoning. In this work, we propose CoT-Influx, a novel approach that pushes the…
LLM-FP4: 4-Bit Floating-Point Quantized Transformers
Shih-yang Liu, Zechun Liu, Xijie Huang +2
We propose LLM-FP4 for quantizing both weights and activations in large language models (LLMs) down to 4-bit floating-point values, in a post-training manner. Existing post-trainin…
Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precision
Xijie Huang, Zhiqiang Shen, Pingcheng Dong +1
Despite the outstanding performance of transformers in both language and vision tasks, the expanding computation and model size have increased the demand for efficient deployment.…
Efficient and Robust Quantization-aware Training via Adaptive Coreset Selection
Xijie Huang, Zechun Liu, Shih-Yang Liu +1
Quantization-aware training (QAT) is a representative model compression method to reduce redundancy in weights and activations. However, most existing QAT methods require end-to-en…
SDQ: Stochastic Differentiable Quantization with Mixed Precision
Xijie Huang, Zhiqiang Shen, Shichao Li +5
In order to deploy deep models in a computationally efficient manner, model quantization approaches have been frequently used. In addition, as new hardware that supports mixed bitw…
FedMix: Mixed Supervised Federated Learning for Medical Image Segmentation
Jeffry Wicaksana, Zengqiang Yan, Dong Zhang +4
The purpose of federated learning is to enable multiple clients to jointly train a machine learning model without sharing data. However, the existing methods for training an image…
Automated Vision-Based Wellness Analysis for Elderly Care Centers
Xijie Huang, Jeffry Wicaksana, Shichao Li +1
The growth in the aging population requires caregivers to improve both efficiency and quality of healthcare. In this study, we develop an automatic, vision-based system for monitor…
Joint stereo 3D object detection and implicit surface reconstruction
Shichao Li, Xijie Huang, Zechun Liu +1
We present a new learning-based framework S-3D-RCNN that can recover accurate object orientation in SO(3) and simultaneously predict implicit rigid shapes from stereo RGB images. F…