18 citations · 21 across the 21 of their papers we have counts for
21 papers
Breaking the Reward Barrier: Accelerating Tree-of-Thought Reasoning via Speculative Exploration
Shuzhang Zhong, Haochen Huang, Shengxuan Qiu +3
Tree-of-Thought (ToT) reasoning structures Large Language Model (LLM) inference as a tree-based search, demonstrating strong potential for solving complex mathematical and programm…
Aging Aware Adaptive Voltage Scaling for Reliable and Efficient AI Accelerators
Tong Xie, Zuodong Zhang, Chao Yang +3
Deep neural networks (DNNs) have showcased remarkable performance across various tasks and are widely deployed on AI accelerators fabricated in advanced technology nodes for effici…
The Quest for Reliable AI Accelerators: Cross-Layer Evaluation and Design Optimization
Meng Li, Tong Xie, Zuodong Zhang +1
As the CMOS technology pushes to the nanoscale, aging effects and process variations have become increasingly pronounced, posing significant reliability challenges for AI accelerat…
CREATE: Cross-Layer Resilience Characterization and Optimization for Efficient yet Reliable Embodied AI Systems
Tong Xie, Yijiahao Qi, Jinqi Wen +9
Embodied Artificial Intelligence (AI) has recently attracted significant attention as it bridges AI with the physical world. Modern embodied AI systems often combine a Large Langua…
Breaking the Layer Barrier: Remodeling Private Transformer Inference with Hybrid CKKS and MPC
Tianshi Xu, Wen-jie Lu, Jiangrui Yu +4
This paper presents an efficient framework for private Transformer inference that combines Homomorphic Encryption (HE) and Secure Multi-party Computation (MPC) to protect data priv…
No Redundancy, No Stall: Lightweight Streaming 3D Gaussian Splatting for Real-time Rendering
Linye Wei, Jiajun Tang, Fan Fei +3
3D Gaussian Splatting (3DGS) enables high-quality rendering of 3D scenes and is getting increasing adoption in domains like autonomous driving and embodied intelligence. However, 3…
SpecASR: Accelerating LLM-based Automatic Speech Recognition via Speculative Decoding
Linye Wei, Shuzhang Zhong, Songqiang Xu +3
Large language model (LLM)-based automatic speech recognition (ASR) has recently attracted a lot of attention due to its high recognition accuracy and enhanced multi-dialect suppor…
HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference
Shuzhang Zhong, Yanfan Sun, Ling Liang +3
The Mixture of Experts (MoE) architecture has demonstrated significant advantages as it enables to increase the model capacity without a proportional increase in computation. Howev…
ReaLM: Reliable and Efficient Large Language Model Inference with Statistical Algorithm-Based Fault Tolerance
Tong Xie, Jiawang Zhao, Zishen Wan +5
The demand for efficient large language model (LLM) inference has propelled the development of dedicated accelerators. As accelerators are vulnerable to hardware faults due to agin…
LightMamba: Efficient Mamba Acceleration on FPGA with Quantization and Hardware Co-design
Renjie Wei, Songqiang Xu, Linfeng Zhong +5
State space models (SSMs) like Mamba have recently attracted much attention. Compared to Transformer-based large language models (LLMs), Mamba achieves linear computation complexit…
PrivQuant: Communication-Efficient Private Inference with Quantized Network/Protocol Co-Optimization
Tianshi Xu, Shuzhang Zhong, Wenxuan Zeng +2
Private deep neural network (DNN) inference based on secure two-party computation (2PC) enables secure privacy protection for both the server and the client. However, existing secu…
AdapMoE: Adaptive Sensitivity-based Expert Gating and Management for Efficient MoE Inference
Shuzhang Zhong, Ling Liang, Yuan Wang +3
Mixture-of-Experts (MoE) models are designed to enhance the efficiency of large language models (LLMs) without proportionally increasing the computational demands. However, their d…
FastQuery: Communication-efficient Embedding Table Query for Private LLM Inference
Chenqi Lin, Tianshi Xu, Zebin Yang +3
With the fast evolution of large language models (LLMs), privacy concerns with user queries arise as they may contain sensitive information. Private inference based on homomorphic…
PrivCirNet: Efficient Private Inference via Block Circulant Transformation
Tianshi Xu, Lemeng Wu, Runsheng Wang +1
Homomorphic encryption (HE)-based deep neural network (DNN) inference protects data and model privacy but suffers from significant computation overhead. We observe transforming the…
EQO: Exploring Ultra-Efficient Private Inference with Winograd-Based Protocol and Quantization Co-Optimization
Wenxuan Zeng, Tianshi Xu, Meng Li +1
Private convolutional neural network (CNN) inference based on secure two-party computation (2PC) suffers from high communication and latency overhead, especially from convolution l…
ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding
Shuzhang Zhong, Zebin Yang, Meng Li +3
Recent advancements in generative large language models (LLMs) have significantly boosted the performance in natural language processing tasks. However, their efficiency is hampere…
ASCEND: Accurate yet Efficient End-to-End Stochastic Computing Acceleration of Vision Transformer
Tong Xie, Yixuan Hu, Renjie Wei +4
Stochastic computing (SC) has emerged as a promising computing paradigm for neural acceleration. However, how to accelerate the state-of-the-art Vision Transformer (ViT) with SC re…
HEQuant: Marrying Homomorphic Encryption and Quantization for Communication-Efficient Private Inference
Tianshi Xu, Meng Li, Runsheng Wang
Secure two-party computation with homomorphic encryption (HE) protects data privacy with a formal security guarantee but suffers from high communication overhead. While previous wo…
Efficient yet Accurate End-to-End SC Accelerator Design
Meng Li, Yixuan Hu, Tengyu Zhang +4
Providing end-to-end stochastic computing (SC) neural network acceleration for state-of-the-art (SOTA) models has become an increasingly challenging task, requiring the pursuit of…
CoPriv: Network/Protocol Co-Optimization for Communication-Efficient Private Inference
Wenxuan Zeng, Meng Li, Haichuan Yang +3
Deep neural network (DNN) inference based on secure 2-party computation (2PC) can offer cryptographically-secure privacy protection but suffers from orders of magnitude latency ove…
READ: Reliability-Enhanced Accelerator Dataflow Optimization using Critical Input Pattern Reduction
Zuodong Zhang, Renjie Wei, Meng Li +3
With the rapid advancements of deep learning in recent years, hardware accelerators are continuously deployed in more and more safety-critical applications such as autonomous drivi…