3 citations · 3 across the 8 of their papers we have counts for
4 papers · 1 filter
MCUBERT: Memory-Efficient BERT Inference on Commodity Microcontrollers
Zebin Yang, Renze Chen, Taiqiang Wu +5
In this paper, we propose MCUBERT to enable language models like BERT on tiny microcontroller units (MCUs) through network and scheduling co-optimization. We observe the embedding…
FastQuery: Communication-efficient Embedding Table Query for Private LLM Inference
Chenqi Lin, Tianshi Xu, Zebin Yang +3
With the fast evolution of large language models (LLMs), privacy concerns with user queries arise as they may contain sensitive information. Private inference based on homomorphic…
ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding
Shuzhang Zhong, Zebin Yang, Meng Li +3
Recent advancements in generative large language models (LLMs) have significantly boosted the performance in natural language processing tasks. However, their efficiency is hampere…
AttentionLego: An Open-Source Building Block For Spatially-Scalable Large Language Model Accelerator With Processing-In-Memory Technology
Rongqing Cong, Wenyang He, Mingxuan Li +5
Large language models (LLMs) with Transformer architectures have become phenomenal in natural language processing, multimodal generative artificial intelligence, and agent-oriented…