6 papers
Resuming Vision-Language Inference across Onboard Interruptions
Xiaolong Huang, Yinyuan Zhang, Daliang Xu +6
Executing vision-language models (VLMs) aboard low-Earth orbit satellites enables flexible processing of observation data without first downlinking it to the ground. On commercial…
Empowering Hybrid Attention Models on NPUs
Yinyuan Zhang, Daliang Xu, Xiaolong Huang +4
Hybrid attention models have emerged as a crucial architecture for Large Language Models (LLMs) (e.g., the Qwen3.5 and Kimi series). Their memory and computational efficiency make…
NanoSpec: Accelerating Speculative Decoding using Minimalist In-Context Vocabularies
Zhiyang Chen, Daliang Xu, Yinyuan Zhang +3
The massive vocabulary sizes of large language models, often exceeding 100k tokens, impose a computational bottleneck on the final linear projection layer during speculative decodi…
Quant.npu: Enabling Efficient Mobile NPU Inference for on-device LLMs via Fully Static Quantization
Jinghe Zhang, Daliang Xu, Chenghua Wang +5
Large language models (LLMs) are increasingly deployed on mobile devices, where Neural Processing Units (NPUs) necessitate fully static quantization for optimal inference efficienc…
A First Look at Bugs in LLM Inference Engines
Mugeng Liu, Siqi Zhong, Weichen Bi +5
Large language model-specific inference engines (in short as \emph{LLM inference engines}) have become a fundamental component of modern AI infrastructure, enabling the deployment…
DRAGON: Domain-specific Robust Automatic Data Generation for RAG Optimization
Haiyang Shen, Hang Yan, Zhongshi Xing +6
Retrieval-augmented generation (RAG) can substantially enhance the performance of LLMs on knowledge-intensive tasks. Various RAG paradigms - including vanilla, planning-based, and…