9 citations · 9 across the 8 of their papers we have counts for
8 papers
Affix Cache for Diffusion Large Language Models
Kaihua Liang, An Zhong, Xin Tan +4
Diffusion Large Language Models (DLLMs) enable non-autoregressive decoding, but efficient inference support remains immature: unlike autoregressive models, whose requests reuse a s…
UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scaling
Kaiyu Huang, Xingyu Wang, Mingze Kong +6
In real-world deployments of large language models (LLMs), balancing inference quality and computational cost has become a central challenge. Existing approaches tackle this trade-…
PRISM: Parametrically Refactoring Inference for Speculative Sampling Draft Models
Xuliang Wang, Yuetao Chen, Maochan Zhen +5
Large Language Models (LLMs), constrained by their auto-regressive nature, suffer from slow decoding. Speculative decoding methods have emerged as a promising solution to accelerat…
FOCUS: DLLMs Know How to Tame Their Compute Bound
Kaihua Liang, Xin Tan, An Zhong +2
Diffusion Large Language Models (DLLMs) offer a compelling alternative to Auto-Regressive models, but their deployment is constrained by high decoding cost. In this work, we identi…
Multimodal Learning for Fake News Detection in Short Videos Using Linguistically Verified Data and Heterogeneous Modality Fusion
Shanghong Li, Chiam Wen Qi Ruth, Hong Xu +1
The rapid proliferation of short video platforms has necessitated advanced methods for detecting fake news. This need arises from the widespread influence and ease of sharing misin…
MambaLite-Micro: Memory-Optimized Mamba Inference on MCUs
Hongjun Xu, Junxi Xia, Weisi Yang +2
Deploying Mamba models on microcontrollers (MCUs) remains challenging due to limited memory, the lack of native operator support, and the absence of embedded-friendly toolchains. W…
Teola: Towards End-to-End Optimization of LLM-based Applications
Xin Tan, Yimin Jiang, Yitao Yang +1
Large language model (LLM)-based applications consist of both LLM and non-LLM components, each contributing to the end-to-end latency. Despite great efforts to optimize LLM inferen…
BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models
Bodun Hu, Jiamin Li, Le Xu +5
The increasing demand for Large Language Models (LLMs) across various applications has led to a significant shift in the design of deep learning serving systems. Deploying LLMs, pa…