8 papers
LAPS: A Length-Aware-Prefill LLM Serving System
Jianshu She, Zonghang Li, Hongchao Du +7
LAPS identifies and disaggregates requests with different prompt lengths in LLM serving to reduce TTFT latency. While recent systems have decoupled the prefill and decode stages to…
CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
Wenhao Zheng, Yixiao Chen, Weitong Zhang +6
Large language models have achieved remarkable success in various tasks but suffer from high computational costs during inference, limiting their deployment in resource-constrained…
AI for NONMEM Coding in Pharmacometrics Research and Education: Shortcut or Pitfall?
Wenhao Zheng, Wanbing Wang, Carl M. J. Kirkpatrick +3
Artificial intelligence (AI) is increasingly being explored as a tool to support pharmacometric modeling, particularly in addressing the coding challenges associated with NONMEM. I…
Token Level Routing Inference System for Edge Devices
Jianshu She, Wenhao Zheng, Zhengzhong Liu +4
The computational complexity of large language model (LLM) inference significantly constrains their deployment efficiency on edge devices. In contrast, small language models offer…
MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models
Peng Xia, Siwei Han, Shi Qiu +9
Interleaved multimodal comprehension and generation, enabling models to produce and interpret both images and text in arbitrary sequences, have become a pivotal area in multimodal…
Verifiable Format Control for Large Language Model Generations
Zhaoyang Wang, Jinqi Jiang, Huichi Zhou +4
Recent Large Language Models (LLMs) have demonstrated satisfying general instruction following ability. However, small LLMs with about 7B parameters still struggle fine-grained for…