9 papers
A Cloud-Edge System for Multimodal Clinical Screening in Resource-Constrained Rural Settings
Hei Ting, Chan, Chenwei Wu +8
Medical AI has demonstrated specialist-level diagnostic accuracy, yet these capabilities remain largely inaccessible in resource-constrained rural settings where bandwidth is scarc…
LLMVisor: A Real-Time Latency Attribution Model for Multi-Tenant LLM Serving
Shuowei Jin, Xueshen Liu, Jiaxin Shan +4
As LLM inference shifts to multi-tenant GPU clusters, co-batching improves throughput but obscures per-tenant usage and limits control. Enabling fractional sharing of the inference…
Dynamic Linear Attention
Xin Wang, Hui Shen, Boyuan Zheng +7
The scalability of Large Language Models (LLMs) to long contexts is fundamentally constrained by the quadratic complexity of standard attention, motivating the adoption of linear a…
AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs
Haizhong Zheng, Yizhuo Di, Jiahui Wang +7
Reinforcement learning (RL) is increasingly used to improve the reasoning, coding, and tool-use capabilities of large language models, but agentic RL remains prohibitively expensiv…
Dr. Post-Training: A Data Regularization Perspective on LLM Post-Training
Pingbang Hu, Xueshen Liu, Z. Morley Mao +1
Data selection methods address a critical challenge in LLM post-training: effectively leveraging scarce, high-fidelity target data alongside abundant but imperfectly aligned genera…
Foundry: Template-Based CUDA Graph Context Materialization for Fast LLM Serving Cold Start
Xueshen Liu, Yongji Wu, Yuncheng Yao +3
Modern LLM service providers increasingly rely on autoscaling and parallelism reconfiguration to respond to rapidly changing workloads, but cold-start latency remains a major bottl…