27 citations · 33 across the 17 of their papers we have counts for
17 papers
LLMVisor: A Real-Time Latency Attribution Model for Multi-Tenant LLM Serving
Shuowei Jin, Xueshen Liu, Jiaxin Shan +4
As LLM inference shifts to multi-tenant GPU clusters, co-batching improves throughput but obscures per-tenant usage and limits control. Enabling fractional sharing of the inference…
CoMem: Context Management with A Decoupled Long-Context Model
Yuwei Zhang, Chengyu Dong, Shuowei Jin +11
Context management enables agentic models to solve long-horizon tasks through iterative summarization of previous interaction histories. However, this process typically incurs subs…
AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs
Haizhong Zheng, Yizhuo Di, Jiahui Wang +7
Reinforcement learning (RL) is increasingly used to improve the reasoning, coding, and tool-use capabilities of large language models, but agentic RL remains prohibitively expensiv…
VIP: Visual-guided Prompt Evolution for Efficient Dense Vision-Language Inference
Hao Zhu, Shuo Jin, Wenbin Liao +4
Pursuing training-free open-vocabulary semantic segmentation in an efficient and generalizable manner remains challenging due to the deep-seated spatial bias in CLIP. To overcome t…
Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation
Yuwei Zhang, Sha Li, Changlong Yu +9
Enabling Large Language Models (LLMs) to continuously improve from environmental interactions is a central challenge in post-training. While on-policy self-distillation offers a pr…
Rethinking Importance Sampling in LLM Policy Optimization: A Cumulative Token Perspective
Yuheng Zhang, Chenlu Ye, Shuowei Jin +4
Reinforcement learning, including reinforcement learning with verifiable rewards (RLVR), has emerged as a powerful approach for LLM post-training. Central to these approaches is th…
TPO: Uncertainty-Guided Exploration Control for Stable Multi-Turn Agentic Reinforcement Learning
Haixin Wang, Hejie Cui, Chenwei Zhang +7
Recent progress in multi-turn reinforcement learning (RL) has significantly improved reasoning LLMs' performances on complex interactive tasks. Despite advances in stabilization te…
MARS: Harmonizing Multimodal Convergence via Adaptive Rank Search
Minkyoung Cho, Insu Jang, Shuowei Jin +5
Fine-tuning Multimodal Large Language Models (MLLMs) with parameter-efficient methods like Low-Rank Adaptation (LoRA) is crucial for task adaptation. However, imbalanced training d…
HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning
Weiqi Wang, Xin Liu, Binxuan Huang +13
RLVR has become a standard recipe for training LLMs on reasoning tasks with verifiable outcomes, but when rollout generation dominates the cost, efficiency hinges on which prompts…
HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUs
Yongji Wu, Xueshen Liu, Shuowei Jin +6
The Mixture-of-Experts (MoE) architecture has become increasingly popular as a method to scale up large language models (LLMs). To save costs, heterogeneity-aware training solution…
AIBrix: Towards Scalable, Cost-Effective Large Language Model Inference Infrastructure
The AIBrix Team, Jiaxin Shan, Varun Gupta +24
We introduce AIBrix, a cloud-native, open-source framework designed to optimize and simplify large-scale LLM deployment in cloud environments. Unlike traditional cloud-native stack…
Eagle: Efficient Training-Free Router for Multi-LLM Inference
Zesen Zhao, Shuowei Jin, Z. Morley Mao
The proliferation of Large Language Models (LLMs) with varying capabilities and costs has created a need for efficient model selection in AI systems. LLM routers address this need…
Compute Or Load KV Cache? Why Not Both?
Shuowei Jin, Xueshen Liu, Qingzhao Zhang +1
Large Language Models (LLMs) are increasingly deployed in large-scale online services, enabling sophisticated applications. However, the computational overhead of generating key-va…
AutoSpec: Automated Generation of Neural Network Specifications
Shuowei Jin, Taobo Liao, Anuj Kalia +5
The increasing adoption of neural networks in learning-augmented systems highlights the growing need for model safety and robustness, especially in safety-critical domains. While r…
Plato: Plan to Efficiently Decode for Large Language Model Inference
Shuowei Jin, Xueshen Liu, Yongji Wu +7
Large language models (LLMs) have achieved remarkable success in natural language tasks, but their inference incurs substantial computational and memory overhead. To improve effici…
On Data Fabrication in Collaborative Vehicular Perception: Attacks and Countermeasures
Qingzhao Zhang, Shuowei Jin, Ruiyang Zhu +4
Collaborative perception, which greatly enhances the sensing capability of connected and autonomous vehicles (CAVs) by incorporating data from external resources, also brings forth…
QUIC is not Quick Enough over Fast Internet
Xumiao Zhang, Shuowei Jin, Yi He +4
QUIC is expected to be a game-changer in improving web application performance. In this paper, we conduct a systematic examination of QUIC's performance over high-speed networks. W…