27 citations · 33 across the 17 of their papers we have counts for
4 papers · 1 filter
Eagle: Efficient Training-Free Router for Multi-LLM Inference
Zesen Zhao, Shuowei Jin, Z. Morley Mao
The proliferation of Large Language Models (LLMs) with varying capabilities and costs has created a need for efficient model selection in AI systems. LLM routers address this need…
Compute Or Load KV Cache? Why Not Both?
Shuowei Jin, Xueshen Liu, Qingzhao Zhang +1
Large Language Models (LLMs) are increasingly deployed in large-scale online services, enabling sophisticated applications. However, the computational overhead of generating key-va…
AutoSpec: Automated Generation of Neural Network Specifications
Shuowei Jin, Taobo Liao, Anuj Kalia +5
The increasing adoption of neural networks in learning-augmented systems highlights the growing need for model safety and robustness, especially in safety-critical domains. While r…
Plato: Plan to Efficiently Decode for Large Language Model Inference
Shuowei Jin, Xueshen Liu, Yongji Wu +7
Large language models (LLMs) have achieved remarkable success in natural language tasks, but their inference incurs substantial computational and memory overhead. To improve effici…