19 citations · 21 across the 3 of their papers we have counts for
6 papers
VL-RouterBench: A Benchmark for Vision-Language Model Routing
Zhehao Huang, Baijiong Lin, Jingyuan Zhang +5
Multi-model routing has evolved from an engineering technique into essential infrastructure, yet existing work lacks a systematic, reproducible benchmark for evaluating vision-lang…
AIBrix: Towards Scalable, Cost-Effective Large Language Model Inference Infrastructure
The AIBrix Team, Jiaxin Shan, Varun Gupta +24
We introduce AIBrix, a cloud-native, open-source framework designed to optimize and simplify large-scale LLM deployment in cloud environments. Unlike traditional cloud-native stack…
NotebookOS: A Replicated Notebook Platform for Interactive Training with On-Demand GPUs
Benjamin Carver, Jingyuan Zhang, Haoliang Wang +2
Interactive notebook programming is universal in modern ML and AI workflows, with interactive deep learning training (IDLT) emerging as a dominant use case. To ensure responsivenes…
AdaSVD: Adaptive Singular Value Decomposition for Large Language Models
Zhiteng Li, Mingyuan Xia, Jingyuan Zhang +5
Large language models (LLMs) have achieved remarkable success in natural language processing (NLP) tasks, yet their substantial memory requirements present significant challenges f…
InfiniCache: Exploiting Ephemeral Serverless Functions to Build a Cost-Effective Memory Cache
Ao Wang, Jingyuan Zhang, Xiaolong Ma +6
Internet-scale web applications are becoming increasingly storage-intensive and rely heavily on in-memory object caching to attain required I/O performance. We argue that the emerg…
In Search of a Fast and Efficient Serverless DAG Engine
Benjamin Carver, Jingyuan Zhang, Ao Wang +1
Python-written data analytics applications can be modeled as and compiled into a directed acyclic graph (DAG) based workflow, where the nodes are fine-grained tasks and the edges a…