19 citations · 22 across the 9 of their papers we have counts for
5 papers · 1 filter
AIBrix: Towards Scalable, Cost-Effective Large Language Model Inference Infrastructure
The AIBrix Team, Jiaxin Shan, Varun Gupta +24
We introduce AIBrix, a cloud-native, open-source framework designed to optimize and simplify large-scale LLM deployment in cloud environments. Unlike traditional cloud-native stack…
NotebookOS: A Replicated Notebook Platform for Interactive Training with On-Demand GPUs
Benjamin Carver, Jingyuan Zhang, Haoliang Wang +2
Interactive notebook programming is universal in modern ML and AI workflows, with interactive deep learning training (IDLT) emerging as a dominant use case. To ensure responsivenes…
FS: A Scalable and Elastic Distributed File System Metadata Service using Serverless Functions
Benjamin Carver, Runzhou Han, Jingyaun Zhang +2
The metadata service (MDS) sits on the critical path for distributed file system (DFS) operations, and therefore it is key to the overall performance of a large-scale DFS. Common "…
InfiniCache: Exploiting Ephemeral Serverless Functions to Build a Cost-Effective Memory Cache
Ao Wang, Jingyuan Zhang, Xiaolong Ma +6
Internet-scale web applications are becoming increasingly storage-intensive and rely heavily on in-memory object caching to attain required I/O performance. We argue that the emerg…
In Search of a Fast and Efficient Serverless DAG Engine
Benjamin Carver, Jingyuan Zhang, Ao Wang +1
Python-written data analytics applications can be modeled as and compiled into a directed acyclic graph (DAG) based workflow, where the nodes are fine-grained tasks and the edges a…