161 citations · 187 across the 8 of their papers we have counts for
8 papers
An Internet for the KV Cache: Rethinking Classical Infrastructure Boundaries in the LLM Inference Age
Siddhant Ray, Nick Feamster, Junchen Jiang
LLM inference has become a global-scale, heterogeneous workload spanning agents, retrieval, tool-use, code execution and multi-modal reasoning. These workloads naturally enable con…
HyperRAG: Enhancing Quality-Efficiency Tradeoffs in Retrieval-Augmented Generation with Reranker KV-Cache Reuse
Yuwei An, Yihua Cheng, Seo Jin Park +1
Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for enhancing the performance of large language models (LLMs) by integrating external knowledge into the gen…
Do Large Language Models Need a Content Delivery Network?
Yihua Cheng, Kuntai Du, Jiayi Yao +1
As the use of large language models (LLMs) expands rapidly, so does the range of knowledge needed to supplement various LLM queries. Thus, enabling flexible and efficient injection…
SwiftQueue: Optimizing Low-Latency Applications with Swift Packet Queuing
Siddhant Ray, Xi Jiang, Jack Luo +2
Low Latency, Low Loss, and Scalable Throughput (L4S), as an emerging router-queue management technique, has seen steady deployment in the industry. An L4S-enabled router assigns ea…
Earth+: on-board satellite imagery compression leveraging historical earth observations
Kuntai Du, Yihua Cheng, Peder Olsen +3
With the increasing deployment of earth observation satellite constellations, the downlink (satellite-to-ground) capacity often limits the freshness, quality, and coverage of the i…
NetLLM: Adapting Large Language Models for Networking
Duo Wu, Xianda Wang, Yaqi Qiao +4
Many networking tasks now employ deep learning (DL) to solve complex prediction and optimization problems. However, current design philosophy of DL-based algorithms entails intensi…
Eloquent: A More Robust Transmission Scheme for LLM Token Streaming
Hanchen Li, Yuhan Liu, Yihua Cheng +3
To render each generated token in real-time for users, the Large Language Model (LLM) server generates tokens one by one and streams each token (or group of a few tokens) through t…
VidPlat: A Tool for Fast Crowdsourcing of Quality-of-Experience Measurements
Xu Zhang, Hanchen Li, Paul Schmitt +3
For video or web services, it is crucial to measure user-perceived quality of experience (QoE) at scale under various video quality or page loading delays. However, fast QoE measur…