3 papers
cs.LG2026
Not All Tokens Are Worth Caching: Learning Semantic-Aware Eviction for LLM Prefix Caches
Shaoke Fang, Ziang Li, Wenfei Wu +3
Prefix caching is a key optimization in Large Language Model (LLM) serving, reusing attention Key-Value (KV) states across requests with shared prompt prefixes to reduce expensive…
cs.NI2021
HyperNAT: Scaling Up Network AddressTranslation with SmartNICs for Clouds
Shaoke Fang, Qingsong Liu, Wenfei Wu
Network address translation (NAT) is a basic functionality in cloud gateways. With the increasing traffic volume and number of flows introduced by the cloud tenants, the NAT gatewa…
math.OC2021
Simultaneously Achieving Sublinear Regret and Constraint Violations for Online Convex Optimization with Time-varying Constraints
Qingsong Liu, Wenfei Wu, Longbo Huang +1
In this paper, we develop a novel virtual-queue-based online algorithm for online convex optimization (OCO) problems with long-term and time-varying constraints and conduct a perfo…