1 citations · 3 across the 12 of their papers we have counts for
12 papers
GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
Boxiao Du, Boning Huangfu, Yizhou Luo +5
Large Language Models (LLMs) play a critical role in emerging agentic applications, where the timely completion of each entire inference is critical. Meanwhile, agentic LLM inferen…
Towards Compute-Aware In-Switch Computing for LLMs Tensor-Parallelism on Multi-GPU Systems
Chen Zhang, Qijun Zhang, Zhuoshan Zhou +10
Tensor parallelism (TP) in large-scale LLM inference and training introduces frequent collective operations that dominate inter-GPU communication. While in-switch computing, exempl…
ThinkTrap: Denial-of-Service Attacks against Black-box LLM Services via Infinite Thinking
Yunzhe Li, Jianan Wang, Hongzi Zhu +3
Large Language Models (LLMs) have become foundational components in a wide range of applications, including natural language understanding and generation, embodied intelligence, an…
BlinkBud: Detecting Hazards from Behind via Sampled Monocular 3D Detection on a Single Earbud
Yunzhe Li, Jiajun Yan, Yuzhou Wei +7
Failing to be aware of speeding vehicles approaching from behind poses a huge threat to the road safety of pedestrians and cyclists. In this paper, we propose BlinkBud, which utili…
LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving
Huanqi Hu, Bowen Xiao, Shixuan Sun +8
Quantization is a critical technique for accelerating LLM inference by reducing memory footprint and improving computational efficiency. Among various schemes, 4-bit weight and 8-b…
Efficient Serving of LLM Applications with Probabilistic Demand Modeling
Yifei Liu, Zuo Gan, Zhenghao Gan +8
Applications based on Large Language Models (LLMs) contains a series of tasks to address real-world problems with boosted capability, which have dynamic demand volumes on diverse b…
Efficient Unified Caching for Accelerating Heterogeneous AI Workloads
Tianze Wang, Yifei Liu, Chen Chen +8
Modern AI clusters, which host diverse workloads like data pre-processing, training and inference, often store the large-volume data in cloud storage and employ caching frameworks…
eLLM: Elastic Memory Management Framework for Efficient LLM Serving
Jiale Xu, Rui Zhang, Yi Xiong +11
Large Language Models are increasingly being deployed in datacenters. Serving these models requires careful memory management, as their memory usage includes static weights, dynami…
Revisiting the Design of In-Memory Dynamic Graph Storage
Jixian Su, Chiyu Hao, Shixuan Sun +7
The effectiveness of in-memory dynamic graph storage (DGS) for supporting concurrent graph read and write queries is crucial for real-time graph analytics and updates. Various meth…
A Microservice Graph Generator with Production Characteristics
Fanrong Du, Jiuchen Shi, Quan Chen +2
A production microservice application may provide multiple services, queries of a service may have different call graphs, and a microservice may be shared across call graphs. It is…
Cicero: Addressing Algorithmic and Architectural Bottlenecks in Neural Rendering by Radiance Warping and Memory Optimizations
Yu Feng, Zihan Liu, Jingwen Leng +2
Neural Radiance Field (NeRF) is widely seen as an alternative to traditional physically-based rendering. However, NeRF has not yet seen its adoption in resource-limited mobile syst…
MMBench: Benchmarking End-to-End Multi-modal DNNs and Understanding Their Hardware-Software Implications
Cheng Xu, Xiaofeng Hou, Jiacheng Liu +9
The explosive growth of various types of big data and advances in AI technologies have catalyzed a new type of workloads called multi-modal DNNs. Multi-modal DNNs are capable of in…