9 papers
Homebot: A Personal AI Agent for Conversational Home Assistance and Automation
Shengyuan Ye, Yixin Zhang, Han Liang +3
\texttt{Homebot} is a locally deployable AI agent for conversational household assistance and automation. It accepts voice and instant-messaging requests through a shared runtime t…
Communication-Aware Placement and Pruning for Efficient Mixture-of-Experts Inference
Xiao Shi, Yingying Sun, Jiangsu Du +2
As MoE models scale to hundreds of experts, placement and pruning decisions increasingly dictate communication volume, affecting the performance of distributed inference across GPU…
AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System
Fengyao Bai, Hongbin Zhang, Zhitao Chen +3
High-throughput inference serving is essential for applications built on large language models (LLMs). Existing serving frameworks reduce request-level and batch-level bubbles thro…
PipeMax: Enhancing Offline LLM Inference on Commodity GPU Servers
Hongbin Zhang, Taosheng Wei, Jiazhi Jiang +3
Offline LLM inference seeks to maximize request processing under fixed budgets, making commodity GPU servers a promising choice. However, prior work typically considers offloading…
Resource-Efficient Personal Large Language Models Fine-Tuning with Collaborative Edge Computing
Shengyuan Ye, Bei Ouyang, Tianyi Qian +6
Large language models (LLMs) have unlocked a plethora of powerful applications at the network edge, such as intelligent personal assistants. Data privacy and security concerns have…
TD-Pipe: Temporally-Disaggregated Pipeline Parallelism Architecture for High-Throughput LLM Inference
Hongbin Zhang, Taosheng Wei, Zhenyi Zheng +3
As the model size continuously increases, pipeline parallelism shows great promise in throughput-oriented LLM inference due to its low demand on communications. However, imbalanced…