2 citations · 3 across the 9 of their papers we have counts for
10 papers
OServe: Accelerating LLM Serving via Spatial-Temporal Workload Orchestration
Youhe Jiang, Fangcheng Fu, Taiyi Wang +2
Serving Large Language Models (LLMs) can benefit immensely from parallelizing both the model and input requests across multiple devices, but incoming workloads exhibit substantial…
Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUs
Guoliang He, Youhe Jiang, Wencong Xiao +8
The scaling law for large language models (LLMs) depicts that the path towards machine intelligence necessitates training at large scale. Thus, companies continuously build large-s…
LMDeploy Accelerates Mixed-Precision LLM Inference with TurboMind
Li Zhang, Youhe Jiang, Guoliang He +6
Mixed-precision inference techniques reduce the memory and computational demands of Large Language Models (LLMs) by applying hybrid precision formats to model weights, activations,…
Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
Youhe Jiang, Fangcheng Fu, Xiaozhe Yao +6
Recent advancements in Large Language Models (LLMs) have led to increasingly diverse requests, accompanied with varying resource (compute and memory) demands to serve them. However…
CuAsmRL: Optimizing GPU SASS Schedules via Deep Reinforcement Learning
Guoliang He, Eiko Yoneki
Large language models (LLMs) are remarked by their substantial computational requirements. To mitigate the cost, researchers develop specialized CUDA kernels, which often fuse seve…
PromptKeeper: Safeguarding System Prompts for LLMs
Zhifeng Jiang, Zhihua Jin, Guoliang He
System prompts are widely used to guide the outputs of large language models (LLMs). These prompts often contain business logic and sensitive information, making their protection e…
Optimizing Tensor Computation Graphs with Equality Saturation and Monte Carlo Tree Search
Jakob Hartmann, Guoliang He, Eiko Yoneki
The real-world effectiveness of deep neural networks often depends on their latency, thereby necessitating optimization techniques that can reduce a model's inference time while pr…
vPALs: Towards Verified Performance-aware Learning System For Resource Management
Guoliang He, Gingfung Yeung, Sheriffo Ceesay +1
Accurately predicting task performance at runtime in a cluster is advantageous for a resource management system to determine whether a task should be migrated due to performance de…
SIP: Autotuning GPU Native Schedules via Stochastic Instruction Perturbation
Guoliang He, Eiko Yoneki
Large language models (LLMs) have become a significant workload since their appearance. However, they are also computationally expensive as they have billions of parameters and are…
X-RLflow: Graph Reinforcement Learning for Neural Network Subgraphs Transformation
Guoliang He, Sean Parker, Eiko Yoneki
Tensor graph superoptimisation systems perform a sequence of subgraph substitution to neural networks, to find the optimal computation graph structure. Such a graph transformation…