2 papers
cs.LG2024
No Request Left Behind: Tackling Heterogeneity in Long-Context LLM Inference with Medha
Amey Agrawal, Haoran Qiu, Junda Chen +6
Deploying million-token Large Language Models (LLMs) is challenging because production workloads are highly heterogeneous, mixing short queries and long documents. This heterogenei…
cs.LG2023
STRIDE: Structure and Embedding Distillation with Attention for Graph Neural Networks
Anshul Ahluwalia, Payman Behnam, Rohit Das +4
Recent advancements in Graph Neural Networks (GNNs) have led to increased model sizes to enhance their capacity and accuracy. Such large models incur high memory usage, latency, an…