8 papers
When Does Disaggregation Pay? Simulating Prefill--Decode--Attention--FFN Specialization for Agentic LLM Inference
Przemyslaw Forys, Haoran Wu, Can Xiao +8
Agentic inference now dominates the LLM inference landscape, requiring LLMs to actively engage in multi-turn interactions with tool-calling capabilities. This introduces a more com…
Open Problems in Constitutional Preference Reconstruction
Eleanor Clifford, Michael Amir, Arduin Findeis +2
Pairwise preference data is widely used for training and evaluating language models (e.g., RLHF), but each datapoint records a \emph{choice}, not the rationale behind it. Methods s…
MemExplorer: Navigating the Heterogeneous Memory Design Space for Agentic Inference NPUs
Haoran Wu, Zeyu Cao, Yao Lai +16
Emerging agentic large language model (LLM) workloads are driving rapidly growing demand for memory capacity and bandwidth. Different phases of inference, such as prefill and decod…
Heterogeneous Computing: The Key to Powering the Future of AI Agent Inference
Yiren Zhao, Junyi Liu
AI agent inference is driving an inference heavy datacenter future and exposes bottlenecks beyond compute - especially memory capacity, memory bandwidth and high-speed interconnect…
NPU Design for Diffusion Language Model Inference
Binglei Lou, Haoran Wu, Kevin Lau +9
Diffusion-based LLMs (dLLMs) fundamentally depart from traditional autoregressive (AR) LLM inference: they leverage bidirectional attention, block-wise KV cache refreshing, cross-s…
Combating the Memory Walls: Optimization Pathways for Long-Context Agentic LLM Inference
Haoran Wu, Can Xiao, Jiayi Nie +15
LLMs now form the backbone of AI agents across a diverse range of applications, including tool use, command-line interfaces, and web or computer interaction. These agentic LLM infe…
AMPLE: Event-Driven Accelerator for Mixed-Precision Inference of Graph Neural Networks
Pedro Gimenes, Yiren Zhao, George Constantinides
Graph Neural Networks (GNNs) have recently gained attention due to their performance on non-Euclidean data. The use of custom hardware architectures proves particularly beneficial…
Revisiting Embeddings for Graph Neural Networks
S. Purchase, A. Zhao, R. D. Mullins
Current graph representation learning techniques use Graph Neural Networks (GNNs) to extract features from dataset embeddings. In this work, we examine the quality of these embeddi…