4 papers
DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference
Dengke Han
Large Mixture-of-Experts (MoE) language models are attractive for end-device deployment because only a small subset of experts is active per token, but their routed expert weights…
A Systematic Characterization of LLM Inference on GPUs
Haonan Wang, Xuxin Xiao, Mingyu Yan +8
This work presents a systematic characterization of Large Language Model (LLM) inference to address fragmented understanding. Through comprehensive experiments, we establish a four…
TLV-HGNN: Thinking Like a Vertex for Memory-efficient HGNN Inference
Dengke Han, Duo Wang, Mingyu Yan +2
Heterogeneous graph neural networks (HGNNs) excel at processing heterogeneous graph data and are widely applied in critical domains. In HGNN inference, the neighbor aggregation sta…
Accelerating GNN Training through Locality-aware Dropout and Merge
Gongjian Sun, Mingyu Yan, Dengke Han +4
Graph Neural Networks (GNNs) have demonstrated significant success in graph learning and are widely adopted across various critical domains. However, the irregular connectivity bet…