4 papers · 1 filter
When Do Graph Foundation Models Transfer? A Data-Centric Theory
Jiajun Zhu, Ying Chen, Peihao Wang +4
Graph foundation models (GFMs) aim to reuse a single backbone across diverse graph domains, yet their transfer is often uneven and can exhibit negative transfer. While most prior w…
Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration
Jucheng Shen, Gaurav Sarkar, Yeonju Ro +4
We present CadLLM, a training-free method to accelerate the inference throughput of diffusion-based LLMs (dLLMs). We first investigate the dynamic nature of token unmasking confide…
On-the-Fly Adaptive Distillation of Transformer to Dual-State Linear Attention
Yeonju Ro, Zhenyu Zhang, Souvik Kundu +2
Large language models (LLMs) excel at capturing global token dependencies via self-attention but face prohibitive compute and memory costs on lengthy inputs. While sub-quadratic me…
HALoS: Hierarchical Asynchronous Local SGD over Slow Networks for Geo-Distributed Large Language Model Training
Geon-Woo Kim, Junbo Li, Shashidhar Gandham +5
Training large language models (LLMs) increasingly relies on geographically distributed accelerators, causing prohibitive communication costs across regions and uneven utilization…