4 papers
End-to-End Dynamic Sparsity for Resource-Adaptive LLM Inference
Yuhang Chen, Jinhao Duan, Ruichen Zhang +11
Large Language Models (LLMs) inference is typically deployed under a static resource assumption, where models execute a fixed computational graph regardless of the runtime environm…
Bifocal Diffusion Language Models: Asymmetric Bidirectional Context for Parallel Generation
Yuhang Chen, Xianfeng Wu, Jinhao Duan +11
Discrete diffusion language models (dLLMs) recover masked tokens in parallel, offering significant speedups over autoregressive (AR) generation. However, such promising frameworks…
External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation
Mingfu Liang, Xi Liu, Rong Jin +104
Ads recommendation is a prominent service of online advertising systems and has been actively studied. Recent studies indicate that scaling-up and advanced design of the recommenda…
APOLLO: SGD-like Memory, AdamW-level Performance
Hanqing Zhu, Zhenyu Zhang, Wenyan Cong +7
Large language models (LLMs) are notoriously memory-intensive during training, particularly with the popular AdamW optimizer. This memory burden necessitates using more or higher-e…