5 papers
SlideDP: Scaling Host-Resident LLM Fine-Tuning Across Multiple GPUs
Ruijia Yang, Shiyuan Lin, Yulong Ao +5
Host-resident layer streaming enables full-parameter LLM fine-tuning beyond GPU memory, but data-parallel ranks compete for shared host resources. Replicated transfers amplify traf…
Efficient Hyperparameter Optimization for LLM Reinforcement Learning
Minping Chen, Bowen Xiao, Du Liang +2
Reinforcement learning (RL) for large language models (LLMs) is highly sensitive to hyperparameter configurations, making hyperparameter optimization (HPO) essential yet computatio…
Quasar: Quantized Self-Speculative Acceleration for Rapid Inference via Memory-Efficient Verification
Guang Huang, Zeyi Wen
Speculative Decoding (SD) has emerged as a premier technique for accelerating Large Language Model (LLM) inference by decoupling token generation into rapid drafting and parallel v…
An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU
Ruijia Yang, Zeyi Wen
Fine-tuning Large Language Models (LLMs) has become essential for domain adaptation, but its memory-intensive property exceeds the capabilities of most GPUs. To address this challe…
Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performance
Borui Xu, Yao Chen, Zeyi Wen +2
The increasing demand for efficient summarization tools in resource-constrained environments highlights the need for effective solutions. While large language models (LLMs) deliver…