Showing 2026Show all
2 papers · 1 filter
cs.LG2026
Distributed Hybrid Parallelism for Large Language Models: Comparative Study and System Design Guide
Hossam Amer, Rezaul Karim, Ali Pourranjbar +3
With the rapid growth of large language models (LLMs), a wide range of methods have been developed to distribute computation and memory across hardware devices for efficient traini…
cs.LG2026
EPAS: Efficient Training with Progressive Activation Sharing
Rezaul Karim, Maryam Dialameh, Yang Liu +2
We present a novel method for Efficient training with Progressive Activation Sharing (EPAS). This method bridges progressive training paradigm with the phenomenon of redundant QK (…