2 papers
cs.DC2025
FpgaHub: Fpga-centric Hyper-heterogeneous Computing Platform for Big Data Analytics
Zeke Wang, Jie Zhang, Hongjing Huang +10
Modern data analytics requires a huge amount of computing power and processes a massive amount of data. At the same time, the underlying computing platform is becoming much more he…
cs.DC2024
DisDP: Disaggregating Compute, Network, and Storage for Model-Sharded Data-Parallel Training
Mo Sun, Zihan Yang, Changyue Liao +5
Model-sharded data parallelism (MSDP), e.g., ZeRO, evenly shards the model states across all GPUs, and thus has been widely adopted by LLM pre-training, such as Llama and DeepSeek,…