2 papers
cs.CL2023
Holmes: Towards Distributed Training Across Clusters with Heterogeneous NIC Environment
Fei Yang, Shuang Peng, Ning Sun +5
Large language models (LLMs) such as GPT-3, OPT, and LLaMA have demonstrated remarkable accuracy in a wide range of tasks. However, training these models can incur significant expe…
cs.DC2023
HiQ -- A Declarative, Non-intrusive, Dynamic and Transparent Observability and Optimization System
Fuheng Wu, Ivan Davchev, Jun Qian
This paper proposes a non-intrusive, declarative, dynamic and transparent system called `HiQ` to track Python program runtime information without compromising on the run-time syste…