3 papers
physics.chem-ph2024
Very-Large-Scale GPU-Accelerated Nuclear Gradient of Time-Dependent Density Functional Theory with Tamm-Dancoff Approximation and Range-Separated Hybrid Functionals
Inkoo Kim, Daun Jeong, Leah Weisburn +11
Modern graphics processing units (GPUs) provide an unprecedented level of computing power. In this study, we present a high-performance, multi-GPU implementation of the analytical…
cs.DC2024
Pipette: Automatic Fine-grained Large Language Model Training Configurator for Real-World Clusters
Jinkyu Yim, Jaeyong Song, Yerim Choi +4
Training large language models (LLMs) is known to be challenging because of the huge computational and memory capacity requirements. To address these issues, it is common to use a…
cs.LG2023
Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression
Jaeyong Song, Jinkyu Yim, Jaewon Jung +4
In training of modern large natural language processing (NLP) models, it has become a common practice to split models using 3D parallelism to multiple GPUs. Such technique, however…