32 citations · 41 across the 3 of their papers we have counts for
3 papers
cs.HC2023★ 5 cited
HPCClusterScape: Increasing Transparency and Efficiency of Shared High-Performance Computing Clusters for Large-scale AI Models
Heungseok Park, Aeree Cho, Hyojun Jeon +5
The emergence of large-scale AI models, like GPT-4, has significantly impacted academia and industry, driving the demand for high-performance computing (HPC) to accelerate workload…
cs.CL2021★ 4 cited
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers
Boseop Kim, HyoungSeok Kim, Sang-Woo Lee +34
GPT-3 shows remarkable in-context learning ability of large-scale language models (LMs) trained on hundreds of billion scale data. Here we address some remaining issues less report…
cs.DC2020★ 32 cited
torchgpipe: On-the-fly Pipeline Parallelism for Training Giant Models
Chiheon Kim, Heungsub Lee, Myungryong Jeong +5
We design and implement a ready-to-use library in PyTorch for performing micro-batch pipeline parallelism with checkpointing proposed by GPipe (Huang et al., 2019). In particular,…