activity
20182022
most citedtorchgpipe: On-the-fly Pipeline Parallelism for Training Giant Models

32 citations · 85 across the 11 of their papers we have counts for

collaborators

15 papers

cs.LG20222 cited

LECO: Learnable Episodic Count for Task-Specific Intrinsic Reward

Daejin Jo, Sungwoong Kim, Daniel Wontae Nam +4

Episodic count has been widely used to design a simple yet effective intrinsic motivation for reinforcement learning with a sparse reward. However, the use of episodic count in a h…

cs.CL2022

Selective Token Generation for Few-shot Natural Language Generation

Daejin Jo, Taehwan Kwon, Eun-Sol Kim +1

Natural language modeling with limited training data is a challenging problem, and many algorithms make use of large-scale pretrained language models (PLMs) for this due to its gre…

cs.LG20221 cited

Insights From the NeurIPS 2021 NetHack Challenge

Eric Hambro, Sharada Mohanty, Dmitrii Babaev +26

In this report, we summarize the takeaways from the first NeurIPS 2021 NetHack Challenge. Participants were tasked with developing a program or agent that can win (i.e., 'ascend' i…

cs.LG20218 cited

Automated Learning Rate Scheduler for Large-batch Training

Chiheon Kim, Saehoon Kim, Jongmin Kim +2

Large-batch training has been essential in leveraging large-scale datasets and models in deep learning. While it is computationally beneficial to use large batch sizes, it often re…

cs.LG20215 cited

Hybrid Generative-Contrastive Representation Learning

Saehoon Kim, Sungwoong Kim, Juho Lee

Unsupervised representation learning has recently received lots of interest due to its powerful generalizability through effectively leveraging large-scale unlabeled data. There ar…

cs.CV2021

Spatially Consistent Representation Learning

Byungseok Roh, Wuhyun Shin, Ildoo Kim +1

Self-supervised learning has been widely used to obtain transferrable representations from unlabeled images. Especially, recent contrastive learning methods have shown impressive p…