823 citations · 1k across the 10 of their papers we have counts for
3 papers · 1 filter
CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning
Hung Le, Yue Wang, Akhilesh Deepak Gotmare +2
Program synthesis or code generation aims to generate a program that satisfies a problem specification. Recent approaches using large-scale pretrained language models (LMs) have sh…
A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation
Akhilesh Gotmare, Nitish Shirish Keskar, Caiming Xiong +1
The convergence rate and final performance of common deep learning models have significantly benefited from heuristics such as learning rate schedules, knowledge distillation, skip…
Using Mode Connectivity for Loss Landscape Analysis
Akhilesh Gotmare, Nitish Shirish Keskar, Caiming Xiong +1
Mode connectivity is a recently introduced frame- work that empirically establishes the connected- ness of minima by finding a high accuracy curve between two independently trained…