AdamD: Improved bias-correction in Adam

John St John

Here I present a small update to the bias-correction term in the Adam optimizer that has the advantage of making smaller gradient updates in the first several steps of training. Wi…