To Drop or Not to Drop: Robustness, Consistency and Differential Privacy Properties of Dropout

Prateek Jain, Vivek Kulkarni, Abhradeep Thakurta +1

Training deep belief networks (DBNs) requires optimizing a non-convex function with an extremely large number of parameters. Naturally, existing gradient descent (GD) based methods…