Phase Diagram of Dropout for Two-Layer Neural Networks in the Mean-Field Regime

Lénaïc Chizat, Pierre Marion, Yerkin Yesbay

Dropout is a standard training technique for neural networks that consists of randomly deactivating units at each step of their gradient-based training. It is known to improve perf…