Stochastic Gradient Descent and Anomaly of Variance-flatness Relation in Artificial Neural Networks

Xia Xiong, Yong-Cong Chen, Chunxiao Shi +1

Stochastic gradient descent (SGD), a widely used algorithm in deep-learning neural networks has attracted continuing studies for the theoretical principles behind its success. A re…