Multiple Descents in Deep Learning as a Sequence of Order-Chaos Transitions in LSTM Networks

Wenbo Wei, Fan Xu, Nicholas Chong Jia Le +2

We observe a novel `multiple-descent' phenomenon during the learning process of a recurrent neural network called long-short-term memory (LSTM) networks during its training on real…