Warming up recurrent neural networks to maximise reachable multistability greatly improves learning

Gaspard Lambrechts, Florent De Geeter, Nicolas Vecoven +2

Training recurrent neural networks is known to be difficult when time dependencies become long. In this work, we show that most standard cells only have one stable equilibrium at i…