4 papers
Scalar Representations of Neural Network Training Dynamics
Pedro Jiménez-González, Miguel C. Soriano, Lucas Lacasa
Training in artificial neural networks can be viewed as a trajectory evolving through a high-dimensional loss landscape. However, the large number of trainable parameters makes the…
Leveraging chaotic transients in the training of artificial neural networks
Pedro Jiménez-González, Miguel C. Soriano, Lucas Lacasa
Traditional algorithms to optimize artificial neural networks when confronted with a supervised learning task are usually exploitation-type relaxational dynamics such as gradient d…
Adaptive control of recurrent neural networks using conceptors
Guillaume Pourcel, Mirko Goldmann, Ingo Fischer +1
Recurrent Neural Networks excel at predicting and generating complex high-dimensional temporal patterns. Due to their inherent nonlinear dynamics and memory, they can learn unbound…
Dynamical stability and chaos in artificial neural network trajectories along training
Kaloyan Danovski, Miguel C. Soriano, Lucas Lacasa
The process of training an artificial neural network involves iteratively adapting its parameters so as to minimize the error of the network's prediction, when confronted with a le…