4 papers
Sampling at intermediate temperatures is optimal for training large language models in protein structure prediction
L. Ghiringhelli, A. Zambon, G. Tiana
Using a statistical mechanics framework, we investigate the parameter space of transformer models trained on protein sequence data. We sample the loss landscape at varying temperat…
Controlled Langevin Dynamics for Sampling of Feedforward Neural Networks Trained with Minibatches
Alessandro Zambon, Francesca Caruso, Riccardo Zecchina +1
Sampling the parameter space of artificial neural networks according to a Boltzmann distribution provides insight into the geometry of low-loss solutions and offers an alternative…
Sampling the space of solutions of an artificial neural network
Alessandro Zambon, Enrico M. Malatesta, Guido Tiana +1
The weight space of an artificial neural network can be systematically explored using tools from statistical mechanics. We employ a combination of a hybrid Monte Carlo algorithm wh…
Structure of the space of folding protein sequences defined by large language models
A. Zambon, R. Zecchina, G. Tiana
Proteins populate a manifold in the high-dimensional sequence space whose geometrical structure guides their natural evolution. Leveraging recently-developed structure prediction t…