Optimization dynamics of Transformer backflow neural quantum states for the two-dimensional Hubbard model
arXiv:2607.14875
The paper studies how various hyperparameters affect the training dynamics of a Transformer‑based neural quantum state ansatz for the two‑dimensional Hubbard model, using a multi‑stage optimization workflow that includes backflow initialization, supervised pre‑training, and variational Monte Carlo with the MARCH optimizer.
Abstract
Building on the multi-determinant Transformer backflow neural quantum state (NQS) ansatz and the associated multi-stage training workflow for the doped two-dimensional Hubbard model, we investigate how the optimization dynamics of the NQS depend on several key optimization and architectural hyperparameters. The workflow consists of neural-network backflow (NNB) initialization, supervised Transformer pre-training, and main energy optimization using the Moment-Adaptive ReConfiguration Heuristic (MARCH) within variational Monte Carlo. Using the doped periodic Hubbard model at as a baseline, we examine how the update-norm threshold, Transformer width, number of determinant channels, and Monte Carlo batch size affect convergence. We find that a moderate update constraint improves the efficiency of MARCH optimization, larger Transformer width and more determinant channels improve the expressive capacity of the ansatz, and larger Monte Carlo batches reduce sampling noise in the update direction. We further test the same workflow at half filling, weaker interaction strength, open boundary conditions, and on a larger doped lattice. These results identify practical optimization trends for Transformer backflow NQSs and highlight the balance between ansatz expressivity, MARCH update stability, and Monte Carlo sampling quality.
11 pages, 4 figures, 1 table