Transfer entropy and O-information to detect grokking in tensor network multi-class classification problems

Abstract

Quantum-enhanced machine learning, encompassing both quantum algorithms and quantum-inspired classical methods such as tensor networks, offers promising tools for extracting structure from complex, high-dimensional data. In this work, we study the training dynamics of Matrix Product State (MPS) classifiers applied to three-class problems, using both fashion MNIST and hyper-spectral satellite imagery as representative datasets. We investigate the phenomenon of grokking, where generalization emerges suddenly after memorization, by tracking entanglement entropy, local magnetization, and model performance across training sweeps. Additionally, we employ information theory tools to gain deeper insights: transfer entropy is used to reveal causal dependencies between label-specific quantum masks, while O-information captures the shift from synergistic to redundant correlations among class outputs. Our results show that grokking in the fashion MNIST task coincides with a sharp entanglement transition and a peak in redundant information, whereas the overfitted hyper-spectral model retains synergistic, disordered behavior. These findings highlight the relevance of high-order information dynamics in quantum-inspired learning and emphasize the distinct learning behaviors that emerge in multi-class classification, offering a principled framework to interpret generalization in quantum machine learning architectures.

References in corpus (30)

• Measuring Information Transfer
• The density-matrix renormalization group in the age of matrix product states
• Estimating Mutual Information
• The density-matrix renormalization group
• Quantum circuits with many photons on a programmable nanophotonic chip
• A Mean Field View of the Landscape of Two-Layers Neural Networks
• Generalization in quantum machine learning from few training data
• A generative modeling approach for benchmarking and training shallow quantum circuits
• Towards Quantum Machine Learning with Tensor Networks
• Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information
• Neural-Network Quantum States, String-Bond States, and Chiral Topological States
• Measurement and entanglement phase transitions in all-to-all quantum circuits, on quantum trees, and in Landau-Ginsburg theory
• Molecular Docking with Gaussian Boson Sampling
• Encoding of Matrix Product States into Quantum Circuits of One- and Two-Qubit Gates
• Isometric Tensor Network States in Two Dimensions
• Tree Tensor Networks for Generative Modeling
• Understanding quantum machine learning also requires rethinking generalization
• Synergy Between Quantum Circuits and Tensor Networks: Short-cutting the Race to Practical Quantum Advantage
• Generalization despite overfitting in quantum machine learning models
• A universal programmable Gaussian Boson Sampler for drug discovery
• A Framework for Demonstrating Practical Quantum Advantage: Racing Quantum against Classical Generative Models
• Dynamical simulation via quantum machine learning with provable generalization
• Entanglement entropy production in Quantum Neural Networks
• Dynamical quantum phase transitions of the Schwinger model: real-time dynamics on IBM Quantum
• Q-Seg: Quantum Annealing-Based Unsupervised Image Segmentation
• Measurement-induced entanglement phase transitions in variational quantum circuits
• On the descriptive power of Neural-Networks as constrained Tensor Networks with exponentially large bond dimension
• Machine learning with tree tensor networks, CP rank constraints, and tensor dropout
• Hybrid Tree Tensor Networks for quantum simulation
• Quantum error mitigation in optimized circuits for particle-density correlations in real-time dynamics of the Schwinger model