ZENN: A Thermodynamics-Inspired Computational Framework for Heterogeneous Data-Driven Modeling

Abstract

Traditional entropy-based methods - such as cross-entropy loss in classification problems - have long been essential tools for representing the information uncertainty and physical disorder in data and for developing artificial intelligence algorithms. However, the rapid growth of data across various domains has introduced new challenges, particularly the integration of heterogeneous datasets with intrinsic disparities. To address this, we introduce a zentropy-enhanced neural network (ZENN), extending zentropy theory into the data science domain via intrinsic entropy, enabling more effective learning from heterogeneous data sources. ZENN simultaneously learns both energy and intrinsic entropy components, capturing the underlying structure of multi-source data. To support this, we redesign the neural network architecture to better reflect the intrinsic properties and variability inherent in diverse datasets. We demonstrate the effectiveness of ZENN on classification tasks and energy landscape reconstructions, showing its superior generalization capabilities and robustness-particularly in predicting high-order derivatives. ZENN demonstrates superior generalization by introducing a learnable temperature variable that models latent multi-source heterogeneity, allowing it to surpass state-of-the-art models on CIFAR-10/100, BBCNews, and AGNews. As a practical application in materials science, we employ ZENN to reconstruct the Helmholtz energy landscape of Fe${}_3$Pt using data generated from density functional theory (DFT) and capture key material behaviors, including negative thermal expansion and the critical point in the temperature-pressure space. Overall, this work presents a zentropy-grounded framework for data-driven machine learning, positioning ZENN as a versatile and robust approach for scientific problems involving complex, heterogeneous datasets.

10 pages, 4 figures

References in corpus (9)

• Thermodynamical Property of Entanglement Entropy for Excited States
• Thermodynamics and its Prediction and CALPHAD Modeling: Review, State of the Art, and Perspectives
• Zentropy Theory for Positive and Negative Thermal Expansions
• Thermodynamic Fluctuations in Magnetic Phase Transition: Invar as a Prototype
• HomPINNs: homotopy physics-informed neural networks for solving the inverse problems of nonlinear differential equations with multiple solutions
• EPR-Net: Constructing non-equilibrium potential landscape via a variational force projection formulation
• A Systematic Computational Framework for Practical Identifiability Analysis in Mathematical Models Arising from Biology
• Revealing Symmetry-Broken Superconducting Configurations by Density Functional Theory
• Companion-Based Multi-Level Finite Element Method for Computing Multiple Solutions of Nonlinear Differential Equations