Knowledge-integrated AutoEncoder Model
arXiv:2303.06721 · doi:10.1016/j.eswa.2024.124108
Abstract
Data encoding is a common and central operation in most data analysis tasks. The performance of other models downstream in the computational process highly depends on the quality of data encoding. One of the most powerful ways to encode data is using the neural network AutoEncoder (AE) architecture. However, the developers of AE cannot easily influence the produced embedding space, as it is usually treated as a black box technique. This means the embedding space is uncontrollable and does not necessarily possess the properties desired for downstream tasks. This paper introduces a novel approach for developing AE models that can integrate external knowledge sources into the learning process, possibly leading to more accurate results. The proposed Knowledge-integrated AutoEncoder (KiAE) model can leverage domain-specific information to make sure the desired distance and neighborhood properties between samples are preservative in the embedding space. The proposed model is evaluated on three large-scale datasets from three scientific fields and is compared to nine existing encoding models. The results demonstrate that the KiAE model effectively captures the underlying structures and relationships between the input data and external knowledge, meaning it generates a more useful representation. This leads to outperforming the rest of the models in terms of reconstruction accuracy.
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
- Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
- A Survey of Deep Meta-Learning
- Searching for New Physics with Deep Autoencoders
- Unsupervised Discovery of Interpretable Directions in the GAN Latent Space
- A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledge
- Autoencoders for unsupervised anomaly detection in high energy physics
- Deep autoencoders for physics-constrained data-driven nonlinear materials modeling
- DeepMDP: Learning Continuous Latent Space Models for Representation Learning
- Cancer-inspired Genomics Mapper Model for the Generation of Synthetic DNA Sequences with Desired Genomics Signatures