Conditional Temporal Neural Processes with Covariance Loss
arXiv:2504.00794
Abstract
We introduce a novel loss function, Covariance Loss, which is conceptually equivalent to conditional neural processes and has a form of regularization so that is applicable to many kinds of neural networks. With the proposed loss, mappings from input variables to target variables are highly affected by dependencies of target variables as well as mean activation and mean dependencies of input and target variables. This nature enables the resulting neural networks to become more robust to noisy observations and recapture missing dependencies from prior information. In order to show the validity of the proposed loss, we conduct extensive sets of experiments on real-world datasets with state-of-the-art models and discuss the benefits and drawbacks of the proposed Covariance Loss.
11 pages, 18 figures
References in corpus (6)
- Sequence to Sequence Learning with Neural Networks
- GMNN: Graph Markov Neural Networks
- Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation
- Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes
- Deep CNNs Meet Global Covariance Pooling: Better Representation and Generalization
- Deep Convolutional Networks as shallow Gaussian Processes