Neural Processes

Abstract

A neural network (NN) is a parameterised function that can be tuned via gradient descent to approximate a labelled collection of data with high precision. A Gaussian process (GP), on the other hand, is a probabilistic model that defines a distribution over possible functions, and is updated in light of data via the rules of probabilistic inference. GPs are probabilistic, data-efficient and flexible, however they are also computationally intensive and thus limited in their applicability. We introduce a class of neural latent variable models which we call Neural Processes (NPs), combining the best of both worlds. Like GPs, NPs define distributions over functions, are capable of rapid adaptation to new observations, and can estimate the uncertainty in their predictions. Like NNs, NPs are computationally efficient during training and evaluation but also learn to adapt their priors to data. We demonstrate the performance of NPs on a range of learning tasks, including regression and optimisation, and compare and contrast with related models in the literature.

Cited by in corpus (12)

• Deep Bayesian Inversion
• Normalizing Flow-based Neural Process for Few-Shot Knowledge Graph Completion
• CDRNP: Cross-Domain Recommendation to Cold-Start Users via Neural Process
• Graph Sequential Neural ODE Process for Link Prediction on Dynamic and Sparse Graphs
• Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel
• Full Scaling Automation for Sustainable Development of Green Data Centers
• Neural Clustering Processes
• SCHA-VAE: Hierarchical Context Aggregation for Few-Shot Generation
• Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes
• Few-Shot Learning Patterns in Financial Time-Series for Trend-Following Strategies
• Continual Learning of Multi-modal Dynamics with External Memory
• MFPC-Net: Multi-fidelity Physics-Constrained Neural Process