Kernels on Sample Sets via Nonparametric Divergence Estimates

Abstract

Most machine learning algorithms, such as classification or regression, treat the individual data point as the object of interest. Here we consider extending machine learning algorithms to operate on groups of data points. We suggest treating a group of data points as an i.i.d. sample set from an underlying feature distribution for that group. Our approach employs kernel machines with a kernel on i.i.d. sample sets of vectors. We define certain kernel functions on pairs of distributions, and then use a nonparametric estimator to consistently estimate those functions based on sample sets. The projection of the estimated Gram matrix to the cone of symmetric positive semi-definite matrices enables us to use kernel machines for classification, regression, anomaly detection, and low-dimensional embedding in the space of distributions. We present several numerical experiments both on real and simulated datasets to demonstrate the advantages of our new approach.

References in corpus (2)

• Probabilistic Latent Semantic Analysis
• Nonparametric Divergence Estimation with Applications to Machine Learning on Distributions

Cited by in corpus (11)

• Machine learning and the physical sciences
• A Machine Learning Approach for Dynamical Mass Measurements of Galaxy Clusters
• Dynamical Mass Measurements of Contaminated Galaxy Clusters Using Machine Learning
• An application of machine learning techniques to galaxy cluster mass estimation using the MACSIS simulations
• Dynamical mass inference of galaxy clusters with neural flows
• Likelihood Non-Gaussianity in Large-Scale Structure Analyses
• Simulation-based inference of dynamical galaxy cluster masses with 3D convolutional neural networks
• Extracting Biomolecular Interactions Using Semantic Parsing of Biomedical Text
• Direct estimation of density functionals using a polynomial basis
• Distance Measure Machines
• Counterfactual Distribution Regression for Structured Inference