Learning from Distributions via Support Measure Machines

Abstract

This paper presents a kernel-based discriminative learning framework on probability measures. Rather than relying on large collections of vectorial training examples, our framework learns using a collection of probability distributions that have been constructed to meaningfully represent training data. By representing these probability distributions as mean embeddings in the reproducing kernel Hilbert space (RKHS), we are able to apply many standard kernel-based learning techniques in straightforward fashion. To accomplish this, we construct a generalization of the support vector machine (SVM) called a support measure machine (SMM). Our analyses of SMMs provides several insights into their relationship to traditional SVMs. Based on such insights, we propose a flexible SVM (Flex-SVM) that places different kernel functions on each training example. Experimental results on both synthetic and real-world data demonstrate the effectiveness of our proposed framework.

Advances in Neural Information Processing Systems 25

References in corpus (2)

• The representer theorem for Hilbert spaces: a necessary and sufficient condition
• Generative and Latent Mean Map Kernels

Cited by in corpus (30)

• Domain Generalization via Invariant Feature Representation
• Kernel Mean Embedding of Distributions: A Review and Beyond
• Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
• Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
• Persistence weighted Gaussian kernel for topological data analysis
• Operator-valued Kernels for Learning from Functional Response Data
• Towards a Learning Theory of Cause-Effect Inference
• On the Error of Random Fourier Features
• On Classification with Bags, Groups and Sets
• A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings
• Stochastic Bandits with Context Distributions
• MONK -- Outlier-Robust Mean Embedding Estimation by Median-of-Means
• The Randomized Causation Coefficient
• Approximation capability of neural networks on spaces of probability measures and tree-structured domains
• A Classification Framework for Partially Observed Dynamical Systems
• Provable Fictitious Play for General Mean-Field Games
• Kernel Mean Estimation via Spectral Filtering
• Kernel-based ANOVA decomposition and Shapley effects -- Application to global sensitivity analysis
• Higher Order Kernel Mean Embeddings to Capture Filtrations of Stochastic Processes
• Distribution Regression for Sequential Data
• Nested Multiple Instance Learning in Modelling of HTTP network traffic
• Towards Self-similarity Consistency and Feature Discrimination for Unsupervised Domain Adaptation
• Stepwise Model Selection for Sequence Prediction via Deep Kernel Learning
• Nonparametric Composite Hypothesis Testing in an Asymptotic Regime
• Kernel Hypothesis Testing with Set-valued Data
• High-Dimensional Multi-Task Averaging and Application to Kernel Mean Embedding
• Nonparametric Topological Layers in Neural Networks
• Isolation Distributional Kernel: A New Tool for Point & Group Anomaly Detection
• Conditional Deep Gaussian Processes: empirical Bayes hyperdata learning
• Kernel Mean Estimation by Marginalized Corrupted Distributions