Deep Sets

Abstract

We study the problem of designing models for machine learning tasks defined on \emph{sets}. In contrast to traditional approach of operating on fixed dimensional vectors, we consider objective functions defined on sets that are invariant to permutations. Such problems are widespread, ranging from estimation of population statistics \cite{poczos13aistats}, to anomaly detection in piezometer data of embankment dams \cite{Jung15Exploration}, to cosmology \cite{Ntampaka16Dynamical,Ravanbakhsh16ICML1}. Our main theorem characterizes the permutation invariant functions and provides a family of functions to which any permutation invariant objective function must belong. This family of functions has a special structure which enables us to design a deep network architecture that can operate on sets and which can be deployed on a variety of scenarios including both unsupervised and supervised learning tasks. We also derive the necessary and sufficient conditions for permutation equivariance in deep models. We demonstrate the applicability of our method on population statistic estimation, point cloud classification, set expansion, and outlier detection.

NIPS 2017

Cited by in corpus (17)

• The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider
• Neural Networks for Full Phase-space Reweighting and Parameter Tuning
• Variational Monte Carlo calculations of $\mathbf{A}\le 4$ nuclei with an artificial neural-network correlator ansatz
• EPiC-GAN: Equivariant Point Cloud Generation for Particle Jets
• PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics
• HANNA: Hard-constraint Neural Network for Consistent Activity Coefficient Prediction
• Equivariant Energy Flow Networks for Jet Tagging
• Nuclei with up to $\mathbf{A}=6$ nucleons with artificial neural network wave functions
• ν-Flows: Conditional Neutrino Regression
• Machine Learning in High Energy Physics: A review of heavy-flavor jet tagging at the LHC
• High-dimensional and Permutation Invariant Anomaly Detection
• A Lorentz-Equivariant Transformer for All of the LHC
• Robust partial Fourier reconstruction for diffusion-weighted imaging using a recurrent convolutional neural network
• Status and Prospects of Exotic Hadrons at Belle II
• RecipeMind: Guiding Ingredient Choices from Food Pairing to Recipe Completion using Cascaded Set Transformer
• Jet energy calibration with deep learning as a Kubeflow pipeline
• 2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets