Learning Temporal Point Processes via Reinforcement Learning

Abstract

Social goods, such as healthcare, smart city, and information networks, often produce ordered event data in continuous time. The generative processes of these event data can be very complex, requiring flexible models to capture their dynamics. Temporal point processes offer an elegant framework for modeling event data without discretizing the time. However, the existing maximum-likelihood-estimation (MLE) learning paradigm requires hand-crafting the intensity function beforehand and cannot directly monitor the goodness-of-fit of the estimated model in the process of training. To alleviate the risk of model-misspecification in MLE, we propose to generate samples from the generative model and monitor the quality of the samples in the process of training until the samples and the real data are indistinguishable. We take inspiration from reinforcement learning (RL) and treat the generation of each event as the action taken by a stochastic policy. We parameterize the policy as a flexible recurrent neural network and gradually improve the policy to mimic the observed event distribution. Since the reward function is unknown in this setting, we uncover an analytic and nonparametric form of the reward function using an inverse reinforcement learning formulation. This new RL framework allows us to derive an efficient policy gradient algorithm for learning flexible point process models, and we show that it performs well in both synthetic and real data.

arXiv commentAdd code link

Cited by in corpus (26)

• Fully Neural Network based Model for General Temporal Point Processes
• Time2Vec: Learning a Vector Representation of Time
• Finding Patient Zero: Learning Contagion Source with Graph Neural Networks
• Transformer Hawkes Process
• Intensity-Free Learning of Temporal Point Processes
• Self-Attentive Hawkes Processes
• Deep Learning for Insider Threat Detection: Review, Challenges and Opportunities
• Modeling of Spatio-Temporal Hawkes Processes with Randomized Kernels
• Thinking While Moving: Deep Reinforcement Learning with Concurrent Control
• On Computation and Generalization of Generative Adversarial Imitation Learning
• Insider Threat Detection via Hierarchical Neural Temporal Point Processes
• An Empirical Study: Extensive Deep Temporal Point Process
• Spatio-Temporal Point Processes with Attention for Traffic Congestion Event Modeling
• Imitation Learning of Neural Spatio-Temporal Point Processes
• Detecting Anomalous Event Sequences with Temporal Point Processes
• Point Process Flows
• Deep Fourier Kernel for Self-Attentive Point Processes
• Reinforcement Learning with Policy Mixture Model for Temporal Point Processes Clustering
• Understanding the Spread of COVID-19 Epidemic: A Spatio-Temporal Point Process View
• Event Outlier Detection in Continuous Time
• Sequential Adversarial Anomaly Detection for One-Class Event Data
• A Temporal Kernel Approach for Deep Learning with Continuous-time Information
• UNIPoint: Universally Approximating Point Processes Intensities
• ABC Learning of Hawkes Processes with Missing or Noisy Event Times
• Neural Spectral Marked Point Processes
• Deep Convolution for Irregularly Sampled Temporal Point Clouds