Deep Reinforcement Learning of Marked Temporal Point Processes

Abstract

In a wide variety of applications, humans interact with a complex environment by means of asynchronous stochastic discrete events in continuous time. Can we design online interventions that will help humans achieve certain goals in such asynchronous setting? In this paper, we address the above problem from the perspective of deep reinforcement learning of marked temporal point processes, where both the actions taken by an agent and the feedback it receives from the environment are asynchronous stochastic discrete events characterized using marked temporal point processes. In doing so, we define the agent's policy using the intensity and mark distribution of the corresponding process and then derive a flexible policy gradient method, which embeds the agent's actions and the feedback it receives into real-valued vectors using deep recurrent neural networks. Our method does not make any assumptions on the functional form of the intensity and mark distribution of the feedback and it allows for arbitrarily complex reward functions. We apply our methodology to two different applications in personalized teaching and viral marketing and, using data gathered from Duolingo and Twitter, we show that it may be able to find interventions to help learners and marketers achieve their goals more effectively than alternatives.

arXiv commentTo appear in Proceedings of the 32nd Conference on Neural Information Processing Systems (NIPS 2018)

Cited by in corpus (18)

• Fully Neural Network based Model for General Temporal Point Processes
• Time2Vec: Learning a Vector Representation of Time
• Intensity-Free Learning of Temporal Point Processes
• Self-Attentive Hawkes Processes
• Modeling of Spatio-Temporal Hawkes Processes with Randomized Kernels
• Long Horizon Forecasting With Temporal Point Processes
• RLTutor: Reinforcement Learning Based Adaptive Tutoring System by Modeling Virtual Student with Fewer Interactions
• Non-submodular Function Maximization subject to a Matroid Constraint, with Applications
• Fast and Flexible Temporal Point Processes with Triangular Maps
• An Empirical Study: Extensive Deep Temporal Point Process
• Spatio-Temporal Point Processes with Attention for Traffic Congestion Event Modeling
• Deep Fourier Kernel for Self-Attentive Point Processes
• Imitation Learning of Neural Spatio-Temporal Point Processes
• Online Learning for Active Cache Synchronization
• Sequential Adversarial Anomaly Detection for One-Class Event Data
• UNIPoint: Universally Approximating Point Processes Intensities
• ABC Learning of Hawkes Processes with Missing or Noisy Event Times
• Neural Spectral Marked Point Processes