Transformer Hawkes Process

Abstract

Modern data acquisition routinely produce massive amounts of event sequence data in various domains, such as social media, healthcare, and financial markets. These data often exhibit complicated short-term and long-term temporal dependencies. However, most of the existing recurrent neural network based point process models fail to capture such dependencies, and yield unreliable prediction performance. To address this issue, we propose a Transformer Hawkes Process (THP) model, which leverages the self-attention mechanism to capture long-term dependencies and meanwhile enjoys computational efficiency. Numerical experiments on various datasets show that THP outperforms existing models in terms of both likelihood and event prediction accuracy by a notable margin. Moreover, THP is quite general and can incorporate additional structural knowledge. We provide a concrete example, where THP achieves improved prediction performance for learning multiple point processes when incorporating their relational information.

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• WaveNet: A Generative Model for Raw Audio
• Convolutional Sequence to Sequence Learning
• Comparative Study of CNN and RNN for Natural Language Processing
• Learning Temporal Point Processes via Reinforcement Learning
• Self-Attentive Hawkes Processes

Cited by in corpus (10)

• An Empirical Study: Extensive Deep Temporal Point Process
• Deep Fourier Kernel for Self-Attentive Point Processes
• Learning Self-Modulating Attention in Continuous Time Space with Applications to Sequential Recommendation
• Cardinality-Regularized Hawkes-Granger Model
• UNIPoint: Universally Approximating Point Processes Intensities
• Survey on Modeling Intensity Function of Hawkes Process Using Neural Models
• Hawkes Processes on Graphons
• Deep Convolution for Irregularly Sampled Temporal Point Clouds
• Mitigating Performance Saturation in Neural Marked Point Processes: Architectures and Loss Functions
• Nonlinear Hawkes Processes in Time-Varying System