Self-Attentive Hawkes Processes

Abstract

Asynchronous events on the continuous time domain, e.g., social media actions and stock transactions, occur frequently in the world. The ability to recognize occurrence patterns of event sequences is crucial to predict which typeof events will happen next and when. A de facto standard mathematical framework to do this is the Hawkes process. In order to enhance expressivity of multivariate Hawkes processes, conventional statistical methods and deep recurrent networks have been employed to modify its intensity function. The former is highly interpretable and requires small size of training data but relies on correct model design while the latter has less dependency on prior knowledge and is more powerful in capturing complicated patterns. We leverage pros and cons of these models and propose a self-attentive Hawkes process(SAHP). The proposed method adapts self-attention to fit the intensity function of Hawkes processes. This design has two benefits:(1) compared with conventional statistical methods, the SAHP is more powerful to identify complicated dependency relationships between temporal events; (2)compared with deep recurrent networks, the self-attention mechanism is able to capture longer historical information, and is more interpretable because the learnt attention weight tensor shows contributions of each historical event. Experiments on four real-world datasets demonstrate the effectiveness of the proposed method.

References in corpus (12)

• Gaussian Error Linear Units (GELUs)
• Visualizing and Understanding Recurrent Networks
• Fully Neural Network based Model for General Temporal Point Processes
• Deep Reinforcement Learning of Marked Temporal Point Processes
• Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models
• Learning Network of Multivariate Hawkes Processes: A Time Series Approach
• Learning Temporal Point Processes via Reinforcement Learning
• COEVOLVE: A Joint Point Process Model for Information Diffusion and Network Co-evolution
• Lecture Notes: Temporal Point Processes and the Conditional Intensity Function
• Intensity-Free Learning of Temporal Point Processes
• Self-Attention with Structural Position Representations
• Benchmarking Deep Sequential Models on Volatility Predictions for Financial Time Series

Cited by in corpus (8)

• Transformer Hawkes Process
• A Graph Regularized Point Process Model For Event Propagation Sequence
• Tensorized Transformer for Dynamical Systems Modeling
• Spatio-Temporal Point Processes with Attention for Traffic Congestion Event Modeling
• Deep Fourier Kernel for Self-Attentive Point Processes
• Address-Event Variable-Length Compression for Time-Encoded Data
• Deep Neyman-Scott Processes
• Deep Convolution for Irregularly Sampled Temporal Point Clouds