CoLES: Contrastive Learning for Event Sequences with Self-Supervision

Abstract

We address the problem of self-supervised learning on discrete event sequences generated by real-world users. Self-supervised learning incorporates complex information from the raw data in low-dimensional fixed-length vector representations that could be easily applied in various downstream machine learning tasks. In this paper, we propose a new method "CoLES", which adapts contrastive learning, previously used for audio and computer vision domains, to the discrete event sequences domain in a self-supervised setting. We deployed CoLES embeddings based on sequences of transactions at the large European financial services company. Usage of CoLES embeddings significantly improves the performance of the pre-existing models on downstream tasks and produces significant financial gains, measured in hundreds of millions of dollars yearly. We also evaluated CoLES on several public event sequences datasets and showed that CoLES representations consistently outperform other methods on different downstream tasks.

SIGMOD'22

References in corpus (9)

• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Learning Representations by Maximizing Mutual Information Across Views
• Deep Metric Learning for Practical Person Re-Identification
• E.T.-RNN: Applying Deep Learning to Credit Loan Applications
• A Comparison Study of Credit Card Fraud Detection: Supervised versus Unsupervised
• A Framework For Contrastive Self-Supervised Learning And Designing A New Approach
• Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Tasks
• Attributed Sequence Embedding

Cited by in corpus (2)

• Towards a Foundation Purchasing Model: Pretrained Generative Autoregression on Transaction Sequences
• Label Attention Network for Temporal Sets Prediction: You Were Looking at a Wrong Self-Attention