6 papers
Be Wary of Your Time Series Preprocessing
Sofiane Ennadir, Tianze Wang, Oleg Smirnov +2
Normalization and scaling are fundamental preprocessing steps in time series modeling, yet their role in Transformer-based models remains underexplored from a theoretical perspecti…
Towards Unified Approaches in Self-Supervised Event Stream Modeling: Progress and Prospects
Levente Zólyomi, Levente Zólyomi, Tianze Wang +3
The proliferation of digital interactions across diverse domains, such as healthcare, e-commerce, gaming, and finance, has resulted in the generation of vast volumes of event strea…
Frequency Matters: When Time Series Foundation Models Fail Under Spectral Shift
Tianze Wang, Sofiane Ennadir, John Pertoft +7
Time series foundation models (TSFMs) have shown strong results on public benchmarks, prompting comparisons to a "BERT moment" for time series. Their effectiveness in industrial se…
Pool Me Wisely: On the Effect of Pooling in Transformer-Based Models
Sofiane Ennadir, Levente Zólyomi, Oleg Smirnov +4
Transformer models have become the dominant backbone for sequence modeling, leveraging self-attention to produce contextualized token representations. These are typically aggregate…
Expressivity of Representation Learning on Continuous-Time Dynamic Graphs: An Information-Flow Centric Review
Sofiane Ennadir, Gabriela Zarzar Gandler, Filip Cornell +6
Graphs are ubiquitous in real-world applications, ranging from social networks to biological systems, and have inspired the development of Graph Neural Networks (GNNs) for learning…
Understanding Players as if They Are Talking to the Game in a Customized Language: A Pilot Study
Tianze Wang, Maryam Honari-Jahromi, Styliani Katsarou +5
This pilot study explores the application of language models (LMs) to model game event sequences, treating them as a customized natural language. We investigate a popular mobile ga…