Diagnostic Spatio-temporal Transformer with Faithful Encoding
arXiv:2305.17149 · doi:10.1016/j.knosys.2023.110639
Abstract
This paper addresses the task of anomaly diagnosis when the underlying data generation process has a complex spatio-temporal (ST) dependency. The key technical challenge is to extract actionable insights from the dependency tensor characterizing high-order interactions among temporal and spatial indices. We formalize the problem as supervised dependency discovery, where the ST dependency is learned as a side product of multivariate time-series classification. We show that temporal positional encoding used in existing ST transformer works has a serious limitation in capturing higher frequencies (short time scales). We propose a new positional encoding with a theoretical guarantee, based on discrete Fourier transform. We also propose a new ST dependency discovery framework, which can provide readily consumable diagnostic information in both spatial and temporal directions. Finally, we demonstrate the utility of the proposed model, DFStrans (Diagnostic Fourier-based Spatio-temporal Transformer), in a real industrial application of building elevator control.
References in corpus (12)
- Scikit-learn: Machine Learning in Python
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Deep Learning for Anomaly Detection: A Review
- InceptionTime: Finding AlexNet for Time Series Classification
- Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding
- Multivariate LSTM-FCNs for Time Series Classification
- FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting
- TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data
- Gated Transformer Networks for Multivariate Time Series Classification
- Long-Range Transformers for Dynamic Spatiotemporal Forecasting
- NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting
- StackVAE-G: An efficient and interpretable model for time series anomaly detection