Overview
The Global Dictionary-enhanced Transformer (GDformer) is proposed as a method for unsupervised anomaly detection in multivariate time series. This method aims to address the limitations of existing approaches that rely on reconstruction error or association divergence, which are confined to isolated subsequences.
Research Context
Unsupervised anomaly detection in multivariate time series presents challenges due to the requirement of deriving a compact detection criterion without access to anomaly points. Current methodologies often depend on reconstruction error or association divergence. These methods are typically limited to isolated subsequences with finite horizons, which may not provide a unified series-level criterion.
The GDformer model seeks to cultivate global representations that are shared across all normal points within an entire time series. It employs a renovated dictionary-based cross-attention mechanism to achieve this.
Approach
The GDformer model integrates a dictionary-based cross-attention mechanism. This mechanism is designed to facilitate the creation of global representations that encompass normal points throughout the full time series. The cross-attention maps generated within this mechanism reflect the correlation weights between individual points and the learned global representations. This structural design intrinsically supports a detection criterion based on representation-wise similarity. To refine the detection boundary and make it more compact, prototypes are introduced. These prototypes are intended to capture the distribution of the correlation weights between points and the global representations.
Findings
- GDformer achieved state-of-the-art performance in unsupervised anomaly detection.
- This performance was observed across five real-world benchmark datasets.
- The global dictionary component of GDformer demonstrates transferability across various datasets.
Why This Matters
The development of GDformer offers a different approach to unsupervised anomaly detection in multivariate time series by moving beyond subsequence isolation. Its reported state-of-the-art performance on real-world datasets suggests its applicability for tasks requiring robust anomaly detection without prior knowledge of anomalies.