Overview
PSMP-CLIP is a proposed method addressing zero-shot anomaly detection, a task focused on localizing anomalies without requiring target-domain samples. The approach specifically targets limitations observed in existing CLIP-based methods, namely coarse anomaly maps and restricted semantic prompting. PSMP-CLIP integrates two core components: Patch-Prompt SAM2 Segmentation (PPSS) and Multi-Semantic Guided Prompt Regularization (MSGPR).
Research Context
Zero-shot anomaly detection aims to identify and localize anomalous regions within images without prior exposure to samples representative of the anomalies themselves. Current CLIP-based methodologies in this field encounter difficulties producing fine-grained anomaly maps. Additionally, these methods often rely on semantic prompts that are described as being limited in their expressiveness or scope. The development of PSMP-CLIP stems from these identified challenges within the zero-shot anomaly detection paradigm.
Approach
PSMP-CLIP's methodology is structured around two principal components designed to address the noted limitations:
Patch-Prompt SAM2 Segmentation (PPSS)
- PPSS is designed to mitigate issues such as threshold drift.
- This component operates by sampling prompts directly from intermediate patch features.
- The sampled prompts subsequently guide SAM2, facilitating the generation of precise masks.
Multi-Semantic Guided Prompt Regularization (MSGPR)
- MSGPR incorporates multiple learnable prompts.
- These learnable prompts are constrained by semantic anchors.
- The regularization mechanism aims to preserve the generalization capability of the model.
Findings
Experiments conducted on 14 datasets indicated that PSMP-CLIP delivered highly competitive performance in zero-shot anomaly detection. Specifically, the method achieved the best pixel-level AUROC (Area Under the Receiver Operating Characteristic curve) on seven distinct datasets:
- MVTec AD
- BTAD
- DTD-Synthetic
- CVC-ClinicDB
- TN3K
- Endo
- Kvasir
Why This Matters
The development of PSMP-CLIP offers a method for zero-shot anomaly detection that addresses identified shortcomings in existing CLIP-based approaches. By improving the precision of anomaly localization and enhancing semantic prompting, the technique contributes to the efficacy of automated anomaly identification systems that do not require extensive domain-specific anomaly examples for training.