Overview
Pathology Autoregressive modeling (PathAR) is a generative framework designed for synthesizing multimodal pathology images. It addresses data scarcity in this domain by explicitly separating pathological structure from appearance during generation. This approach aims to maintain anatomical coherence across diverse modality-specific appearances, leading to improved structural consistency and modality fidelity compared to existing methods.
Research Context
Multimodal pathology often faces data scarcity, necessitating generative models capable of synthesizing modality-specific appearance while preserving anatomically coherent structure. Current methods frequently integrate structure and appearance within a single token stream, which can implicitly couple these factors. This homogeneous modeling may weaken structural controllability, particularly when encountering modality shifts. While anatomical structures like cellular topology and tissue boundaries largely persist across different acquisition protocols, appearance statistics vary among modalities. Existing approaches often struggle to maintain structural integrity under these appearance variations.
Approach
PathAR implements a structure-first autoregressive synthesis framework. Its core mechanism involves the explicit factorization of structure and appearance for modality-label-conditioned pathology generation. The framework comprises two primary components:
- Dual Vector Quantization (Dual-VQ) tokenizer: This component is responsible for decomposing input samples into distinct mask-grounded structure tokens and appearance tokens. This explicit separation is crucial for the framework's factorized representation.
- Interleaved Autoregressive (IAR) transformer: This transformer operates with asymmetric attention visibility. Its design enforces a dependency where appearance generation is conditioned on the previously generated structure. This mechanism ensures that synthesized appearances align with and are guided by the underlying anatomical structure.
The design of PathAR aims to stabilize morphology even when confronted with heterogeneous modality-specific appearances. It facilitates the generation of spatially aligned image-mask pairs.
Findings
Extensive experiments conducted with PathAR yielded several findings:
- PathAR demonstrated improvement in structural consistency when compared to baseline methods.
- The framework exhibited enhanced modality fidelity relative to baselines.
- It maintained sample diversity during image generation.
- PathAR supports downstream segmentation tasks, particularly in scenarios characterized by data scarcity.
- The approach showed extensibility to finer-grained intra-modality organ-label variation.