Overview
Synthetic spectral image generation is crucial for remote sensing simulation and mission design. Traditional physically based radiative transfer models (RTMs) are known for their high computational cost. Existing learning-based emulators have reduced this cost, but typically function as deterministic parameter-to-spectrum regressors, exhibiting limitations in spatial modeling and uncertainty information.
This research formulates spectral image emulation as a parameter-conditioned latent-variable problem. It proposes a variational autoencoder (VAE)-based framework designed to integrate nonlinear spectral-image representations, rapid inference, and per-pixel uncertainty estimations. The framework's instantiation occurs at both spectrum and spatial-spectral levels, employing a two-step VAE pretraining process followed by latent mapping.
Approach
The proposed framework utilizes a VAE architecture. Its design incorporates two primary instantiation levels:
- Spectrum-level instantiation: Focuses on individual spectrum representations.
- Spatial-spectral level instantiation: Addresses both spatial and spectral dimensions simultaneously.
The methodology involves a two-step process:
- VAE pretraining: Initial training of the VAE.
- Latent mapping: Subsequent mapping within the latent space.
Evaluation of the framework was conducted against two types of baselines:
- Classical regression emulators.
- A deep Convolutional Neural Network (CNN) baseline.
Two distinct datasets were used for evaluation:
- PROSAIL-simulated hyperspectral vegetation cubes, featuring 211 bands.
- Real Sentinel-3 OLCI multispectral ocean-colour imagery, comprising 21 bands.
Findings
The evaluation indicated that no single architecture consistently performed optimally across all test cases. Specific findings include:
- Pixel-to-pixel models: These models demonstrated the best performance on controlled hyperspectral simulations.
- Fully convolutional VAE: This architecture proved more robust when applied to noisy real observations, particularly those containing missing or contaminated pixels.
- Throughput: VAE-based emulators achieved high throughput, facilitating large-scale generation tasks.
- Uncertainty on Sentinel-3: On the Sentinel-3 dataset, the spatial-spectral VAE yielded predictive intervals that were closer to empirical errors compared to pixel-wise neural and classical emulators. However, absolute calibration for these intervals was noted as incomplete.
- Reconstruction fidelity vs. end-use: A look-up-table-based retrieval experiment revealed that reconstruction fidelity alone does not guarantee reliable retrieval of leaf area index and chlorophyll.
Why This Matters
The findings indicate that emulators should be assessed within representative remote-sensing end-use scenarios, rather than relying solely on reconstruction metrics. This suggests a need for evaluation criteria that align more closely with the practical applications and objectives of remote sensing, ensuring that developed tools are truly effective for their intended purposes beyond mere data reproduction.