Overview
The PolarScale benchmark addresses the challenge of inferring polarization information from RGB-like inputs. While existing methods predict normalized Stokes components or relative descriptors, they typically omit the radiometric scale required for full Stokes reconstruction. PolarScale explicitly targets the prediction and evaluation of this per-scene radiometric scale.
The benchmark utilizes existing trichromatic full-Stokes measurements. Models are tasked with predicting normalized Stokes components, Angle of Linear Polarization (AoLP), Degree of Linear Polarization (DoLP), Degree of Circular Polarization (DoCP), and a per-scene scale from a scene-referred linear image ($s_0$). As the input $s_0$ has the scale divided out, the scale is not physically identifiable from this input alone. PolarScale, therefore, evaluates dataset-conditioned semantic scale estimation against a constant-scale control. Evaluation metrics also include angular metrics, self-consistency checks, and physical-bound adherence.
Research Context
Polarization imaging offers physical cues that extend beyond traditional intensity imaging. However, its practical application often necessitates specialized hardware. Recent advancements have focused on inferring polarization information from RGB-like inputs. A limitation of these methods is their prediction of only normalized Stokes components or relative descriptors, which lack the radiometric scale crucial for a complete full Stokes reconstruction. This missing radiometric scale is specifically addressed by the PolarScale benchmark.
Approach
PolarScale is constructed upon existing trichromatic full-Stokes measurements. The benchmark defines the input as a per-scene normalized total-intensity image ($s_0$), which is a scene-referred linear image. The objective for models is to predict specific polarization parameters: normalized Stokes components, AoLP, DoLP, DoCP, and the per-scene scale. Acknowledging that the radiometric scale is divided out of the input $s_0$ and is thus not physically identifiable from it, PolarScale's evaluation strategy for scale estimation is dataset-conditioned and compared against a constant-scale control. Beyond scale, the benchmark incorporates angular metrics, self-consistency metrics, and assessments of physical bound violations.
The study applied seven restoration-based and generative backbone architectures across three distinct prediction strategies. These architectures were evaluated on their ability to perform the specified predictions within the PolarScale framework.
Findings
- The strongest restoration models achieved a mean relative error of 3.6-4.3% for radiometric scale estimation. This performance compares favorably to the constant-scale control, which yielded a 5.7% error.
- These restoration models violated physical bounds on fewer than 0.25% of pixels during their predictions.
- Two generative baselines evaluated within the benchmark collapsed to predict a near-zero scale.
- Explicit descriptor supervision led to improved descriptor accuracy, evidenced by a PSNR of 23.66 dB compared to 18.88 dB for MAE (Mean Absolute Error).
- Predicted full-Stokes representations demonstrated improvements in several downstream tasks: diffuse/specular separation, material segmentation, and glare classification.
- In the task of diffuse/specular separation, the learned scale performed comparably to the constant control.
Why This Matters
The development of PolarScale as a benchmark provides a standardized method for evaluating models that infer full Stokes parameters, specifically including the radiometric scale, from RGB-like inputs. This capability could enhance the utility of polarization information in various computer vision applications. The ability to estimate this scale with relatively low error and maintain physical consistency from standard inputs may contribute to the broader adoption of polarization analysis in areas like scene understanding and material characterization.
Potential Applications
- Predicted full-Stokes representations, enabled by frameworks like PolarScale, have been shown to improve performance in diffuse/specular separation tasks.
- The integration of full-Stokes predictions can enhance material segmentation, offering more robust identification and differentiation of various material types within a scene.
- Glare classification benefits from the additional physical cues provided by full-Stokes information, potentially leading to more accurate detection and characterization of reflective phenomena.