Overview
FoundationGeo is a two-stage framework designed for learning spatial pixel-wise fields to improve monocular metric geometry. This framework explicitly connects relative and metric prediction through processes of spatial calibration and data design. It aims to produce metrically consistent 3D point maps from monocular input.
Research Context
The research addresses challenges in monocular metric geometry estimation, particularly regarding cross-domain robustness and consistent metric prediction. Existing methods can exhibit sharp performance drops when evaluated across diverse domains. A specific bottleneck identified is camera intrinsic coverage, particularly the mismatch in focal length distribution between training and test datasets, which can lead to performance degradation during zero-shot metric generalization.
Approach
FoundationGeo operates through two distinct stages:
Stage 1: Affine-Invariant Geometry Model Learning
- Initialization: The first stage initializes with DINOv3.
- Training Data: Training is conducted on a multi-domain corpus containing 10.2 million samples.
- Supervision: This stage incorporates complementary local-detail supervision.
- Outcome: This process yields a high-fidelity, affine-invariant geometry model characterized by sharp boundaries and strong cross-domain generalization capabilities.
Stage 2: Metric Estimation with Pixel-Wise Calibration Fields
This stage moves beyond global scaling by introducing lightweight pixel-wise calibration fields for metric estimation:
- Scale Field: A scale field is employed for spatially varying metric alignment.
- Ray-Direction Correction Field: This field mitigates directional bias observed in point-map geometry.
- Output: The combined application of these fields produces metrically consistent 3D point maps.
Data Design Enhancements
Beyond the model design, the researchers addressed the identified issue of camera intrinsic coverage:
- Problem Identification: A key bottleneck for zero-shot metric generalization was identified as the focal length distribution mismatch between training and test data, leading to sharp performance drops when test intrinsics fell outside the training distribution.
- Solution: Additional training data was synthesized across diverse focal lengths. This synthesis utilized a Blender-based data engine.
- Impact: This approach repaired under-covered focal regimes and improved robustness under intrinsic shift.
Findings
- Extensive zero-shot evaluations were conducted across seven benchmarks.
- FoundationGeo significantly strengthened cross-domain robustness.
- The framework maintained performance near the top across diverse domains.
- It avoided the sharp cross-domain performance drops observed in other methods.
- This consistency resulted in the best overall performance.
- FoundationGeo surpassed heavier baselines by over 5.2% on average.
Why This Matters
The improvements in monocular metric geometry prediction offered by FoundationGeo, particularly its enhanced cross-domain robustness and consistent metric estimation, suggest implications for applications requiring accurate 3D spatial understanding from single camera inputs across varied environments.