Overview
OccPlanner is introduced as a goal-aware, occupancy-conditioned diffusion planner designed for PixelGoal navigation tasks. This system integrates learned egocentric goal and planning-oriented 3D representations to facilitate the generation of trajectories that are both target-directed and aware of obstacles. The methodology addresses challenges inherent in PixelGoal navigation, particularly the unreliability of depth-based metric target estimates under occlusion or sensor noise, and the inability of a sole PointGoal to encode traversability or feasible paths.
Research Context
PixelGoal navigation defines navigation targets directly within an agent's camera view. This approach offers an interface between higher-level visual reasoning processes and lower-level navigation controls. A common method to translate a visible target pixel into a metric PointGoal involves using depth information. However, this conversion's reliability diminishes when occlusions are present or sensor noise affects depth measurements. Furthermore, a PointGoal, by itself, does not convey information regarding traversability or potential paths around obstacles.
Approach
OccPlanner learns complementary egocentric goal and planning-oriented 3D representations. The egocentric goal representation is derived through metric target prediction, while the planning-oriented 3D representation is learned via occupancy prediction. These two representations condition a diffusion trajectory module, which is responsible for generating trajectories that are directed towards the target and account for obstacles.
To provide scalable geometric supervision for training, a system called L3ROcc was developed. L3ROcc processes monocular RGB navigation videos, converting them into aligned 3D occupancy and trajectory annotations. This annotation generation process supports the training of OccPlanner.
Findings
OccPlanner was trained using L3ROcc-processed data from InternData-N1. Its performance was evaluated in closed-loop simulations across four distinct, previously unseen InternScenes categories and two different goal-distance ranges. The evaluation encompassed a total of eight settings.
- Across all eight tested settings, OccPlanner substantially outperformed existing open-source PixelGoal approaches.
- OccPlanner achieved performance competitive with PointGoal planners that utilized direct metric-goal inputs.