Overview
EgoSurg represents a framework designed to synthesize arbitrary egocentric views of operating room (OR) workflows using ambient, fixed camera infrastructure. This system reconstructs dynamic OR scenes from sparse, wall-mounted stereo video, enabling the rendering of role-specific egocentric perspectives. A key objective of EgoSurg is to capture the individual viewpoints that inform clinical decisions, which have historically been undocumented by fixed vantage points or post-procedure recollections. The framework operates without requiring instrumentation of personnel, thereby avoiding interference with clinical workflow.
Research Context
Traditional methods for observing surgical practice often rely on static camera angles or retrospective accounts. These approaches typically fail to capture the specific egocentric perspectives of individual team members within the operating room. While ambient fixed cameras can record the OR at a room scale, they do not inherently provide the viewpoint of what any specific individual saw during a procedure. This gap limits the ability to fully document and analyze the visual information guiding clinical decisions during surgical operations.
Approach
The EgoSurg framework utilizes sparse wall-mounted stereo video as its primary input. The methodological sequence involves several steps:
- Initialization: A per-timestamp 3D Gaussian Splatting representation is initialized. This step leverages scale-aware stereo depth information derived from the input video.
- Refinement: The initialized representation undergoes refinement using an image-conditioned diffusion model. This model is employed to correct auxiliary rendered views, a process designed to mitigate artifacts. These artifacts can arise from factors such as limited camera coverage, crowding within the OR, and occlusions.
The system was evaluated through deployment in real-world and simulated environments. This included four actual robotic pulmonology procedures and two simulated full-workflow sessions. These evaluations were conducted across two distinct hospital sites.
Findings
The evaluation of the EgoSurg framework yielded several quantitative and qualitative findings:
- Near-Field Reconstruction Fidelity: The system demonstrated consistent near-field reconstruction fidelity. This was measured by a Peak Signal-to-Noise Ratio (PSNR) of $26.8$ dB and a Structural Similarity Index Measure (SSIM) of $.895$. This consistency was observed across five distinct workflow phases and both hospital sites where the evaluations took place.
- Synthesized Egocentric Views: Synthesized egocentric views achieved a PSNR of $17.8$ dB and an SSIM of $.766$. These metrics were calculated against paired hand-held point-of-view recordings, indicating the accuracy of the generated viewpoints relative to actual egocentric capture.
- Case Study Demonstrations: The research presented three specific case studies to illustrate intended applications:
- Adjudicating a simulated sterile field violation, suggesting the system's utility in event analysis.
- Replaying a procedure from various role-specific viewpoints, indicating its potential for training and review.
- Testing a counterfactual change in personnel position, demonstrating its capacity for workflow analysis and optimization scenarios.
Why This Matters
The results indicate that existing ambient camera infrastructure within operating rooms can be transformed into a navigable 3D record of surgical work. This capability supports retrospective review of safety events by providing diverse perspectives. Additionally, it offers a tool for training new personnel and conducting detailed workflow analysis from multiple angles, thereby addressing the previous limitations of fixed viewpoint observation.
Potential Applications
The research explicitly identifies several potential applications for the EgoSurg framework based on its capabilities:
- Retrospective Review of Safety Events: The system can facilitate the examination of incidents by providing a detailed, navigable 3D record of the surgical environment from various perspectives.
- Training: The ability to replay procedures from role-specific viewpoints offers a mechanism for surgical training, allowing trainees to experience different perspectives without direct clinical involvement.
- Workflow Analysis: EgoSurg supports detailed analysis of surgical workflows, including the examination of personnel positioning and its impact, through its capacity to test counterfactual scenarios.