Projection-based Spatial Augmented Reality for Interactive Visual Guidance in Surgery

Abstract

This work introduces a projector-camera system into the surgical procedure to develop a direct AR based surgical planning and navigation mechanism. Through overlaying the projection of planning data on the specific position of the patient (skin) surface, surgeons can directly supervise robot-assisted execution according to the presurgical planning. New solutions are proposed to overcome the existing visual and operational limitations in image-guided surgery, and specifically, in the image-guided surgery which integrates a robotic assistance. The proposed AR guidance mechanism was tested in surgical experiments with percutaneous robot-assisted radiofrequency needle insertion and direct augmented interaction. Clinical viability of this advanced human-computer interaction is investigated via ex vivo and in vivo experiments. The experimental results on the phantom and porcine models demonstrated its clinical viability in the robot-assisted RF surgery.