• by Emil Salazar
  • May 30 2017


Robotic Surgery Develops as a Platform for Augmented Reality and Virtual Reality

Robotic Surgery Develops as a Platform for Augmented Reality and Virtual Reality

Image-Guided Surgery (IGS) Procedures Are Especially Ideal for VR/AR Integration

In the recently published market research reportVirtual Reality and Augmented Reality (VR/AR) in Healthcare, Kalorama Information examines application markets for virtual reality (VR) and augmented reality (AR) in U.S. healthcare, including in medical education, healthcare training, rehabilitation, therapy, and surgery.

Robot-assisted surgery (RAS) systems represent a leading avenue for the integration of virtual reality (VR) and augmented reality (AR) technology in the operating room (OR). Live and virtual imaging featured on RAS user interfaces assist surgeon’s manipulation of robotic instruments and represent an open platform for the addition of VR and AR capabilities. Live surgical imaging can be enhanced on RAS systems through image injection or the superimposition of location-specific objects. The most prevalent application of VR/AR technology in RAS has been motion tracking of robotic instruments within an interactive model of patient anatomy displayed on a console screen.

Virtual reality technology has been particularly useful in robot-assisted orthopedic surgery. For instance, the Stryker Mako system highlights bone sections during surgical planning to be removed or shaved away for implant placement. The pre-operative 3D planning model is registered to the patient structure during the procedure and becomes an interactive template. The Mako burring instrument is directly held in the surgeon’s hand, but provides resistance against surgeon movements to keep the burr within the highlighted implant area of the 3D model. Bone removal or instrument operation within the surgical updates the virtual model by reducing the highlighted model segmentation. Real-time tracking and consultation of the virtual model improves the precision of orthopedic implant placement and mitigates excessive bone removal.

Interactive 3D models of orthopedic surgical sites are also used with the Smith & Nephew Navio RAS system. Intra-operative capture updates bone removal in the 3D model. Insufficient bone removal in the implant site is highlighted by the system in the model. The burring instrument is disabled when placed in an area outside the surgical site or on a surface not requiring further bone removal.

Augmented reality is being considered with RAS for telesurgery and teleproctoring. Surgeons outside of the OR could remotely control RAS systems with sophisticated AR solutions that would replicate the entire OR environment for the surgeon. Image injection as part of AR could also be useful for teleproctoring, where a supervising surgeon could provide visual instruction or indications to the RAS user within the surgical field of view. While some RAS platforms can already technically support telesurgery, significant regulatory barriers remain for the practice.

The more immediate application of AR in robot-assisted surgery would be image injection as already demonstrated on surgical navigation or freehand image-guided surgery (IGS) platforms. Computer-generated objects imported from pre-operative medical imaging can be displayed over endoscopic or laparoscopic live video feed – highlighting obscured or non-visible structures of interest or identifying the surgical target. Despite the capabilities of image injection and potential benefits to patient outcome, RAS vendors have yet to meaningfully deploy or demonstrate the technology. Intuitive Surgical, supplier of the industry-leading da Vinci RAS system, has been averse to added system costs without clearly demonstrated benefits for its clients. The company has cited cost concerns when discussing haptic feedback for robotic instruments. On the other hand, Intuitive Surgical embraced fluorescent imaging on the da Vinci system, which is capable of selectively labeling tissues with blood flow. Image injection on RAS systems could provide similar context to surgeons by defining safe parameters of movement, areas to avoid, and which tissues to perform on.