J Korean Soc Radiol.  2019 Mar;80(2):226-238. 10.3348/jksr.2019.80.2.226.

Medical Augmented Reality and Virtual Reality

• Hong J

• KMID: 2442515
• DOI: http://doi.org/10.3348/jksr.2019.80.2.226

Abstract

Augmented reality (AR) or virtual reality is used in many fields, including medicine, education, manufacturing, and entertainment. With technical advances in optics, computer systems, and surgical instruments, AR applications to medicine are being vigorously researched. In particular, as surgeries using laparoscopy, endoscopy, or catheterized intervention have increased, AR plays an important role in many medical applications. AR is defined as a technique to combine the real world and virtual objects, which are digital content artificially generated by a computer. As another aspect of AR is the registration between the real world and virtual objects, it aims at an accurate estimation of the three-dimensional (3D) position and orientation of virtual objects related to the real world. Essentially, AR can allow users to see 3D virtual objects superimposed upon the real world. With the help of AR, a surgeon can view invisible organs during the surgery and improve the accuracy and safety of treatment procedures. After a brief description of the technical issues of medical AR, its applications will be introduced in this article.

MeSH Terms

Catheters
Computer Systems
Education
Endoscopy
Laparoscopy
Surgery, Computer-Assisted
Surgical Instruments

Figure

• 

  Fig. 1 Surgical navigation for pituitary tumor resection.

• 

  Fig. 2 Surgical navigation using virtual reality (A) and augmented reality (B).

• 

  Fig. 3 Skin markers cause large target registration error in spite of small fiducial registration error. Target represents a tumor in axial images.

• 

  Fig. 4 Depth information is provided in quantity at lower left corner, and background color (yellow) shows the level of potential risk.

• 

  Fig. 5 Augmented reality for bone tumor resection.

Reference

1. Choi H, Park Y, Lee S, Ha H, Kim S, Cho HS, et al. A portable surgical navigation device to display resection planes for bone tumor surgery. Minim Invasive Ther Allied Technol. 2017; 26:144–150.

2. Choi H, Cho B, Masamune K, Hashizume M, Hong J. An effective visualization technique for depth perception in augmented reality-based surgical navigation. Int J Med Robot. 2016; 12:62–72.

3. Kim S, Hong J, Joung S, Yamada A, Matsumoto N, Kim SI, et al. Dual surgical navigation using augmented and virtual environment techniques. Int J Optomechatroni. 2011; 5:155–169.

4. Inoue D, Cho B, Mori M, Kikkawa Y, Amano T, Nakamizo A, et al. Preliminary study on the clinical application of augmented reality neuronavigation. J Neurol Surg A Cent Eur Neurosurg. 2013; 74:71–76.

5. Li L, Yang J, Chu Y, Wu W, Xue J, Liang P, et al. A novel augmented reality navigation system for endoscopic sinus and skull base surgery: a feasibility study. PLoS One. 2016; 11:e0146996.

6. Rosenthal M, State A, Lee J, Hirota G, Ackerman J, Keller K, et al. Augmented reality guidance for needle biopsies: an initial randomized, controlled trial in phantoms. Med Image Anal. 2002; 6:313–320.

7. Liao H, Hata N, Nakajima S, Iwahara M, Sakuma I, Dohi T. Surgical navigation by autostereoscopic image overlay of integral videography. IEEE Trans Inf Technol Biomed. 2004; 8:114–121.

8. Navab N, Heining SM, Traub J. Camera augmented mobile C-arm (CAMC): calibration, accuracy study, and clinical applications. IEEE Trans Med Imaging. 2010; 29:1412–1423.

9. Jeon S, Lee GW, Jeon YD, Park IH, Hong J, Kim JD. A preliminary study on surgical navigation for epiduroscopic laser neural decompression. Proc Inst Mech Eng H. 2015; 229:693–702.

10. Cho B, Oka M, Matsumoto N, Ouchida R, Hong J, Hashizume M. Warning nWarning navigation system using real-time safe region monitoring for otologic surgery. Int J Comput Assist Radiol Surg. 2013; 8:395–405.

11. Hong J, Hashizume M. An effective point-based registration tool for surgical navigation. Surg Endosc. 2010; 24:944–948.

12. Lee S, Kim JY, Hong J, Baek SH, Kim SY. CT-based navigation system using a patient-specific instrument for femoral component positioning: an experimental in vitro study with a sawbone model. Yonsei Med J. 2018; 59:769–780.

13. Oka M, Cho B, Matsumoto N, Hong J, Jinnouchi M, Ouchida R, et al. A preregistered STAMP method for image-guided temporal bone surgery. Int J Comput Assist Radiol Surg. 2014; 9:119–126.

14. Jeon S, Park J, Chien J, Hong J. A hybrid method to improve target registration accuracy in surgical navigation. Minim Invasive Ther Allied Technol. 2015; 24:356–363.

15. Hong J, Matsumoto N, Ouchida R, Komune S, Hashizume M. Medical navigation system for otologic surgery based on hybrid registration and virtual intraoperative computed tomography. IEEE Trans Biomed Eng. 2009; 56:426–432.

16. Lee S, Lee H, Choi H, Jeo S, Ha H, Hong J. Comparative study of hand-eye calibration methods for augmented reality using an endoscope. J Electron Imaging. 2018; 27:043017.

17. Lee S, Lee H, Choi H, Jeon S, Hong J. Effective calibration of an endoscope to an optical tracking system for medical augmented reality. Cogent Eng. 2017; 4:1359955.

18. Cho B, Oka M, Matsumoto N, Ouchida R, Hong J, Hashizume M. Warning navigation system using real-time safe region monitoring for otologic surgery. Int J Comput Assist Radiol Surg. 2013; 8:395–405.

19. Cho HS, Park YK, Gupta S, Yoon C, Han I, Kim HS, et al. Augmented reality in bone tumour resection: an experimental study. Bone Joint Res. 2017; 6:137–143.

20. Ha HG, Jeon S, Lee S, Choi H, Hong J. Perspective pinhole model with planar source for augmented reality surgical navigation based on C-arm imaging. Int J Comput Assist Radiol Surg. 2018; 13:1671–1682.