Hanyang Med Rev.  2016 Nov;36(4):242-247. 10.7599/hmr.2016.36.4.242.

Augmented Reality in Medicine

• Ha HG
• Hong J

• KMID: 2361199
• DOI: http://doi.org/10.7599/hmr.2016.36.4.242

Abstract

Augmented reality is popular in various fields, and the importance of this technology has been increasing. Its medical application has also been widely studied. Particularly, augmented reality can be a more promising technique of a surgery which requires great precision. This paper introduces an overview of augmented reality and reviews the recent applications in medicine. After describing the basic concepts, brief characteristics of the three components that comprise augmented reality are provided. Various applications implemented in the authors' laboratory are reviewed.

Keyword

augmented reality; camera calibration; image registration

Figure

• 

  Fig. 1 Proposed AR system configuration for cardiac intervention

• 

  Fig. 2 Prototype software for the proposed cardiac intervention AR system, including the computed tomographic angiography model overlaid onto the X-ray image and VR image

• 

  Fig. 3 Proposed AR system configuration for bone tumor resection surgery

• 

  Fig. 4 Proposed AR system for bone tumor resection surgery; (a) resection margin of 3D tumor model and (b) captured scene from the proposed AR system.

• 

  Fig. 5 Proposed AR based surgical navigation system for sinus surgery

• 

  Fig. 6 Configuration of the position tracking section in the proposed AR system

• 

  Fig. 7 Proposed AR and VR switchable surgical navigation system for spinal surgery; (a) AR mode and (b) VR mode

Reference

1. Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B. Recent advances in augmented reality. IEEE Comput Graph Appl. 2001; 21:34–47.

2. Sielhorst T, Feuerstein M, Navab N. Advanced medical displays: A literature review of augmented reality. J Disp Technol. 2008; 4:451–467.

3. Silva R, Oliveira JC, Giraldi GA. Introduction to augmented reality. Av Getulio Vargas: National Laboratory for Scientific Computation;2003.

4. Carmigniani J, Furht B, Anisetti M, Ceravolo P, Damiani E, Ivkovic M. Augmented reality technologies, systems and applications. Multimed Tools Appl. 2011; 51:341–377.

5. Fischer J, Neff M, Freudenstein D, Bartz D. Medical augmented reality based on commercial image guided surgery. In : Eurographics Symposium on Virtual Environments (EGVE); 2004. p. 83–86.

6. De Paolis LT, Aloisio G. Augmented reality in minimally invasive surgery. InAdvances in Biomedical Sensing, Measurements, Instrumentation and Systems. Springer;2010. p. 305–320.

7. Lamata P, Freudenthal A, Cano A, Kalkofen D, Schmalstieg D, Naerum E, et al. Augmented reality for minimally invasive surgery: overview and some recent advances. INTECH Open Access Publisher;2010.

8. Ha H, Bok Y, Joo K, Jung J, So Kweon I. Accurate Camera Calibration Robust to Defocus Using a Smartphone. In : Proceedings of the IEEE International Conference on Computer Vision; 2015. p. 828–836.

9. Poling B. A Tutorial On Camera Models.

10. Medioni G, Kang SB. Emerging topics in computer vision. Prentice Hall PTR;2004.

11. Riba Pi E. Implementation of a 3D pose estimation algorithm. 2015.

12. Moreno D, Taubin G. Simple, accurate, and robust projector-camera calibration. In : 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission: IEEE; 2012. p. 464–471.

13. Besl PJ, McKay ND. Method for registration of 3-D shapes. In : Robotics-DL tentative: International Society for Optics and Photonics; 1992. p. 586–606.

14. Zhang Z. Iterative point matching for registration of free-form curves and surfaces. Int J Comput Vis. 1994; 13:119–152.

15. Sharp GC, Lee SW, Wehe DK. Invariant features and the registration of rigid bodies. In : Robotics and Automation, 1999 Proceedings 1999 IEEE International Conference on: IEEE; 1999. p. 932–937.

16. Balachandran R, Fitzpatrick JM. Iterative solution for rigid-body point-based registration with anisotropic weighting. In : SPIE Medical Imaging: International Society for Optics and Photonics; 2009. p. 72613D-D-10.

17. Burschka D, Mair E. Direct pose estimation with a monocular camera. In : International Workshop on Robot Vision; Springer;2008. p. 440–453.

18. Choi H, Cho B, Masamune K, Hashizume M, Hong J. An effective visualization technique for depth perception in augmented reality‐based surgical navigation. Int J Med Robot. 2015.

19. Jeon S, Hwangbo S, Hong J. A Surgical Navigation System to Assist in Chronic Total Occlusion Intervention. In : URAI 2016;

20. Choi H, Park Y, Cho H, Hong J. An augmented reality based simple navigation system for pelvic tumor resection. In : Pro. of American Acadamy of Orthopaedic Surgeons; 2016.

21. Lee S, Yoon H-S, Park J, Chung Y-S, Hong J, Yi B-J. A Surgical Navigation and Endoscope Holder Integrated System for Sinus Surgery. In : Proc. of The 11th Asian Conference on Computer Aided Surgery; 2015.

22. Jeon S, Kim J, Hong J. Surgical navigation system for assisting epiduroscopic laser neural decompression (ELND) procedure. its clinical application in 14 patients. In : Computer Assisted Radiology and Surgery; 2014.