Korean J Radiol.  2020 Feb;21(2):133-145. 10.3348/kjr.2019.0625.

Advanced Medical Use of Three-Dimensional Imaging in Congenital Heart Disease: Augmented Reality, Mixed Reality, Virtual Reality, and Three-Dimensional Printing

• Goo HW
• Park SJ
• Yoo SJ

• KMID: 2471631
• DOI: http://doi.org/10.3348/kjr.2019.0625

Abstract

Three-dimensional (3D) imaging and image reconstruction play a prominent role in the diagnosis, treatment planning, and post-therapeutic monitoring of patients with congenital heart disease. More interactive and realistic medical experiences take advantage of advanced visualization techniques like augmented, mixed, and virtual reality. Further, 3D printing is now used in medicine. All these technologies improve the understanding of the complex morphologies of congenital heart disease. In this review article, we describe the technical advantages and disadvantages of various advanced visualization techniques and their medical applications in the field of congenital heart disease. In addition, unresolved issues and future perspectives of these evolving techniques are described.

Keyword

Augmented reality; Congenital heart disease; 3D imaging; 3D modeling; 3D printing; Virtual reality

MeSH Terms

Diagnosis
Heart Defects, Congenital*
Humans
Image Processing, Computer-Assisted
Imaging, Three-Dimensional*
Printing, Three-Dimensional*

Figure

• 

  Fig. 1 3D cardiac MRI. A. Oblique coronal view of time-resolved non-ECG-synchronized contrast-enhanced 3D cardiac MRI showing motion artifacts (arrows) in AA and LV, which is problematic in illustrating accurate cardiovascular morphology on advanced visualization techniques. B. Oblique coronal view of ECG-triggered, navigator-gated T2-prepared balanced steady-state free precession imaging demonstrating motionless cardiovascular morphology but suboptimal heterogeneous signal intensities in LA, LV, and RPA. AA = ascending aorta, ECG = electrocardiography, LA = left atrium, LV = left ventricle, MRI = magnetic resonance imaging, RPA = right pulmonary artery, 3D = three-dimensional

• 

  Fig. 2 Advanced post-processing of 3D cardiothoracic CT imaging. A. In coronal volume-rendered CT images with major aortopulmonary collateral arteries, each artery can be illustrated in distinct color. In addition, other structures such as airways and lungs (B) and thoracic cage (C) may be rendered simultaneously to enhance their spatial relationships with collateral arteries, which is helpful for pre-procedural planning. CT = computed tomography

• 

  Fig. 3 CT ventricular volumetry using 3D threshold-based segmentation. Coronal volume-rendered CT images highlight LV (A) and RV (B) segmented for ventricular volumetry using 3D threshold-based approach. RV = right ventricle

• 

  Fig. 4 Transparent-lumen cinematic rendering. Depth perception of cinematic-rendered images (B, D) are superior to those on regular volume-rendered images (A, C). Therefore, papillary muscles (arrows; A, B) and trabeculations are better visualized on cinematic-rendered images (B, D). With improved depth perception, anterior marginal muscular VSDs (arrows; C, D) are demonstrated without additional back clipping that must be used for regular volume-rendered image (C). Realistic endocardial appearance of transparent-lumen cinematic rendering mimics 3D printed, hollow cardiac model. RA = right atrium, VSD = ventricular septal defect

• 

  Fig. 5 Workflow of advanced visualization technology. Segmented and refined 3D model can be used not only for augmented reality, virtual reality, and interactive web or mobile 3D displays but also for 3D printing. STL = Standard Tessellation Language

• 

  Fig. 6 3D printing workflow tailored to congenital heart disease. A. Axial CT image shows segmented blood pool mask and segmented myocardium mask with distinct colors. B. 3D virtual cardiac myocardial (upper) and blood pool (lower) models. C. 3D printed cardiac myocardial (upper) and blood pool (lower) models. Small VSD (large arrow) needs to be widened in direction of small arrows.

• 

  Fig. 7 Patient-specific 3D printed hollow cardiac model using flexible printing material (TangoPlus, Stratasys) from 3D cardiothoracic CT data obtained preoperatively in infant with double outlet RV and interrupted aortic arch type B. Both smaller AA and dilated MPA arise from double outlet RV. Aortic arch is interrupted (asterisk) between left common carotid artery and left subclavian artery, indicating type B of interrupted aortic arch. MPA = main pulmonary artery

• 

  Fig. 8 Direct file conversion from segmented DICOM file to STL file. Segmented cardiovascular volume in DICOM format can be directly converted to STL format. Segmented part may be single volume (A, B) or merged volume comprising multiple parts (C). DICOM = Digital Imaging and Communication in Medicine

• 

  Fig. 9 3D printed aortic valve model for surgical simulation. A. Aortic valve (yellow) is seen in exported polygon rendered STL file. B. 3D printed model is used for surgical simulation of aortic valve. Surgeon may open aortic valve with their finger.

Reference

1. Hong J. Medical augmented reality and virtual reality. J Korean Soc Radiol. 2019; 80:226–238.

2. Yoo SJ, Thabit O, Kim EK, Ide H, Yim D, Dragulescu A, et al. 3D printing in medicine of congenital heart diseases. 3D Print Med. 2015; 2:3.

3. Kim GB, Lee S, Kim H, Yang DH, Kim YH, Kyung YS, et al. Three-dimensional printing: basic principles and applications in medicine and radiology. Korean J Radiol. 2016; 17:182–197.

4. Byrne N, Velasco Forte M, Tandon A, Valverde I, Hussain T. A systematic review of image segmentation methodology, used in the additive manufacture of patient-specific 3D printed models of the cardiovascular system. JRSM Cardiovasc Dis. 2016; 5:2048004016645467.

5. Goo HW. State-of-the-art CT imaging techniques for congenital heart disease. Korean J Radiol. 2010; 11:4–18.

6. Goo HW. Current trends in cardiac CT in children. Acta Radiol. 2013; 54:1055–1062.

7. Goo HW. CT radiation dose optimization and estimation: an update for radiologists. Korean J Radiol. 2012; 13:1–11.

8. Hui PKT, Goo HW, Du J, Ip JJK, Kanzaki S, Kim YJ, et al. Asian consortium on radiation dose of pediatric cardiac CT (ASCI-REDCARD). Pediatr Radiol. 2017; 47:899–910.

9. Hong SH, Goo HW, Maeda E, Choo KS, Tsai IC. Asian Society of Cardiovascular Imaging Congenital Heart Disease Study Group. User-rriendly vendor-specific guideline for pediatric cardiothoracic computed tomography provided by the Asian Society of Cardiovascular Imaging Congenital Heart Disease Study Group: part 1. Imaging techniques. Korean J Radiol. 2019; 20:190–204.

10. Fratz S, Chung T, Greil GF, Samyn MM, Taylor AM, Valsangiacomo Buechel ER, et al. Guidelines and protocols for cardiovascular magnetic resonance in children and adults with congenital heart disease: SCMR expert consensus group on congenital heart disease. J Cardiovasc Magn Reson. 2013; 15:51.

11. Makowski MR, Wiethoff AJ, Uribe S, Parish V, Botnar RM, Bell A, et al. Congenital heart disease: cardiovascular MR imaging by using an intravascular blood pool contrast agent. Radiology. 2011; 260:680–688.

12. Zhou Z, Han F, Rapacchi S, Nguyen KL, Brunengraber DZ, Kim GJ, et al. Accelerated ferumoxytol-enhanced 4D multiphase, steady-state imaging with contrast enhancement (MUSIC) cardiovascular MRI: validation in pediatric congenital heart disease. NMR Biomed. 2017; 30:e3663.

13. Goo HW. Comparison between three-dimensional navigator-gated whole-heart MRI and two-dimensional cine MRI in quantifying ventricular volumes. Korean J Radiol. 2018; 19:704–714.

14. Goo HW. Semiautomatic three-dimensional threshold-based cardiac computed tomography ventricular volumetry in repaired tetralogy of Fallot: comparison with cardiac magnetic resonance imaging. Korean J Radiol. 2019; 20:102–113.

15. Goo HW. Volumetric severity assessment of Ebstein anomaly using three-dimensional cardiac CT: a feasibility study. Cardiovasc Imaging Asia. 2019; 3:61–67.

16. Rowe SP, Johnson PT, Fishman EK. Cinematic rendering of cardiac CT volumetric data: principles and initial observations. J Cardiovasc Comput Tomogr. 2018; 12:56–59.

17. Avendi MR, Kheradvar A, Jafarkhani H. A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI. Med Image Anal. 2016; 30:108–119.

18. Milgram P, Kishino F. A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst. 1994; E77-D:1321–1329.

19. Seslar SP, Shepard CW, Giroud JM, Aiello VD, Cook AC, Spicer DE, et al. Archiving Working Group of The International Society for Nomenclature of Paediatric and Congenital Heart Disease. Lost treasures: a plea for the systematic preservation of cadaveric heart specimens through three-dimensional digital imaging. Cardiol Young. 2015; 25:1457–1459.

20. Kiraly L, Kiraly B, Szigeti K, Tamas CZ, Daranyi S. Virtual museum of congenital heart defects: digitization and establishment of a database for cardiac specimens. Quant Imaging Med Surg. 2019; 9:115–126.

21. Farooqi KM, Uppu SC, Nguyen K, Srivastava S, Ko HH, Choueiter N, et al. Application of virtual three-dimensional models for simultaneous visualization of intracardiac anatomic relationships in double outlet right ventricle. Pediatr Cardiol. 2016; 37:90–98.

22. Speggiorin S, Durairaj S, Mimic B, Corno AF. Virtual 3D modeling of airways in congenital heart defects. Front Pediatr. 2016; 4:116.

23. Peters TM, Linte CA. Image-guided interventions and computer-integrated therapy: quo vadis? Med Image Anal. 2016; 33:56–63.

24. Choi H, Cho B, Masamune K, Hashizume M, Hong J. An effective visualization technique for depth perception in augmented reality-based surgical navigation. Int J Med Robot. 2016; 12:62–72.

25. Bruckheimer E, Rotschild C, Dagan T, Amir G, Kaufman A, Gelman S, et al. Computer-generated real-time digital holography: first time use in clinical medical imaging. Eur Heart J Cardiovasc Imaging. 2016; 17:845–849.

26. Ender J, Koncar-Zeh J, Mukherjee C, Jacobs S, Borger MA, Viola C, et al. Value of augmented reality-enhanced transesophageal echocardiography (TEE) for determining optimal annuloplasty ring size during mitral valve repair. Ann Thorac Surg. 2008; 86:1473–1478.

27. Belhaj Soulami R, Verhoye JP, Nguyen Duc H, Castro M, Auffret V, Anselmi A, et al. Computer-assisted transcatheter heart valve implantation in valve-in-valve procedures. Innovations (Phila). 2016; 11:193–200.

28. Opolski MP, Michałowska IM, Borucki BA, Nicińska B, Szumowski Ł, Sterliński M. Augmented-reality computed tomography-guided transcatheter pacemaker implantation in dextrocardia and congenitally corrected transposition of great arteries. Cardiol J. 2018; 25:412–413.

29. Brun H, Bugge RAB, Suther LKR, Birkeland S, Kumar R, Pelanis E, et al. Mixed reality holograms for heart surgery planning: first user experience in congenital heart disease. Eur Heart J Cardiovasc Imaging. 2019; 20:883–888.

30. Wierzbicki M, Drangova M, Guiraudon G, Peters T. Validation of dynamic heart models obtained using non-linear registration for virtual reality training, planning, and guidance of minimally invasive cardiac surgeries. Med Image Anal. 2004; 8:387–401.

31. Sørensen TS, Therkildsen SV, Makowski P, Knudsen JL, Pedersen EM. A new virtual reality approach for planning of cardiac interventions. Artif Intell Med. 2001; 22:193–214.

32. Ong CS, Krishnan A, Huang CY, Spevak P, Vricella L, Hibino N, et al. Role of virtual reality in congenital heart disease. Congenit Heart Dis. 2018; 13:357–361.

33. Valverde I, Gomez-Ciriza G, Hussain T, Suarez-Mejias C, Velasco-Forte MN, Byrne N, et al. Three-dimensional printed models for surgical planning of complex congenital heart defects: an international multicentre study. Eur J Cardiothorac Surg. 2017; 52:1139–1148.

34. Olivieri LJ, Krieger A, Loke YH, Nath DS, Kim PC, Sable CA. Three-dimensional printing of intracardiac defects from three-dimensional echocardiographic images: feasibility and relative accuracy. J Am Soc Echocardiogr. 2015; 28:392–397.

35. Parimi M, Buelter J, Thanugundla V, Condoor S, Parkar N, Danon S, et al. Feasibility and validity of printing 3D heart models from rotational angiography. Pediatr Cardiol. 2018; 39:653–658.

36. Tandon A, Byrne N, Nieves Velasco Forte Mde L, Zhang S, Dyer AK, Dillenbeck JM, et al. Use of a semi-automated cardiac segmentation tool improves reproducibility and speed of segmentation of contaminated right heart magnetic resonance angiography. Int J Cardiovasc Imaging. 2016; 32:1273–1279.

37. Giannopoulos AA, Chepelev L, Sheikh A, Wang A, Dang W, Akyuz E, et al. 3D printed ventricular septal defect patch: a primer for the 2015 Radiological Society of North America (RSNA) hands-on course in 3D printing. 3D Print Med. 2015; 1:3.

38. Giannopoulos AA, Mitsouras D, Yoo SJ, Liu PP, Chatzizisis YS, Rybicki FJ. Applications of 3D printing in cardiovascular diseases. Nat Rev Cardiol. 2016; 13:701–718.

39. Anwar S, Singh GK, Miller J, Sharma M, Manning P, Billadello JJ, et al. 3D printing is a transformative technology in congenital heart disease. JACC Basic Transl Sci. 2018; 3:294–312.

40. Townsend K, Pietila T. 3D printing and modeling of congenital heart defects: a technical review. Birth Defects Res. 2018; 110:1091–1097.

41. Shin J, Truong QA. Manufacturing better outcomes in cardiovascular intervention: 3D printing in clinical practice today. Curr Treat Options Cardiovasc Med. 2018; 20:95.

42. Hoashi T, Ichikawa H, Nakata T, Shimada M, Ozawa H, Higashida A, et al. Utility of a super-flexible threedimensional printed heart model in congenital heart surgery. Interact Cardiovasc Thorac Surg. 2018; 27:749–755.

43. Scanlan AB, Nguyen AV, Ilina A, Lasso A, Cripe L, Jegatheeswaran A, et al. Comparison of 3D echocardiogram-derived 3D printed valve models to molded models for simulated repair of pediatric atrioventricular valves. Pediatr Cardiol. 2018; 39:538–547.

44. Vukicevic M, Mosadegh B, Min JK, Little SH. Cardiac 3D printing and its future directions. JACC Cardiovasc Imaging. 2017; 10:171–184.

45. Biglino G, Capelli C, Wray J, Schievano S, Leaver LK, Khambadkone S, et al. 3D-manufactured patient-specific models of congenital heart defects for communication in clinical practice: feasibility and acceptability. BMJ Open. 2015; 5:e007165.

46. Cantinotti M, Valverde I, Kutty S. Three-dimensional printed models in congenital heart disease. Int J Cardiovasc Imaging. 2017; 33:137–144.

47. Lau I, Sun Z. Three-dimensional printing in congenital heart disease: a systematic review. J Med Radiat Sci. 2018; 65:226–236.

48. Lau I, Wong YH, Yeong CH, Abdul Aziz YF, Md Sari NA, Hashim SA, et al. Quantitative and qualitative comparison of low- and high-cost 3D-printed heart models. Quant Imaging Med Surg. 2019; 9:107–114.

49. Binder TM, Moertl D, Mundigler G, Rehak G, Franke M, Delle-Karth G, et al. Stereolithographic biomodeling to create tangible hard copies of cardiac structures from echocardiographic data: in vitro and in vivo validation. J Am Coll Cardiol. 2000; 35:230–237.

50. Olivieri LJ, Zurakowski D, Ramakrishnan K, Su L, Alfares FA, Irwin MR, et al. Novel, 3D display of heart models in the postoperative care setting improves CICU caregiver confidence. World J Pediatr Congenit Heart Surg. 2018; 9:206–213.

51. Costello JP, Olivieri LJ, Su L, Krieger A, Alfares F, Thabit O, et al. Incorporating three-dimensional printing into a simulation-based congenital heart disease and critical care training curriculum for resident physicians. Congenit Heart Dis. 2015; 10:185–190.

52. Olivieri LJ, Su L, Hynes CF, Krieger A, Alfares FA, Ramakrishnan K, et al. “Just-In-Time” simulation training using 3-D printed cardiac models after congenital cardiac surgery. World J Pediatr Congenit Heart Surg. 2016; 7:164–168.

53. Kurup HK, Samuel BP, Vettukattil JJ. Hybrid 3D printing: a game-changer in personalized cardiac medicine? Expert Rev Cardiovasc Ther. 2015; 13:1281–1284.

54. Gosnell J, Pietila T, Samuel BP, Kurup HK, Haw MP, Vettukattil JJ. Integration of computed tomography and three-dimensional echocardiography for hybrid three-dimensional printing in congenital heart disease. J Digit Imaging. 2016; 29:665–669.

55. Biglino G, Verschueren P, Zegels R, Taylor AM, Schievano S. Rapid prototyping compliant arterial phantoms for in-vitro studies and device testing. J Cardiovasc Magn Reson. 2013; 15:2.

56. Bramlet M, Olivieri L, Farooqi K, Ripley B, Coakley M. Impact of three-dimensional printing on the study and treatment of congenital heart disease. Circ Res. 2017; 120:904–907.

57. Moore RA, Riggs KW, Kourtidou S, Schneider K, Szugye N, Troja W, et al. Three-dimensional printing and virtual surgery for congenital heart procedural planning. Birth Defects Res. 2018; 110:1082–1090.

58. Short DB, Sirinterlikci A, Badger P, Artieri B. Environmental, health, and safety issues in rapid prototyping. Rapid Prototyp J. 2015; 21:105–110.

59. Randolph SA. 3D printing: what are the hazards? Workplace Health Saf. 2018; 66:164.

60. Chan FL, House R, Kudla I, Lipszyc JC, Rajaram N, Tarlo SM. Health survey of employees regularly using 3D printers. Occup Med (Lond). 2018; 68:211–214.

61. Melchiorri AJ, Hibino N, Best CA, Yi T, Lee YU, Kraynak CA, et al. 3D-printed biodegradable polymeric vascular grafts. Adv Healthc Mater. 2016; 5:319–325.

62. Sun Y, Zhang X, Li W, Di Y, Xing Q, Cao Q. 3D printing and biocompatibility study of a new biodegradable occluder for cardiac defect. J Cardiol. 2019; 74:182–188.