Clin Endosc.  2020 Mar;53(2):132-141. 10.5946/ce.2020.038.

Artificial Intelligence in Gastrointestinal Endoscopy

• Abadir AP
• Ali MF
• Karnes W
• Samarasena JB

• KMID: 2500882
• DOI: http://doi.org/10.5946/ce.2020.038

Abstract

Artificial intelligence (AI) is rapidly integrating into modern technology and clinical practice. Although in its nascency, AI has become a hot topic of investigation for applications in clinical practice. Multiple fields of medicine have embraced the possibility of a future with AI assisting in diagnosis and pathology applications.
In the field of gastroenterology, AI has been studied as a tool to assist in risk stratification, diagnosis, and pathologic identification. Specifically, AI has become of great interest in endoscopy as a technology with substantial potential to revolutionize the practice of a modern gastroenterologist. From cancer screening to automated report generation, AI has touched upon all aspects of modern endoscopy.
Here, we review landmark AI developments in endoscopy. Starting with broad definitions to develop understanding, we will summarize the current state of AI research and its potential applications. With innovation developing rapidly, this article touches upon the remarkable advances in AI-assisted endoscopy since its initial evaluation at the turn of the millennium, and the potential impact these AI models may have on the modern clinical practice. As with any discussion of new technology, its limitations must also be understood to apply clinical AI tools successfully.

Keyword

Artificial intelligence; Colonoscopy; Computer assisted diagnosis; Early detection of cancer; Endoscopy

Figure

• 

  Fig. 1. Convolutional neural network (CNN) identification of a polyp. The CNN identifies the edges of a polyp via color and contour patterns. It further expands its recognition, going through multiple pathways to expand the identification area. The CNN identifies a pattern that it suspects as a polyp, tracing the borders of the lesion. Finally, the CNN reports the lesion to the endoscopist via a rectangular box around the polyp on the colonoscope output feed.

• 

  Fig. 2. Optical pathology algorithm of colon polyps showing an adenoma prediction (A) and serrated polyp prediction (B).

• 

  Fig. 3. Optical pathology algorithm predicting an area of dysplasia within a segment of Barrett’s esophagus.

Reference

1. Alagappan M, Brown JRG, Mori Y, Berzin TM. Artificial intelligence in gastrointestinal endoscopy: the future is almost here. World J Gastrointest Endosc. 2018; 10:239–249.

2. Silver D, Huang A, Maddison CJ, et al. Mastering the game of Go with deep neural networks and tree search. Nature. 2016; 529:484–489.

3. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017; 542:115–118.

4. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016; 316:2402–2410.

5. Urban G, Tripathi P, Alkayali T, et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology. 2018; 155:1069–1078. e8.

6. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015; 521:436–444.

7. Taigman Y, Yang M, Ranzato M, Wolf L. Deepface: closing the gap to human-level performance in face verification. In : In: 2014 IEEE Conference on Computer Vision and Pattern Recognition; 2014 Jun 23-28; Columbus (OH), USA. Piscataway (NJ). IEEE. 2014. p. 1701–1708.

8. Ruffle JK, Farmer AD, Aziz Q. Artificial intelligence-assisted gastroenterology- promises and pitfalls. Am J Gastroenterol. 2019; 114:422–428.

9. National Cancer Institute. Cancer stat facts: common cancer sites [Internet]. Bethesda (MD): NCI;c2019. [cited 2020 Feb 3]. Available from: https://seer.cancer.gov/statfacts/html/common.html.

10. Corley DA, Jensen CD, Marks AR, et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med. 2014; 370:1298–1306.

11. Kaminski MF, Wieszczy P, Rupinski M, et al. Increased rate of adenoma detection associates with reduced risk of colorectal cancer and death. Gastroenterology. 2017; 153:98–105.

12. Karnes WE, Alkayali T, Mittal M, et al. Automated polyp detection using deep learning: leveling the field. Gastrointest Endosc. 2017; 85(5 Suppl):AB376–AB377.

13. Karnes WE, Ninh A, Urban G, Baldi P. Adenoma detection through deep learning: 2017 presidentialposter award. Am J Gastroenterol. 2017; 112:S136.

14. Tripathi PV, Urban G, Alkayali T, et al. Computer-assisted polyp detection identifies all polyps found by expert colonoscopists – and then some. Gastroenterology. 2018; 154(6 Suppl 1):S–36.

15. Tajbakhsh N, Gurudu SR, Liang J. Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans Med Imaging. 2016; 35:630–644.

16. Fernández-Esparrach G, Bernal J, López-Cerón M, et al. Exploring the clinical potential of an automatic colonic polyp detection method based on the creation of energy maps. Endoscopy. 2016; 48:837–842.

17. Puig I, Kaltenbach T. Optical diagnosis for colorectal polyps: a useful technique now or in the future? Gut Liver. 2018; 12:385–392.

18. Min M, Su S, He W, Bi Y, Ma Z, Liu Y. Computer-aided diagnosis of colorectal polyps using linked color imaging colonoscopy to predict histology. Sci Rep. 2019; 9:2881.

19. Kominami Y, Yoshida S, Tanaka S, et al. Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy. Gastrointest Endosc. 2016; 83:643–649.

20. Aihara H, Saito S, Inomata H, et al. Computer-aided diagnosis of neoplastic colorectal lesions using ‘real-time’ numerical color analysis during autofluorescence endoscopy. Eur J Gastroenterol Hepatol. 2013; 25:488–494.

21. Renner J, Phlipsen H, Haller B, et al. Optical classification of neoplastic colorectal polyps - a computer-assisted approach (the COACH study). Scand J Gastroenterol. 2018; 53:1100–1106.

22. Zachariah R, Ninh A, Dao T, Requa J, Karnes W. Can artificial intelligence (AI) achieve real-time ‘resect and discard‘ thresholds independently of device or operator? Am J Gastroenterol. 2018; 113:S129.

23. Mori Y, Kudo SE, Wakamura K, et al. Novel computer-aided diagnostic system for colorectal lesions by using endocytoscopy (with videos). Gastrointest Endosc. 2015; 81:621–629.

24. Misawa M, Kudo S-E, Mori Y, et al. Characterization of colorectal lesions using a computer-aided diagnostic system for narrow-band imaging endocytoscopy. Gastroenterology. 2016; 150:1531–1532. e3.

25. Karnes WE, Ninh A, Dao T, Requa J, Samarasena JB. Real-time identification of anatomic landmarks during colonoscopy using deep learning. Gastrointest Endosc. 2018; 87(6 Suppl):AB252.

26. Samarasena J, Yu AR, Torralba EJ, et al. Artificial intelligence can accurately detect tools used during colonoscopy: another step forward toward autonomous report writing: presidential poster award. Am J Gastroenterol. 2018; 113:S619–S620.

27. Karnes WE, Ninh A, Dao T, Requa J, Samarasena JB. Unambiguous real-time scoring of bowel preparation using artificial intelligence. Gastrointest Endosc. 2018; 87(6 Suppl):AB258.

28. Requa J, Dao T, Ninh A, Karnes W. Can a convolutional neural network solve the polyp size dilemma? Category award (colorectal cancer prevention) presidential poster award. Am J Gastroenterol. 2018; 113:S158.

29. Mossotto E, Ashton JJ, Coelho T, Beattie RM, MacArthur BD, Ennis S. Classification of paediatric inflammatory bowel disease using machine learning. Sci Rep. 2017; 7:2427.

30. Maeda Y, Kudo SE, Mori Y, et al. Fully automated diagnostic system with artificial intelligence using endocytoscopy to identify the presence of histologic inflammation associated with ulcerative colitis (with video). Gastrointest Endosc. 2019; 89:408–415.

31. Paine ER. Colonoscopic evaluation in ulcerative colitis. Gastroenterol Rep (Oxf). 2014; 2:161–168.

32. Panés J, Feagan BG, Hussain F, Levesque BG, Travis SP. Central endoscopy reading in inflammatory bowel diseases. J Crohns Colitis. 2016; 10(Suppl 2):S542–S547.

33. Abadir AP, Requa J, Ninh A, Karnes W, Mattar M. Unambiguous real- time endoscopic scoring of ulcerative colitis using a convolutional neural network. Am J Gastroenterol. 2018; 113:S349.

34. Bossuyt P, Vermeire S, Bisschops R. Scoring endoscopic disease activity in IBD: artificial intelligence sees more and better than we do. Gut. 2020; 69:788–789.

35. Hirasawa T, Aoyama K, Tanimoto T, et al. Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer. 2018; 21:653–660.

36. Wu L, Zhang J, Zhou W, et al. Randomised controlled trial of WISENSE, a real-time quality improving system for monitoring blind spots during esophagogastroduodenoscopy. Gut. 2019; 68:2161–2169.

37. Kanesaka T, Lee TC, Uedo N, et al. Computer-aided diagnosis for identifying and delineating early gastric cancers in magnifying narrow-band imaging. Gastrointest Endosc. 2018; 87:1339–1344.

38. Itoh T, Kawahira H, Nakashima H, Yata N. Deep learning analyzes Helicobacter pylori infection by upper gastrointestinal endoscopy images. Endosc Int Open. 2018; 6:E139–E144.

39. Shichijo S, Nomura S, Aoyama K, et al. Application of convolutional neural networks in the diagnosis of Helicobacter pylori infection based on endoscopic images. EBioMedicine. 2017; 25:106–111.

40. Kubota K, Kuroda J, Yoshida M, Ohta K, Kitajima M. Medical image analysis: computer-aided diagnosis of gastric cancer invasion on endoscopic images. Surg Endosc. 2012; 26:1485–1489.

41. Sharma H, Zerbe N, Klempert I, Hellwich O, Hufnagl P. Deep convolutional neural networks for automatic classification of gastric carcinoma using whole slide images in digital histopathology. Comput Med Imaging Graph. 2017; 61:2–13.

42. Schölvinck DW, van der Meulen K, Bergman J, Weusten B. Detection of lesions in dysplastic Barrett’s esophagus by community and expert endoscopists. Endoscopy. 2017; 49:113–120.

43. van der Sommen F, Zinger S, Curvers WL, et al. Computer-aided detection of early neoplastic lesions in Barrett’s esophagus. Endoscopy. 2016; 48:617–624.

44. de Groof J, van der Sommen F, van der Putten J, et al. The Argos project: the development of a computer-aided detection system to improve detection of Barrett’s neoplasia on white light endoscopy. United European Gastroenterol J. 2019; 7:538–547.

45. Hashimoto R, Lugo M, Mai D, et al. Artificial intelligence dysplasia detection (Aidd) algorithm for Barrett’s esophagus. Gastrointest Endosc. 2019; 89(6 Suppl):AB99–AB100.

46. Horie Y, Yoshio T, Aoyama K, et al. Diagnostic outcomes of esophageal cancer by artificial intelligence using convolutional neural networks. Gastrointest Endosc. 2019; 89:25–32.

47. Cai SL, Li B, Tan WM, et al. Using a deep learning system in endoscopy for screening of early esophageal squamous cell carcinoma (with video). Gastrointest Endosc. 2019; 90:745–753. e2.

48. Byrne MF, Donnellan F. Artificial intelligence and capsule endoscopy: is the truly “smart” capsule nearly here? Gastrointest Endosc. 2019; 89:195–197.

49. Shiotani A, Honda K, Kawakami M, et al. Analysis of small-bowel capsule endoscopy reading by using quickview mode: training assistants for reading may produce a high diagnostic yield and save time for physicians. J Clin Gastroenterol. 2012; 46:e92–e95.

50. Leenhardt R, Vasseur P, Li C, et al. A neural network algorithm for detection of GI angiectasia during small-bowel capsule endoscopy. Gastrointest Endosc. 2019; 89:189–194.

51. Lui F, Rusconi-Rodrigues Y, Ninh A, Requa J, Karnes W. Highly sensitive and specific identification of anatomical landmarks and mucosal abnormalities in video capsule endoscopy with convolutional neural networks. Am J Gastroenterol. 2018; 113:S670–S671.

52. Ding Z, Shi H, Zhang H, et al. Gastroenterologist-level identification of small-bowel diseases and normal variants by capsule endoscopy using a deep-learning model. Gastroenterology. 2019; 157:1044–1054. e5.

53. Hwang Y, Park J, Lim YJ, Chun HJ. Application of artificial intelligence in capsule endoscopy: where are we now? Clin Endosc. 2018; 51:547–551.

54. Wang P, Berzin TM, Glissen Brown JR, et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut. 2019; 68:1813–1819.