Ann Surg Treat Res.  2022 Mar;102(3):147-152. 10.4174/astr.2022.102.3.147.

External validation of risk prediction platforms for pancreatic fistula after pancreatoduodenectomy using nomograms and artificial intelligence

• Yoon SJ
• Kwon W
• Lee OJ
• Jung JH
• Shin YC
• Lim CS
• Kim H
• Jang JY
• Shin SH
• Heo JS
• Han IW

• KMID: 2526935
• DOI: http://doi.org/10.4174/astr.2022.102.3.147

Abstract

Purpose
Postoperative pancreatic fistula (POPF) is a life-threatening complication following pancreatoduodenectomy (PD). We previously developed nomogram- and artificial intelligence (AI)-based risk prediction platforms for POPF after PD. This study aims to externally validate these platforms.
Methods
Between January 2007 and December 2016, a total of 1,576 patients who underwent PD in Seoul National University Hospital, Ilsan Paik Hospital, and Boramae Medical Center were retrospectively reviewed. The individual risk scores for POPF were calculated using each platform by Samsung Medical Center. The predictive ability was evaluated using a receiver operating characteristic curve and the area under the curve (AUC). The optimal predictive value was obtained via backward elimination in accordance with the results from the AI development process.
Results
The AUC of the nomogram after external validation was 0.679 (P < 0.001). The values of AUC after backward elimination in the AI model varied from 0.585 to 0.672. A total of 13 risk factors represented the maximal AUC of 0.672 (P < 0.001).
Conclusion
We performed external validation of previously developed platforms for predicting POPF. Further research is needed to investigate other potential risk factors and thereby improve the predictability of the platform.

Keyword

Artificial intelligence; Nomograms; Pancreatic fistula; Pancreatoduodenectomy; Postoperative complications

Figure

• 

  Fig. 1 (A) The web-based nomogram calculator (http://popf.smchbp.org). (B) The web-based artificial intelligence (AI) calculator (http://popfrisk.smchbp.org).

• 

  Fig. 2 (A) The receiver operating characteristic (ROC) of the nomogram. Area under the curve (AUC) = 0.679, P < 0.001. (B) The ROC of the artificial intelligence predictor. AUC = 0.672, P < 0.001.

• 

  Fig. 3 The area under the curve (AUC) values with backward elimination.

Reference

1. McMillan MT, Vollmer CM Jr, Asbun HJ, Ball CG, Bassi C, Beane JD, et al. The characterization and prediction of ISGPF grade c fistulas following pancreatoduodenectomy. J Gastrointest Surg. 2016; 20:262–276. PMID: 26162925.

2. Nahm CB, Connor SJ, Samra JS, Mittal A. Postoperative pancreatic fistula: a review of traditional and emerging concepts. Clin Exp Gastroenterol. 2018; 11:105–118. PMID: 29588609.

3. Bassi C, Marchegiani G, Dervenis C, Sarr M, Abu Hilal M, Adham M, et al. The 2016 update of the International Study Group (ISGPS) definition and grading of postoperative pancreatic fistula: 11 years after. Surgery. 2017; 161:584–591. PMID: 28040257.

4. Eshmuminov D, Schneider MA, Tschuor C, Raptis DA, Kambakamba P, Muller X, et al. Systematic review and meta-analysis of postoperative pancreatic fistula rates using the updated 2016 International Study Group Pancreatic Fistula definition in patients undergoing pancreatic resection with soft and hard pancreatic texture. HPB (Oxford). 2018; 20:992–1003. PMID: 29807807.

5. Callery MP, Pratt WB, Kent TS, Chaikof EL, Vollmer CM Jr. A prospectively validated clinical risk score accurately predicts pancreatic fistula after pancreatoduodenectomy. J Am Coll Surg. 2013; 216:1–14. PMID: 23122535.

6. Mungroop TH, van Rijssen LB, van Klaveren D, Smits FJ, van Woerden V, Linnemann RJ, et al. Alternative fistula risk score for pancreatoduodenectomy (a-FRS): design and international external validation. Ann Surg. 2019; 269:937–943. PMID: 29240007.

7. Ryu Y, Shin SH, Park DJ, Kim N, Heo JS, Choi DW, et al. Validation of original and alternative fistula risk scores in postoperative pancreatic fistula. J Hepatobiliary Pancreat Sci. 2019; 26:354–359. PMID: 31125494.

8. Kang JS, Park T, Han Y, Lee S, Kim JR, Kim H, et al. Clinical validation of scoring systems of postoperative pancreatic fistula after pancreatoduodenectomy: applicability to Eastern cohorts? Hepatobiliary Surg Nutr. 2019; 8:211–218. PMID: 31245401.

9. You Y, Han IW, Choi DW, Heo JS, Ryu Y, Park DJ, et al. Nomogram for predicting postoperative pancreatic fistula. HPB (Oxford). 2019; 21:1436–1445. PMID: 30982739.

10. Han IW, Cho K, Ryu Y, Shin SH, Heo JS, Choi DW, et al. Risk prediction platform for pancreatic fistula after pancreatoduodenectomy using artificial intelligence. World J Gastroenterol. 2020; 26:4453–4464. PMID: 32874057.

11. Shinde RS, Acharya R, Chaudhari VA, Bhandare MS, Mungroop TH, Klompmaker S, et al. External validation and comparison of the original, alternative and updated-alternative fistula risk scores for the prediction of postoperative pancreatic fistula after pancreatoduodenectomy. Pancreatology. 2020; 20:751–756. PMID: 32340876.

12. Lao M, Zhang X, Guo C, Chen W, Zhang Q, Ma T, et al. External validation of alternative fistula risk score (a-FRS) for predicting pancreatic fistula after pancreatoduodenectomy. HPB (Oxford). 2020; 22:58–66. PMID: 31272847.

13. Huang XT, Huang CS, Liu C, Chen W, Cai JP, Cheng H, et al. Development and validation of a new nomogram for predicting clinically relevant postoperative pancreatic fistula after pancreatoduodenectomy. World J Surg. 2021; 45:261–269. PMID: 32901325.

14. Suzuki S, Shimoda M, Shimazaki J, Oshiro Y, Nishida K, Shiihara M, et al. Drain lipase levels and decreased rate of drain amylase levels as independent predictors of pancreatic fistula with nomogram after pancreaticoduodenectomy. World J Surg. 2021; 45:1921–1928. PMID: 33721069.

15. Molinari E, Bassi C, Salvia R, Butturini G, Crippa S, Talamini G, et al. Amylase value in drains after pancreatic resection as predictive factor of postoperative pancreatic fistula: results of a prospective study in 137 patients. Ann Surg. 2007; 246:281–287. PMID: 17667507.

16. Bertens KA, Crown A, Clanton J, Alemi F, Alseidi AA, Biehl T, et al. What is a better predictor of clinically relevant postoperative pancreatic fistula (CR-POPF) following pancreaticoduodenectomy (PD): postoperative day one drain amylase (POD1DA) or the fistula risk score (FRS)? HPB (Oxford). 2017; 19:75–81. PMID: 27825541.

17. Melloul E, Lassen K, Roulin D, Grass F, Perinel J, Adham M, et al. Guidelines for perioperative care for pancreatoduodenectomy: Enhanced Recovery After Surgery (ERAS) Recommendations 2019. World J Surg. 2020; 44:2056–2084. PMID: 32161987.

18. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015; 521:436–444. PMID: 26017442.

19. Borsboom D, Mellenbergh GJ, van Heerden J. The theoretical status of latent variables. Psychol Rev. 2003; 110:203–219. PMID: 12747522.

20. Song M, Greenbaum J, Luttrell J 4th, Zhou W, Wu C, Shen H, et al. A review of integrative imputation for multi-omics datasets. Front Genet. 2020; 11:570255. PMID: 33193667.

21. Kambakamba P, Mannil M, Herrera PE, Müller PC, Kuemmerli C, Linecker M, et al. The potential of machine learning to predict postoperative pancreatic fistula based on preoperative, non-contrast-enhanced CT: a proof-of-principle study. Surgery. 2020; 167:448–454. PMID: 31727325.

22. Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts H. Artificial intelligence in radiology. Nat Rev Cancer. 2018; 18:500–510. PMID: 29777175.