Ann Lab Med.  2024 Jan;44(1):6-20. 10.3343/alm.2024.44.1.6.

Bias in Laboratory Medicine: The Dark Side of the Moon

• Coskun A

• KMID: 2550211
• DOI: http://doi.org/10.3343/alm.2024.44.1.6

Abstract

Physicians increasingly use laboratory-produced information for disease diagnosis, patient monitoring, treatment planning, and evaluations of treatment effectiveness. Bias is the systematic deviation of laboratory test results from the actual value, which can cause misdiagnosis or misestimation of disease prognosis and increase healthcare costs. Properly estimating and treating bias can help to reduce laboratory errors, improve patient safety, and considerably reduce healthcare costs. A bias that is statistically and medically significant should be eliminated or corrected. In this review, the theoretical aspects of bias based on metrological, statistical, laboratory, and biological variation principles are discussed. These principles are then applied to laboratory and diagnostic medicine for practical use from clinical perspectives.

Keyword

Bias; Confidence interval; Diagnostic error; Quality control; Total quality management; Uncertainty

Figure

• 

  Fig. 1 Bias is the difference between the target/reference value and the mean value of repeated measurements of the sample. (A) The estimation of bias requires two main components: (1) reference quantity or assigned value and (2) replicate measurements of the quantity. (B) If the reference quantity value is not available, an assigned value can be used to estimate the bias.

• 

  Fig. 2 Constant and proportional bias. If a≠1 and b≠0, the significance of a and b should be evaluated using the 95% confidence intervals of a and b.

• 

  Fig. 3 Characteristics of bias change over time. Data collected for sodium under intermediate precision or reproducibility conditions contain both random and systematic (bias) variations.

• 

  Fig. 4 Effect of bias on population values inside and outside the reference interval. Given the geometric shape of the normal distribution curve, an increase in bias results in an exponential shift of the population from within the reference intervals to beyond them. (A) When bias=0, 5% of the population is situated outside the reference intervals. (B) When bias >0, the proportion of the population located outside the reference intervals exceeds 5%.

• 

  Fig. 5 Bias and area under the curve inside and outside reference intervals. For a given bias value, this diagram can be used to easily estimate the population values inside and outside reference intervals.

• 

  Fig. 6 Sigma metrics is the number of SDs located between the target and upper/lower limits. 1.5 SD shift is considered the standard bias.

• 

  Fig. 7 Effect of bias on true and false positive and negatives.

• 

  Fig. 8 Bias and the diagnostic accuracy of laboratory tests. Positive or negative bias dramatically affects the diagnostic accuracy of laboratory tests. (A) Acceptable bias does not have a significant negative effect on the diagnostic accuracy of laboratory tests. (B) Increasing bias can lead to misdiagnosis of diseases (blue area) and can dramatically impair the diagnostic power of laboratory tests.

Reference

1. Fraser CG. 2001. Biological variation: from principles to practice. AACC Press;Washington, DC:

2. Forsman RW. 1996; Why is the laboratory an afterthought for managed care organizations? Clin Chem. 42:813–6. DOI: 10.1093/clinchem/42.5.813. PMID: 8653920.

3. McRae MP, Rajsri KS, Alcorn TM, McDevitt JT. 2022; Smart diagnostics: Combining artificial intelligence and in vitro diagnostics. Sensors (Basel). 22:6355. DOI: 10.3390/s22176355. PMID: 36080827. PMCID: PMC9459970.

4. Hicks AJ, Carwardine ZL, Hallworth MJ, Kilpatrick ES. 2021; Using clinical guidelines to assess the potential value of laboratory medicine in clinical decision-making. Biochem Med (Zagreb). 31:010703. DOI: 10.11613/BM.2021.010703. PMID: 33380890. PMCID: PMC7745157.

5. Dybkaer R. 1995; Result, error and uncertainty. Scand J Clin Lab Invest. 55:97–118. DOI: 10.3109/00365519509089602. PMID: 7667613.

6. Frenkel R, Farrance I, Badrick T. 2019; Bias in analytical chemistry: A review of selected procedures for incorporating uncorrected bias into the expanded uncertainty of analytical measurements and a graphical method for evaluating the concordance of reference and test procedures. Clin Chim Acta. 495:129–38. DOI: 10.1016/j.cca.2019.03.1633. PMID: 30935874.

7. Nielsen AA, Petersen PH, Green A, Christensen C, Christensen H, Brandslund I. 2014; Changing from glucose to HbA1c for diabetes diagnosis: Predictive values of one test and importance of analytical bias and imprecision. Clin Chem Lab Med. 52:1069–77. DOI: 10.1515/cclm-2013-0337. PMID: 24659606.

8. Weykamp C, John G, Gillery P, English E, Ji L, Lenters-Westra E, et al. 2015; Investigation of 2 models to set and evaluate quality targets for hb a1c: Biological variation and sigma-metrics. Clin Chem. 61:752–9. DOI: 10.1373/clinchem.2014.235333. PMID: 25737535. PMCID: PMC4946649.

9. Mower WR. 1999; Evaluating bias and variability in diagnostic test reports. Ann Emerg Med. 33:85–91. DOI: 10.1016/S0196-0644(99)70422-1. PMID: 9867892.

10. Schmidt RL, Factor RE. 2013; Understanding sources of bias in diagnostic accuracy studies. Arch Pathol Lab Med. 137:558–65. DOI: 10.5858/arpa.2012-0198-RA. PMID: 23544945.

11. Haeckel R, Gurr E, Hoff T. on behalf of the working group Guide Limits of the German Society of Clinical Chemistry and Laboratory Medicine (DGKL). 2016; Bias, its minimization or circumvention to simplify internal quality assurance. J Lab Med. 40:263–70. DOI: 10.1515/labmed-2016-0036.

12. Theodorsson E, Magnusson B, Leito I. 2014; Bias in clinical chemistry. Bioanalysis. 6:2855–75. DOI: 10.4155/bio.14.249. PMID: 25486232.

13. Internatıonal Organızation of Legal Metrology. International vocabulary of metrology-Basic and general concepts and associated terms (VIM) 3rd ed. https://www.oiml.org/en/files/pdf_v/v002-200-e07.pdf. Updated on Aug 2008.

14. Eurachem. Treatment of an observed bias. https://www.eurachem.org/index.php/publications/leaflets/bias-trt-01. Updated on Oct 2022.

15. CLSI. 2018. Measurement procedure comparison and bias estimation using patient samples. 3rd ed. CLSI EP09c. Clinical and Laboratory Standards Institute;Wayne, PA:

16. Tate J, Panteghini M. 2007; Standardisation - The theory and the practice. Clin Biochem Rev. 28:93–6.

17. Johnson R. Assessment of bias with emphasis on method comparison. Clin Biochem Rev. 2008; 29(Suppl 1):S37–42.

18. Millsap RE. 2012. Statistical approaches to measurement invariance. Routledge;Milton Park:

19. Livesey JH, Ellis MJ, Evans MJ. 2008; Pre-analytical requirements. Clin Biochem Rev. 29(suppl 1):S11–5.

20. Sandberg S, Carobene A, Bartlett B, Coskun A, Fernandez-Calle P, Jonker N, et al. 2022; Biological variation: Recent development and future challenges. Clin Chem Lab Med. 61:741–50. DOI: 10.1515/cclm-2022-1255. PMID: 36537071.

21. Wikipedia. Albert Einstein. https://en.wikiquote.org/wiki/Albert_Einstein. Updated on March 2023.

22. Menditto A, Patriarca M, Magnusson B. 2007; Understanding the meaning of accuracy, trueness and precision. Accredit Qual Assur. 12:45–7. DOI: 10.1007/s00769-006-0191-z.

23. Jhang JS, Chang CC, Fink DJ, Kroll MH. 2004; Evaluation of linearity in the clinical laboratory. Arch Pathol Lab Med. 128:44–8. DOI: 10.5858/2004-128-44-EOLITC. PMID: 14692813.

24. Jeong TD, Kim SK, Kim S, Lim CY, Chung JW. 2022; Comparison between polynomial regression and weighted least squares regression analysis for verification of analytical measurement range. Clin Chem Lab Med. 60:989–94. DOI: 10.1515/cclm-2022-0018. PMID: 35531706.

25. Hazra A, Gogtay N. 2016; Biostatistics series module 6: Correlation and linear regression. Indian J Dermatol. 61:593–601. DOI: 10.4103/0019-5154.193662. PMID: 27904175. PMCID: PMC5122272.

26. Magari RT. 2000; Evaluating agreement between two analytical methods in clinical chemistry. Clin Chem Lab Med. 38:1021–5. DOI: 10.1515/CCLM.2000.151. PMID: 11140617.

27. Ludbrook J. 1997; Comparing methods of measurements. Clin Exp Pharmacol Physiol. 24:193–203. DOI: 10.1111/j.1440-1681.1997.tb01807.x. PMID: 9075596.

28. Martínez À, Del Río FJ, Riu J, Rius FX. 1999; Detecting proportional and constant bias in method comparison studies by using linear regression with errors in both axes. Chemom Intell Lab Syst. 49:179–93. DOI: 10.1016/S0169-7439(99)00036-2.

29. Lu MJ, Zhong WH, Liu YX, Miao HZ, Li YC, Ji MH. 2016; Sample size for assessing agreement between two methods of measurement by Bland-Altman method. Int J Biostat. 12:20150039. DOI: 10.1515/ijb-2015-0039. PMID: 27838682.

30. Zaki R, Bulgiba A, Ismail NA. 2013; Testing the agreement of medical instruments: Overestimation of bias in the Bland-Altman analysis. Prev Med. 57:S80–2. DOI: 10.1016/j.ypmed.2013.01.003. PMID: 23313586.

31. Giavarina D. 2015; Understanding Bland Altman analysis. Biochem Med (Zagreb). 25:141–51. DOI: 10.11613/BM.2015.015. PMID: 26110027. PMCID: PMC4470095.

32. Bilić-Zulle L. 2011; Comparison of methods: Passing and Bablok regression. Biochem Med (Zagreb). 21:49–52. DOI: 10.11613/BM.2011.010. PMID: 22141206.

33. Ludbrook J. 2002; Statistical techniques for comparing measurers and methods of measurement: A critical review. Clin Exp Pharmacol Physiol. 29:527–36. DOI: 10.1046/j.1440-1681.2002.03686.x. PMID: 12060093.

34. Ludbrook J. 2010; Linear regression analysis for comparing two measurers or methods of measurement: But which regression? Clin Exp Pharmacol Physiol. 37:692–9. DOI: 10.1111/j.1440-1681.2010.05376.x. PMID: 20337658.

35. Martı́nez A, del Rı́o FJ, Riu J, Rius FX. 1999; Detecting proportional and constant bias in method comparison studies by using linear regression with errors in both axes. Chemometr Intell Lab. 49:179–93. DOI: 10.1016/S0169-7439(99)00036-2.

36. Theodorsson E. Reference materials and reference measuring systems. https://cms.jctlm.org/wp-content/uploads/2023/02/Reference-materials-and-reference-measuring-systems-2022-03-27.pdf. Updated on March 2023.

37. Bunk DM. 2007; Reference materials and reference measurement procedures: An overview from a National Metrology Institute. Clin Biochem Rev. 28:131–7.

38. Westgard S, Bayat H, Westgard JO. 2018; Analytical Sigma metrics: A review of Six Sigma implementation tools for medical laboratories Special issue: Six Sigma metrics Review. Biochem Med. 28:20502. DOI: 10.11613/BM.2018.020502. PMID: 30022879. PMCID: PMC6039161.

39. Coskun A, Theodorsson E, Oosterhuis WP, Sandberg S. European Federation of Clinical Chemistry and Laboratory Medicine Task and Finish Group on Practical Approach to Measurement Uncertainty. 2022; Measurement uncertainty for practical use. Clin Chim Acta. 531:352–60. DOI: 10.1016/j.cca.2022.04.1003. PMID: 35513038.

40. Becker D, Christensen R, Currie L, Gills T, Hertz H, Klouda G, et al. 1992. Use of NIST Standard Reference Materials for decisions on performance of analytical chemical methods and laboratories. NIST Special Publication 829:US Department of Commerce;Washington, DC: DOI: 10.6028/NIST.SP.829.

41. Ioannidis JPA. 2019; Retiring statistical significance would give bias a free pass. Nature. 567:461. DOI: 10.1038/d41586-019-00969-2. PMID: 30903096.

42. Badrick T, Punyalack W, Graham P. 2018; Commutability and traceability in EQA programs. Clin Biochem. 56:102–4. DOI: 10.1016/j.clinbiochem.2018.04.018. PMID: 29684367.

43. Miller WG. 2003; Specimen materials, target values and commutability for external quality assessment (proficiency testing) schemes. Clin Chim Acta. 327:25–37. DOI: 10.1016/S0009-8981(02)00370-4. PMID: 12482616.

44. Braga F, Panteghini M. 2019; Commutability of reference and control materials: An essential factor for assuring the quality of measurements in Laboratory Medicine. Clin Chem Lab Med. 57:967–73. DOI: 10.1515/cclm-2019-0154. PMID: 30903757.

45. Greg Miller W, Greenberg N, Budd J, Delatour V. IFCC Working Group on Commutability in Metrological Traceability. 2021; The evolving role of commutability in metrological traceability. Clin Chim Acta. 514:84–9. DOI: 10.1016/j.cca.2020.12.021. PMID: 33359496.

46. Horvath AR, Bossuyt PMM, Sandberg S, John AS, Monaghan PJ, Verhagen-Kamerbeek WDJ, et al. 2015; Setting analytical performance specifications based on outcome studies - is it possible? Clin Chem Lab Med. 53:841–8. DOI: 10.1515/cclm-2015-0214.

47. JCGM. Evaluation of measurement data-Guide to the expression of uncertainty in measurement. https://www.bipm.org/documents/20126/2071204/JCGM_100_2008_E.pdf. Updated on Sept 2008.

48. Westgard JO. 2018; Error methods are more practical, but uncertainty methods may still be preferred. Clin Chem. 64:636–8. DOI: 10.1373/clinchem.2017.284406. PMID: 29311055.

49. Hahn GJ, Meeker WQ, Escobar LA. 2016. Statistical intervals: A guide for practitioners and researchers. Wiley;Hoboken: p. 651. DOI: 10.1002/9781118594841.

50. Rao RS, Kumar CG, Prakasham RS, Hobbs PJ. 2008; The Taguchi methodology as a statistical tool for biotechnological applications: A critical appraisal. Biotechnol J. 3:510–23. DOI: 10.1002/biot.200700201. PMID: 18320563.

51. Kiran DR. 2017. Quality loss function. Total quality management: Key concepts and case studies. Butterworth-Heinemann;Oxford: p. 439–45.

52. Panteghini M, Sandberg S. 2015; Defining analytical performance specifications 15 years after the Stockholm conference. Clin Chem Lab Med. 53:829–32. DOI: 10.1515/cclm-2015-0303. PMID: 25901719.

53. Fraser CG. 2015; The 1999 Stockholm Consensus Conference on quality specifications in laboratory medicine. Clin Chem Lab Med. 53:837–40. DOI: 10.1515/cclm-2014-0914. PMID: 25720125.

54. Kallner A, McQueen M, Heuck C. 1999; The Stockholm Consensus Conference on Quality Specifications in Laboratory Medicine, 25-26 April 1999. Scand J Clin Lab Invest. 59:475. DOI: 10.1080/00365519950185175. PMID: 10667681.

55. Sandberg S, Fraser CG, Horvath AR, Jansen R, Jones G, Oosterhuis W, et al. 2015; Defining analytical performance specifications: Consensus Statement from the 1st Strategic Conference of the European Federation of Clinical Chemistry and Laboratory Medicine. Clin Chem Lab Med. 53:833–5. DOI: 10.1515/cclm-2015-0067. PMID: 25719329.

56. Haeckel R, Wosniok W, Kratochvila J, Carobene A. 2012; A pragmatic proposal for permissible limits in external quality assessment schemes with a compromise between biological variation and the state of the art. Clin Chem Lab Med. 50:833–9. DOI: 10.1515/cclm-2011-0862. PMID: 22628326.

57. Ceriotti F, Fernandez-Calle P, Klee GG, Nordin G, Sandberg S, Streichert T, et al. 2017; Criteria for assigning laboratory measurands to models for analytical performance specifications defined in the 1st EFLM Strategic Conference. Clin Chem Lab Med. 55:189–94. DOI: 10.1515/cclm-2016-0091. PMID: 27506603.

58. Bartlett WA, Braga F, Carobene A, Coşkun A, Prusa R, Fernandez-Calle P, et al. 2015; A checklist for critical appraisal of studies of biological variation. Clin Chem Lab Med. 53:879–85. DOI: 10.1515/cclm-2014-1127. PMID: 25996385.

59. Aarsand AK, Røraas T, Fernandez-Calle P, Ricos C, Díaz-Garzón J, Jonker N, et al. 2018; The Biological Variation Data Critical Appraisal Checklist: A standard for evaluating studies on biological variation. Clin Chem. 64:501–14. DOI: 10.1373/clinchem.2017.281808. PMID: 29222339.

60. Carobene A, Aarsand AK, Bartlett WA, Coskun A, Diaz-Garzon J, Fernandez-Calle P, et al. 2021; The European Biological Variation Study (EuBIVAS): A summary report. Clin Chem Lab Med. 60:505–17. DOI: 10.1515/cclm-2021-0370. PMID: 34049424.

61. Carobene A, Strollo M, Jonker N, Barla G, Bartlett WA, Sandberg S, et al. 2016; Sample collections from healthy volunteers for biological variation estimates' update: A new project undertaken by the Working Group on Biological Variation established by the European Federation of Clinical Chemistry and Laboratory Medicine. Clin Chem Lab Med. 54:1599–608. DOI: 10.1515/cclm-2016-0035. PMID: 27169681.

62. Díaz-Garzón J, Fernández-Calle P, Minchinela J, Aarsand AK, Bartlett WA, Aslan B, et al. 2019; Biological variation data for lipid cardiovascular risk assessment biomarkers. A systematic review applying the biological variation data critical appraisal checklist (BIVAC). Clin Chim Acta. 495:467–75. DOI: 10.1016/j.cca.2019.05.013. PMID: 31103621.

63. Marques-Garcia F, Boned B, González-Lao E, Braga F, Carobene A, Coskun A, et al. 2022; Critical review and meta-analysis of biological variation estimates for tumor markers. Clin Chem Lab Med. 60:494–504. DOI: 10.1515/cclm-2021-0725. PMID: 35143717.

64. Coskun A, Braga F, Carobene A, Tejedor Ganduxe X, Aarsand AK, Fernández-Calle P, et al. 2019; Systematic review and meta-analysis of within-subject and between-subject biological variation estimates of 20 haematological parameters. Clin Chem Lab Med. 58:25–32. DOI: 10.1515/cclm-2019-0658. PMID: 31503541.

65. Coşkun A, Aarsand AK, Braga F, Carobene A, Díaz-Garzón J, Fernandez-Calle P, et al. 2021; Systematic review and meta-analysis of within-subject and between-subject biological variation estimates of serum zinc, copper and selenium. Clin Chem Lab Med. 60:479–82. DOI: 10.1515/cclm-2021-0723. PMID: 34225400.

66. Diaz-Garzon J, Fernandez-Calle P, Sandberg S, Özcürümez M, Bartlett WA, Coskun A, et al. 2021; Biological variation of cardiac troponins in health and disease: A systematic review and meta-analysis. Clin Chem. 67:256–64. DOI: 10.1093/clinchem/hvaa261. PMID: 33279972.

67. González-Lao E, Corte Z, Simón M, Ricós C, Coskun A, Braga F, et al. 2019; Systematic review of the biological variation data for diabetes related analytes. Clin Chim Acta. 488:61–7. DOI: 10.1016/j.cca.2018.10.031. PMID: 30389455.

68. Fernández-Calle P, Díaz-Garzón J, Bartlett W, Sandberg S, Braga F, Beatriz B, et al. 2021; Biological variation estimates of thyroid related measurands - meta-analysis of BIVAC compliant studies. Clin Chem Lab Med. 60:483–93. DOI: 10.1515/cclm-2021-0904. PMID: 34773727.

69. Jonker N, Aslan B, Boned B, Marqués-García F, Ricós C, Alvarez V, et al. 2020; Critical appraisal and meta-analysis of biological variation estimates for kidney related analytes. Clin Chem Lab Med. 60:469–78. DOI: 10.1515/cclm-2020-1168. PMID: 32970605.

70. Aarsand AK, Fernandez-Calle P, Webster C, Coskun A, GonzalesLao E, Diaz-Garzon J, et al. EFLM Biological Variation Database. https://biologicalvariation.eu/. Updated on July 2023.

71. Petersen PH, De Verdier CH, Groth T, Fraser CG, Blaabjerg O, Hørder M. 1997; The influence of analytical bias on diagnostic misclassifications. Clin Chim Acta. 260:189–206. DOI: 10.1016/S0009-8981(96)06496-0. PMID: 9177913.

72. Hyltoft Petersen P, Lund F, Fraser CG, Sandberg S, Sölétormos G. 2018; Valid analytical performance specifications for combined analytical bias and imprecision for the use of common reference intervals. Ann Clin Biochem. 55:612–5. DOI: 10.1177/0004563217752963. PMID: 29310466.

73. Ricós C, Doménech MV, Perich C. 2004; Analytical quality specifications for common reference intervals. Clin Chem Lab Med. 42:858–62. DOI: 10.1515/CCLM.2004.140. PMID: 15327023.

74. Gowans EMS, Hyltoft Petersen P, Blaabjerg O, Hørder M. 1988; Analytical goals for the acceptance of common reference intervals for laboratories throughout a geographical area. Scand J Clin Lab Invest. 48:757–64. DOI: 10.3109/00365518809088757. PMID: 3238321.

75. Krishnamoorthy K. 2016. Handbook of statistical distributions with applications. 2nd ed. Chapman and Hall;New York: DOI: 10.1201/b19191.

76. Coskun A, Oosterhuis WP. 2020; Statistical distributions commonly used in measurement uncertainty in laboratory medicine. Biochem Med (Zagreb). 30:010101. DOI: 10.11613/BM.2020.010101. PMID: 32063728. PMCID: PMC6999182.

77. Petersen PH, Fraser CG, Jørgensen L, Brandslund I, Stahl M, Gowans EM, et al. 2002; Combination of analytical quality specifications based on biological within- and between-subject variation. Ann Clin Biochem. 39:543–50. DOI: 10.1177/000456320203900601. PMID: 12564835.

78. Thelen MHM, Jansen RTP, Weykamp CW, Steigstra H, Meijer R, Cobbaert CM. 2017; Expressing analytical performance from multi-sample evaluation in laboratory EQA. Clin Chem Lab Med. 55:1509–16. DOI: 10.1515/cclm-2016-0970. PMID: 28182577.

79. Stockl D, Reinauer H. 1993; Candidate reference methods for determining target values for cholesterol, creatinine, uric acid, and glucose in external quality assessment and internal accuracy control. I. Method setup. Clin Chem. 39:993–1000. DOI: 10.1093/clinchem/39.6.993. PMID: 8504568.

80. Uldall A, Blaabjerg O, Elfving S, Elg P, Gerhardt W, Holmberg H, et al. 1993; A programme for assigning target values for external quality assessment schemes in countries with no authorized reference laboratories. Annex. Experiences with deviating results on Ektachem 700 XR. Scand J Clin Lab Invest Suppl. 212:31–7. DOI: 10.3109/00365519309085452. PMID: 8465150.

81. Miller WG, Myers GL, Rej R. 2006; Why commutability matters. Clin Chem. 52:553–4. DOI: 10.1373/clinchem.2005.063511. PMID: 16595820.

82. Zegers I, Beetham R, Keller T, Sheldon J, Bullock D, MacKenzie F, et al. 2013; The importance of commutability of reference materials used as calibrators: The example of ceruloplasmin. Clin Chem. 59:1322–9. DOI: 10.1373/clinchem.2012.201954. PMID: 23649128.

83. Miller WG, Myers GL. 2013; Commutability still matters. Clin Chem. 59:1291–3. DOI: 10.1373/clinchem.2013.208785. PMID: 23780914.

84. CLSI. 2010. Characterization and qualification of commutable reference materials for laboratory medicine. 1st ed. CLSI EP30AE. Clinical and Laboratory Standards Institute;Wayne, PA:

85. CLSI. 2022. Evaluation of commutability of processed samples. 4th ed. CLSI EP14Ed4. Clinical and Laboratory Standards Institute;Wayne, PA:

86. Korzun WJ, Nilsson G, Bachmann LM, Myers GL, Sakurabayashi I, Nakajima K, et al. 2015; Difference in bias approach for commutability assessment: Application to frozen pools of human serum measured by 8 direct methods for HDL and LDL cholesterol. Clin Chem. 61:1107–13. DOI: 10.1373/clinchem.2015.240861. PMID: 26071490.

87. Budd JR, Weykamp C, Rej R, MacKenzie F, Ceriotti F, Greenberg N, et al. 2018; IFCC Working Group recommendations for assessing commutability Part 3: Using the calibration effectiveness of a reference material. Clin Chem. 64:465–74. DOI: 10.1373/clinchem.2017.277558. PMID: 29348164.

88. Nilsson G, Budd JR, Greenberg N, Delatour V, Rej R, Panteghini M, et al. 2018; IFCC Working Group recommendations for assessing commutability Part 2: Using the difference in bias between a reference material and clinical samples. Clin Chem. 64:455–64. DOI: 10.1373/clinchem.2017.277541. PMID: 29348165. PMCID: PMC5835923.

89. Miller WG, Schimmel H, Rej R, Greenberg N, Ceriotti F, Burns C, et al. 2018; IFCC Working Group recommendations for assessing commutability Part 1: General experimental design. Clin Chem. 64:447–54. DOI: 10.1373/clinchem.2017.277525. PMID: 29348163. PMCID: PMC5832613.

90. Fuentes-Arderiu X. 2006; Bio-metrological uncertainty in clinical laboratory sciences. EJIFCC. 17:6–7.

91. Farrance I, Frenkel R. 2012; Uncertainty of measurement: A review of the rules for calculating uncertainty components through functional relationships. Clin Biochem Rev. 33:49–75.

92. Coskun A, İnal BB, Serdar M. 2019; Measurement uncertainty in laboratory medicine: The bridge between medical and industrial metrology. Turk J Biochem. 44:121–5. DOI: 10.1515/tjb-2019-0170.

93. Lim YK, Kweon OJ, Lee MK, Kim B, Kim HR. 2020; Top-down and bottom-up approaches for the estimation of measurement uncertainty in coagulation assays. Clin Chem Lab Med. 58:1525–33. DOI: 10.1515/cclm-2020-0038. PMID: 32238603.

94. Martinello F, Snoj N, Skitek M, Jerin A. 2020; The top-down approach to measurement uncertainty: Which formula should we use in laboratory medicine? Biochem Med (Zagreb). 30:020101. DOI: 10.11613/BM.2020.020101. PMID: 32292278. PMCID: PMC7138004.

95. Burr T, Croft S, Favalli A, Krieger T, Weaver B. 2021; Bottom-up and top-down uncertainty quantification for measurements. Chemom Intell Lab Syst. 211:104224. DOI: 10.1016/j.chemolab.2020.104224.

96. Lee JH, Choi JH, Youn JS, Cha YJ, Song W, Park AJ. 2015; Comparison between bottom-up and top-down approaches in the estimation of measurement uncertainty. Clin Chem Lab Med. 53:1025–32. DOI: 10.1515/cclm-2014-0801. PMID: 25539513.

97. Magnusson B, Ossowicki H, Rienitz O, Theodorsson E. 2012; Routine internal- and external-quality control data in clinical laboratories for estimating measurement and diagnostic uncertainty using GUM principles. Scand J Clin Lab Invest. 72:212–20. DOI: 10.3109/00365513.2011.649015. PMID: 22233479.

98. Ceriotti F. 2018; Deriving proper measurement uncertainty from Internal Quality Control data: An impossible mission? Clin Biochem. 57:37–40. DOI: 10.1016/j.clinbiochem.2018.03.019. PMID: 29605551.

99. International Organization for Standardization. ISO 15189:2012(en), Medical laboratories - Requirements for quality and competence. https://www.iso.org/obp/ui/#iso:std:iso:15189:ed-3:v2:en. Updated on 2012.

100. International Organization for Standardization. ISO/TS 20914:2019(en), Medical laboratories - Practical guidance for the estimation of measurement uncertainty. https://www.iso.org/obp/ui/fr/#iso:std:iso:ts:20914:ed-1:v1:en. Updated on 2019.

101. Coskun A, Serteser M, Karpuzoglu HF, Unsal I. 2017; How can we evaluate differences between serial measurements on the same sample? A new approach based on within-subject biological variation. Clin Chem Lab Med. 55:e44–6. DOI: 10.1515/cclm-2016-0574. PMID: 27505091.

102. Pishro-Nik H. 2014. Introduction to probability, statistics, and random processes. Kappa Research, LLC;Amherst, MA: p. 732.

103. Coskun A, Serteser M, Unsal I. 2019; The short story of the long-term Sigma metric: Shift cannot be treated as a linear parameter. Clin Chem Lab Med. 57:E211–3. DOI: 10.1515/cclm-2018-1139. PMID: 30817295.

104. Harry M, Schroeder R. 2005. Six Sigma, the breakthrough management strategy revolutionizing the world's top corporations. Doubleday;New York:

105. Cao S, Qin X. 2018; Application of Sigma metrics in assessing the clinical performance of verified versus non-verified reagents for routine biochemical analytes. Biochem Med (Zagreb). 28:020709. DOI: 10.11613/BM.2018.020709. PMID: 30022884. PMCID: PMC6039166.

106. Keleş M. 2022; Evaluation of the clinical chemistry tests analytical performance with Sigma Metric by using different quality specifications - Comparison of analyser actual performance with manufacturer data. Biochem Med (Zagreb). 32:010703. DOI: 10.11613/BM.2022.010703. PMID: 34955671. PMCID: PMC8672391.

107. Coskun A, Oosterhuis WP, Serteser M, Unsal I. 2016; Sigma metric or defects per million opportunities (DPMO): The performance of clinical laboratories should be evaluated by the Sigma metrics at decimal level with DPMOs. Clin Chem Lab Med. 54:e217–9. DOI: 10.1515/cclm-2015-1219.

108. Coskun A, Ialongo C. 2020; Six Sigma revisited: We need evidence to include a 1.5 SD shift in the extraanalytical phase of the total testing process. Biochem Med (Zagreb). 30:010901. DOI: 10.11613/BM.2020.010901. PMID: 32063732. PMCID: PMC6999184.

109. Coskun A, Serteser M, Ünsal I. 2019; Sigma metric revisited: True known mistakes. Biochem Med (Zagreb). 29:010902. DOI: 10.11613/BM.2019.010902. PMID: 30591816. PMCID: PMC6294160.

110. Oosterhuis WP, Coskun A. 2018; Sigma metrics in laboratory medicine revisited: We are on the right road with the wrong map. Biochem Med (Zagreb). 28:020503. DOI: 10.11613/BM.2018.020503. PMID: 30022880. PMCID: PMC6039171.

111. Shaikh MS, Ali SA, Rashid A, Karim F, Moiz B. Performance evaluation of a coagulation laboratory using Sigma metrics. Int J Health Care Qual Assur. 2018; 31:600–8. DOI: 10.1108/IJHCQA-07-2017-0134. PMID: 29954266.

112. Teshome M, Worede A, Asmelash D. 2021; Total clinical chemistry laboratory errors and evaluation of the analytical quality control using Sigma metric for routine clinical chemistry tests. J Multidiscip Healthc. 14:125–36. DOI: 10.2147/JMDH.S286679. PMID: 33488088. PMCID: PMC7815085.

113. Ozdemir S, Ucar F. 2022; Determination of Sigma metric based on various TEa sources for CBC parameters: The need for Sigma metrics harmonization. J Lab Med. 46:133–41. DOI: 10.1515/labmed-2021-0116.

114. Kumar BV, Mohan T. 2018; Sigma metrics as a tool for evaluating the performance of internal quality control in a clinical chemistry laboratory. J Lab Physicians. 10:194. DOI: 10.4103/JLP.JLP_102_17. PMID: 29692587. PMCID: PMC5896188.

115. Coskun A, Serteser M, Kilercik M, Aksungar F, Unsal I. 2015; A new approach to calculating the Sigma Metric in clinical laboratories. Accredit Qual Assur. 20:147–52. DOI: 10.1007/s00769-015-1113-8.

116. Magnusson B, Ellison SLR. 2008; Treatment of uncorrected measurement bias in uncertainty estimation for chemical measurements. Anal Bioanal Chem. 390:201–13. DOI: 10.1007/s00216-007-1693-1. PMID: 18026721.

117. Ashwood ER, Bruns DE. Burtis CA, Ashwood ER, Bruns DE, editors. 2012. Clinical utility of laboratory tests. Tietz textbook of Clinical Chemistry and Molecular Diagnostics. 6th ed. Elsevier;Saint Louis: p. 49–60. DOI: 10.1016/B978-1-4160-6164-9.00003-2.

118. Chu K. 1999; An introduction to sensitivity, specificity, predictive values and likelihood ratios. Emerg Med. 11:175–81. DOI: 10.1046/j.1442-2026.1999.00041.x.

119. Trevethan R. 2017; Sensitivity, specificity, and predictive values: Foundations, pliabilities, and pitfalls in research and practice. Front Public Health. 5:307. DOI: 10.3389/fpubh.2017.00307. PMID: 29209603. PMCID: PMC5701930.

120. Shreffler J, Huecker MR. Diagnostic testing accuracy: Sensitivity, specificity, predictive values and likelihood ratios. StatPearls;https://www.ncbi.nlm.nih.gov/books/NBK557491. Updated on March 2023.

121. Leeflang MMG, Moons KGM, Reitsma JB, Zwinderman AH. 2008; Bias in sensitivity and specificity caused by data-driven selection of optimal cutoff values: Mechanisms, magnitude, and solutions. Clin Chem. 54:729–37. DOI: 10.1373/clinchem.2007.096032. PMID: 18258670.

122. Ringham BM, Alonzo TA, Grunwald GK, Glueck DH. 2010; Estimates of sensitivity and specificity can be biased when reporting the results of the second test in a screening trial conducted in series. BMC Med Res Methodol. 10:3. DOI: 10.1186/1471-2288-10-3. PMID: 20064254. PMCID: PMC2819240.