Ann Rehabil Med.  2023 Aug;47(4):261-271. 10.5535/arm.23019.

Validation of Wearable Digital Devices for Heart Rate Measurement During Exercise Test in Patients With Coronary Artery Disease

• Kim C
• Song JH
• Kim SH

• KMID: 2545322
• DOI: http://doi.org/10.5535/arm.23019

Abstract

Objective
To assess the accuracy of recently commercialized wearable devices in heart rate (HR) measurement during cardiopulmonary exercise test (CPX) under gradual increase in exercise intensity, while wearable devices with HR monitors are reported to be less accurate in different exercise intensities.
Methods
CPX was performed for patients with coronary artery disease (CAD). Twelve lead electrocardiograph (ECG) was the gold standard and Apple watch 7 (AW7), Galaxy watch 4 (GW4) and Bio Patch Mobicare 200 (MC200) were applied for comparison. Paired absolute difference (PAD), mean absolute percentage error (MAPE) and intraclass correlation coefficient (ICC) were evaluated for each device.
Results
Forty-four participants with CAD were included. All the devices showed MAPE under 2% and ICC above 0.9 in rest, exercise and recovery phases (MC200=0.999, GW4=0.997, AW7=0.998). When comparing exercise and recovery phase, PAD of MC200 and AW7 in recovery phase were significantly bigger than PAD of exercise phase (p<0.05). Although not significant, PAD of GW4 tended to be bigger in recovery phase, too. Also, when stratified by HR 20, ICC of all the devices were highest under HR of 100, and ICC decreased as HR increased. However, except for ICC of GW4 at HR above 160 (=0.867), all ICCs exceeded 0.9 indicating excellent accuracy.
Conclusion
The HR measurement of the devices validated in this study shows a high concordance with the ECG device, so CAD patients may benefit from the devices during high-intensity exercise under conditions where HR is measured reliably.

Keyword

Wearable electronic devices; Cardiac rehabilitation; Heart rate; Exercise test

Figure

• 

  Fig. 1. Heart rate (HR) measurement during cardiopulmonary exercise test. HR was recorded at 2, 4, and 6 minutes in rest phase. HR was recorded every minute beginning from 1 minute during exercise phase until termination of the test. HR at termination, which was the beginning of recovery phase, was also recorded and once every minute subsequently in recovery phase. The time points for HR measurement are in bold and underlined. Also, these time points are marked with heart shape (♥), respectively.

• 

  Fig. 2. Bland–Altman plots show agreement between 12 lead electrocardiograph, the gold standard and each wearable device. Solid horizontal lines indicate average heart rate (HR) differences in each device. Dashed lines indicate 95% confidence limits of agreement for each device. SD, standard deviation.

• 

  Fig. 3. Scatter plots show correlation between 12 lead electrocardiograph (ECG) vs. each wearable device. Diagonal lines are lines of identity. Each dot is a separate measurement of heart rate. ICC, intraclass correlation coefficient.

• 

Reference

1. Thomas RJ, Beatty AL, Beckie TM, Brewer LC, Brown TM, Forman DE, et al. Home-based cardiac rehabilitation: a scientific statement from the American Association of Cardiovascular and Pulmonary Rehabilitation, the American Heart Association, and the American College of Cardiology. Circulation. 2019; 140:e69–89.

2. Kim C, Sung J, Han JY, Jee S, Lee JW, Lee JH, et al. Current status of cardiac rehabilitation in the Regional Cardiocerebrovascular Centers in Korea. J Clin Med. 2021; 10:5079.

3. Buckingham SA, Taylor RS, Jolly K, Zawada A, Dean SG, Cowie A, et al. Home-based versus centre-based cardiac rehabilitation: abridged Cochrane systematic review and meta-analysis. Open Heart. 2016; 3:e000463.

4. Antoniou V, Davos CH, Kapreli E, Batalik L, Panagiotakos DB, Pepera G. Effectiveness of home-based cardiac rehabilitation, using wearable sensors, as a multicomponent, cutting-edge intervention: a systematic review and meta-analysis. J Clin Med. 2022; 11:3772.

5. Anderson L, Sharp GA, Norton RJ, Dalal H, Dean SG, Jolly K, et al. Home-based versus centre-based cardiac rehabilitation. Cochrane Database Syst Rev. 2017; 6:CD007130.

6. Kim C, Kim SH, Suh MR. Accuracy and validity of commercial smart bands for heart rate measurements during cardiopulmonary exercise test. Ann Rehabil Med 2022;46:209-18. Erratum in: Ann Rehabil Med. 2023; 47:78.

7. Bidargaddi NP, Sarela A. Activity and heart rate-based measures for outpatient cardiac rehabilitation. Methods Inf Med. 2008; 47:208–16.

8. Gillinov S, Etiwy M, Wang R, Blackburn G, Phelan D, Gillinov AM, et al. Variable accuracy of wearable heart rate monitors during aerobic exercise. Med Sci Sports Exerc. 2017; 49:1697–703.

9. Boudreaux BD, Hebert EP, Hollander DB, Williams BM, Cormier CL, Naquin MR, et al. Validity of wearable activity monitors during cycling and resistance exercise. Med Sci Sports Exerc. 2018; 50:624–33.

10. Dooley EE, Golaszewski NM, Bartholomew JB. Estimating accuracy at exercise intensities: a comparative study of self-monitoring heart rate and physical activity wearable devices. JMIR Mhealth Uhealth. 2017; 5:e34.

11. Etiwy M, Akhrass Z, Gillinov L, Alashi A, Wang R, Blackburn G, et al. Accuracy of wearable heart rate monitors in cardiac rehabilitation. Cardiovasc Diagn Ther 2019;9:262-71. Erratum in: Cardiovasc Diagn Ther. 2020; 10:644–5.

12. Martinato M, Lorenzoni G, Zanchi T, Bergamin A, Buratin A, Azzolina D, et al. Usability and accuracy of a smartwatch for the assessment of physical activity in the elderly population: observational study. JMIR Mhealth Uhealth. 2021; 9:e20966.

13. Prieto-Avalos G, Cruz-Ramos NA, Alor-Hernández G, Sánchez-Cervantes JL, Rodríguez-Mazahua L, Guarneros-Nolasco LR. Wearable devices for physical monitoring of heart: a review. Biosensors (Basel). 2022; 12:292.

14. Arifin WN. A web-based sample size calculator for reliability studies. Educ Med J. 2018; 10:67–76.

15. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016;15:155-63. Erratum in: J Chiropr Med. 2017; 16:346.

16. Fletcher GF, Ades PA, Kligfield P, Arena R, Balady GJ, Bittner VA, American Heart Association Exercise, Cardiac Rehabilitation; Prevention Committee of the Council on Clinical Cardiology, Council on Nutrition, Physical Activity and Metabolism, Council on Cardiovascular; Stroke Nursing, and Council on Epidemiology and Prevention, et al. Exercise standards for testing and training: a scientific statement from the American Heart Association. Circulation. 2013; 128:873–934.

17. Fokkema T, Kooiman TJ, Krijnen WP, VAN DER Schans CP, DE Groot M. Reliability and validity of ten consumer activity trackers depend on walking speed. Med Sci Sports Exerc. 2017; 49:793–800.

18. Nelson BW, Allen NB. Accuracy of consumer wearable heart rate measurement during an ecologically valid 24-hour period: intraindividual validation study. JMIR Mhealth Uhealth. 2019; 7:e10828.

19. Kwon S, Lee SR, Choi EK, Ahn HJ, Song HS, Lee YS, et al. Validation of adhesive single-lead ECG device compared with Holter monitoring among non-atrial fibrillation patients. Sensors (Basel). 2021; 21:3122.

20. Ghaffari S, Kazemi B, Aliakbarzadeh P. Abnormal heart rate recovery after exercise predicts coronary artery disease severity. Cardiol J. 2011; 18:47–54.

21. Johnson NP, Goldberger JJ. Prognostic value of late heart rate recovery after treadmill exercise. Am J Cardiol 2012;110:45-9. Erratum in: Am J Cardiol. 2012; 110:763.

22. Gayda M, Bourassa MG, Tardif JC, Fortier A, Juneau M, Nigam A. Heart rate recovery after exercise and long-term prognosis in patients with coronary artery disease. Can J Cardiol. 2012; 28:201–7.

23. Navalta JW, Montes J, Bodell NG, Salatto RW, Manning JW, DeBeliso M. Concurrent heart rate validity of wearable technology devices during trail running. PLoS One. 2020; 15:e0238569.

24. Alzahrani A, Hu S, Azorin-Peris V, Barrett L, Esliger D, Hayes M, et al. A multi-channel opto-electronic sensor to accurately monitor heart rate against motion artefact during exercise. Sensors (Basel). 2015; 15:25681–702.

25. El-Amrawy F, Nounou MI. Are currently available wearable devices for activity tracking and heart rate monitoring accurate, precise, and medically beneficial? Healthc Inform Res. 2015; 21:315–20.

26. Laukkanen RM, Virtanen PK. Heart rate monitors: state of the art. J Sports Sci. 1998; 16 Suppl:S3–7.

27. Léger L, Thivierge M. Heart rate monitors: validity, stability, and functionality. Phys Sportsmed. 1988; 16:143–51.

28. Patel MS, Asch DA, Volpp KG. Wearable devices as facilitators, not drivers, of health behavior change. JAMA. 2015; 313:459–60.

29. Bent B, Goldstein BA, Kibbe WA, Dunn JP. Investigating sources of inaccuracy in wearable optical heart rate sensors. NPJ Digit Med. 2020; 3:18.

30. Sarhaddi F, Kazemi K, Azimi I, Cao R, Niela-Vilén H, Axelin A, et al. A comprehensive accuracy assessment of Samsung smartwatch heart rate and heart rate variability. PLoS One. 2022; 17:e0268361.

31. Nissen M, Slim S, Jäger K, Flaucher M, Huebner H, Danzberger N, et al. Heart rate measurement accuracy of Fitbit Charge 4 and Samsung Galaxy Watch Active2: device evaluation study. JMIR Form Res. 2022; 6:e33635.

32. Brophy E, Muehlhausen W, Smeaton AF, Ward TE. Optimised convolutional neural networks for heart rate estimation and human activity recognition in wrist worn sensing applications. arXiv [Online]. 2020; [cited 2023 Apr 28]. Available from: https://doi.org/10.48550/arXiv.2004.00505.

33. Kang JH, Cho BH, Lee JS, Chee YJ, Kim IY, Kim SI. A study on accelerometer based motion artifact reduction in photoplethysmography signal. J Biomed Eng Res. 2007; 28:369–76.

34. Kappert K, Böhm M, Schmieder R, Schumacher H, Teo K, Yusuf S, ONTARGET/TRANSCEND Investigators, et al. Impact of sex on cardiovascular outcome in patients at high cardiovascular risk: analysis of the Telmisartan Randomized Assessment Study in ACE-Intolerant Subjects With Cardiovascular Disease (TRANSCEND) and the Ongoing Telmisartan Alone and in Combination With Ramipril Global End Point Trial (ONTARGET). Circulation. 2012; 126:934–41.

35. Bots SH, Peters SAE, Woodward M. Sex differences in coronary heart disease and stroke mortality: a global assessment of the effect of ageing between 1980 and 2010. BMJ Glob Health. 2017; 2:e000298.

36. Blok S, Piek MA, Tulevski II, Somsen GA, Winter MM. The accuracy of heartbeat detection using photoplethysmography technology in cardiac patients. J Electrocardiol. 2021; 67:148–57.