A smartwatch-based CPR feedback device improves chest compression quality among health care professionals and lay rescuers (2024)

Devices

Devices used in this study were the Study Device, a smartwatch-based compression feedback device (Relay Response BLS application; XiMiO Health, Inc, New York, NY), and a Comparator Device, a defibrillator-based compression feedback device (Zoll OneStep CPR Pads; ZOLL, Chelmsford, MA). The Relay Response BLS application is an investigational compression feedback aid that provides real-time compression feedback. The application uses the embedded mobile sensors from an Android consumer smartwatch (Ticwatch E3; Mobvoi, Kowloon, Hong Kong) to calculate and display compression depth and rate in near–real time. The smartwatch has an adjustable rubber wrist strap for secure fitting on all adult-sized wrists.

All compressions were performed on a standard Laerdal manikin (Laerdal Little Anne QCPR manikins; Laerdal Medical, Stavanger, Norway).

Study design

Three CPR modules were performed on the Laerdal manikin: module A: unaided chest compressions without compression feedback; module B: aided chest compressions with the smartwatch-based compression feedback aid; and module C: aided chest compressions with the defibrillator-based feedback aid. Figure1 illustrates the overall study design and module sequence. Before the initiation of the study and before the commencement of each module, each subject was given a 2-minute self-directed training period consisting of a written basic introduction to the study equipment including a description of the smartwatch-based and defibrillator-based feedback devices, and a description of the modules to be completed. Subjects were then asked to perform American Heart Association–compliant compressions (2.0–2.4 inches, 100–120/min). Subsequently, each subject read and completed a prestudy questionnaire, which consisted of general questions related to the user’s experience, training methods, familiarity with feedback devices, and perception of their proficiency. Each module consisted of a 2-minute compression cycle, followed by a 2-minute rest period, and then a second 2-minute compression cycle. Ten-minute rest periods were allowed between modules. Module A was performed first, and the order of the subsequent modules B and C was randomly assigned to avoid learning bias. Subjects in the lay rescuer group (untrained users) followed the same protocol, except they were not required to perform module C (as this device is not a known benchmark for them), and they were given basic scripted instructions on how to perform chest compressions before module A. At the end of the study, subjects in the IH and OH rescuer groups completed a poststudy questionnaire, which included questions evaluating the usability of both feedback aids, including the aids’ visual and audio feedback, comfort, and intent to use if available.

Outcome measures

Data collected from the second 2-minute compression cycle were used for analysis. To avoid including erroneous data not related to the performance of chest compressions, the first 3 seconds and the last 5 seconds of data from the study and comparator devices were discarded, since these data likely include hand movements during the positioning of each subject prior to initiating and finalizing compressions. The 3 main outcome measures analyzed in the present study were effectiveness, learnability, and usability. Effectiveness was calculated as the percentage of total compressions in which the rescuer keeps compressions within the suggested range for depth or rate (2.0–2.5 in and 100–120/min, respectively). The time to achieve learnability was defined as the time point where both depth and rate effectiveness are greater than or equal to 75% for the remainder of the compression session, considering there must be at least 10 seconds remaining in the compression session to achieve learnability. Usability was determined by comparing the data collected from prestudy and poststudy questionnaires, plotted using a Likertscale.

Statistical analysis

Numerical data are expressed as means ± standard error of the mean when normally distributed. Categorical data are presented as percentages. All comparisons between groups are performed by paired t test or ANOVA test. All tests are2-tailed. P values <.05 are considered statistically significant.

The protocol, informed consent form(s), recruitment materials, and all subject materials were approved by the BRANY Institutional Review Board and the SUNY Institutional Review Board.

Subjects

Table1 illustrates the characteristics of the study population. The IH rescuer and OH rescuer groups had an equally distributed male-to-female ratio; however, the lay rescuer group was composed only of female subjects. Subjects in the OH rescuer group performed CPR more frequently and more recently than the other groups; however, the overall time to their last certification was longer (2.3 years for OH rescuers vs 1.2 years for IH rescuers, P < .05).

Before initiating the study protocol and assessing baseline proficiency, IH rescuers and OH rescuers completed a previsit questionnaire, which showed that most perceived themselves as highly proficient in delivering high-quality chest compressions (85% either strongly agree or agree). Additionally, subjects in these 2 groups also believed their CPR certifications were highly useful and prepared them to compress effectively (90% either strongly agree or agree) and the training devices were user friendly (Figure2).

Despite perceptions of high proficiency, Table2 reveals that when performing chest compressions without a feedback device, baseline compression depth effectiveness was 40.2% ± 27.4%, 35.9% ± 32.3%, and 13.0% ± 12.3% (IH, OH, and lay rescuers, respectively), and rate effectiveness was 27.1% ± 43.9%, 57.1% ± 44.5%, and 6.2% ± 7.0% (IH, OH, and lay rescuers, respectively), significantly below recommended standards. When using the smartwatch-based feedback device, there was a significant improvement in compression depth effectiveness of 75.4% ± 25.6%, 75.1% ± 17.4%, and 54.6% ± 27.2% (IH, OH, and lay rescuers, respectively) and compression rate effectiveness of 94.5% ± 5.5%, 92.8% ± 9.5%, and 75.2% ± 33.5% (IH, OH, and lay rescuers, respectively) compared to baseline, P < .05 for both parameters. Whereas the improvement in depth was of similar magnitude between both feedback aids (P = ns), IH rescuers performed compressions at a significantly better rate with the smartwatch-based feedback aid, P < .05.

A population-based variability assessment is shown in Table3, which describes the chest compression averages (means and medians) and data distribution, reported as the coefficient of variation (CV) at baseline and using the smartwatch-based feedback aid for all groups.

At baseline, IH rescuers maintained unaided averages for depth within the range considered effective by current American Heart Association standards (mean 2.0 in), whereas OH rescuers performed compressions deeper than the desired range (mean 2.5 in). Both groups demonstrated a wide variability (CV 20.3% and 16.3%, respectively). Average unaided depths for lay rescuers were marginally below the acceptable range (mean 1.91 in) with a 41.1% variability. When the smartwatch-based feedback device was used, all depth averages were within the acceptable range (2.05 in), with a significantly narrower CV of 14.9%, implying that rescuers were more likely to stay within the desired range throughout the compression effort (P < .001). Similarly, when compression rate data were analyzed, compared to unaided compressions, utilization of the smartwatch-based feedback device produced averages within an acceptable range of 115 compressions per minute, with a significantly narrower CV of 6.3% for all subjects (P < .001).

Distributions of unaided and smartwatch-aided compression data for the whole cohort and each subgroup are graphically represented in Figure3. Depth and rate data were categorized in bins of 0.1 inches (depth) and 2 compressions per minute (rate), and the normalized probability for data occurring in each bin was calculated. Unaided baseline compression data were broadly distributed across a wide range of depths and rates, frequently demonstrating multimodal distribution patterns outside the desired ranges despite achieving average values within a range considered effective (see Table3). In contrast, when the smartwatch-based feedback device was used, a narrow range of compression depths and rates was maintained by all groups, with the highest probabilities occurring within the American Heart Association–recommended ranges for depth and rate.

The ability of a naïve rescuer to learn how to compress effectively with a smartwatch-based device, despite no prior training on their use, is illustrated in Figure4. Figure4A shows that, on average, only 4 of 30 subjects reached effective depth when compressing without a feedback device, which took 39.3 seconds to achieve. However, when rescuers used the smartwatch-based feedback device, a significant improvement was seen (P < .05) where 19 of 30 subjects compressed at a >75% effective depth in 8.5 seconds (defined as high learnability). Furthermore, no difference in learnability was noted among professional rescuers when the smartwatch-based device was used compared to the defibrillator-based device. Figure4B shows that IH and OH rescuers reached ideal effectiveness quickly with both feedback compression devices (P = ns; 2.6 seconds for the smartwatch-based device and 4.8 seconds for the defibrillator-based device).

Upon completion of the study protocol, subjects completed a poststudy questionnaire regarding the usability of the feedback devices compared in this study. Figure5 illustrates the Likert scale–based results for the smartwatch-based feedback device (Figure5A) and the defibrillator-based feedback device (Figure5B). Overall, subjects perceived that the smartwatch-based device assisted them in performing compressions more effectively than the defibrillator-based device (70% vs 35% strongly agree, smartwatch-based device compared to defibrillator-based device, P = .03). They also found smartwatches to be more comfortable (70% vs 30% strongly agree, smartwatch-based device compared to defibrillator-based device, P = .02), to have clearer visual cues (73% vs 30% strongly agree, smartwatch-based device compared to defibrillator-based device, P = .02), and to be more convenient to use (67% vs 40% strongly agree, smartwatch-based device compared to defibrillator-based device, P = .03) compared to the defibrillator-based device. In addition, rescuers indicated they would use smartwatch-based feedback more frequently during training (P < .05) and emergencies (P < .05) if available.

Effectiveness

Delivering high-quality chest compressions is difficult even for trained professionals, who frequently perform below the recommended standards.4^,5^,11^,21 In line with these findings, our study’s initial observation underscores a critical issue: even among professional rescuers with prior certification and self-reported perception of high proficiency, baseline (unaided) compression performance fell short of the desired effectiveness standards, a clear example of unconscious incompetence.11^,22^,23 Furthermore, it is well documented that skills can deteriorate over time, and in our study, the gap of over 1 year since their last certification may have influenced the suboptimal baseline performance of the rescuers.6^,24 This baseline assessment is particularly significant, as it illustrates the pressing need to develop highly usable and effective CPR feedback devices, given that CPR certifications typically occur on a 2-year cycle. This study demonstrated significant improvements in compression depth and rate effectiveness when using the smartwatch-based CPR device compared to unaided compressions. The improvement was consistent across the different groups of rescuers, including professional and lay rescuers, and was similar to the improvement seen when defibrillator-based feedback devices were used, demonstrating noninferiority between the 2 devices. This suggests that smartwatch-based CPR devices can play a crucial role in helping rescuers achieve and maintain recommended compression depths and rates during CPR.

Variability

Outcome measures generally used to evaluate chest compression performance rely on the average depth or rate achieved by rescuers during resuscitation.5 However, as exemplified in this study, this outcome may be misleading. While mean values may provide insights into overall performance, they often fall short of capturing the dynamics of chest compressions. Evaluation of the variability in the depth and rate of chest compressions during CPR may be a more accurate measure to reflect the quality of resuscitation efforts. This study comprehensively assessed the variability in compression depths and rates among all rescuers. Standard deviation (SD) and CV were calculated to quantify the dispersion and relative variability of compression data. Our analysis revealed that compression data displayed substantial variability at baseline, as evidenced by high SD values and CV percentages. This variability manifested as broad data distributions, with compressions occurring across a wide range of depths and rates. Notably, despite achieving average depth and rate values close to the desired range for effective CPR, the unaided compressions often exhibited multimodal distribution patterns that extended outside these boundaries. In contrast, when the smartwatch-based feedback device was used, compression data exhibited a remarkable reduction in variability. SD values decreased significantly, indicating tighter data clustering around the mean depth and rate values. The CV percentages also demonstrated a noteworthy decrease, signifying a more consistent performance among all rescuer groups.

Finding reduced variability in compression data suggests that the smartwatch-based CPR feedback device effectively guides and assists rescuers in achieving a more consistent and precise performance of chest compressions. Furthermore, variability analysis was performed on pooled data from each group of rescuers, demonstrating that the smartwatch-based feedback device could render a population consistent in performing highly effective chest compressions, akin to a mechanical compression device. This consistency is paramount for delivering high-quality resuscitation efforts during cardiac arrest and may contribute to improved patient outcomes.1^,5^,6 Moreover, the reduction in variability has implications for training and the potential standardization of CPR practices. As such, future studies or guidance statements may consider this performance metric as an additional outcome to evaluate rescuer performances. A CPR aid that promotes consistency in compression depths and rates may facilitate more effective training programs and ensure a higher degree of adherence to recommended practices. Further research in this area may yield insights into the long-term effects of reduced variability on patient outcomes and inform future resuscitation guidelines.

Usability

The effective adoption of new medical technologies hinges on factors such as familiarity, ease of use (usability), and the speed at which users can learn to operate the technology (learnability). In this context, rescuers may not always be familiar with different automated external defibrillator–based aids, which they might encounter for the first time at a cardiac arrest scene, compared to potentially greater familiarity with mobile technology.13^,14 We therefore systematically assessed the usability of the smartwatch-based CPR aid, comparing it with a defibrillator-based compression feedback aid. Our analysis relied on prestudy and poststudy questionnaires using a Likert scale. Our findings indicate that rescuers quickly adapted to using smartwatch-based aids. In a relatively short time, many subjects achieved effective compressions, highlighting the user-friendliness and training efficacy of the smartwatch-based CPR aid. Furthermore, the poststudy questionnaire revealed that professional rescuers found the smartwatch-based aid to have better visual cues and be more comfortable and convenient than the defibrillator-based aid. Additionally, all subjects expressed a strong willingness to use the smartwatch-based CPR aid more frequently, both during training and in emergency situations, if it were available for clinical use. This significant improvement in the usability of smartwatch-based aids might therefore positively impact rescuers’ proficiency and encourage more frequent use and CPR training.1

Learnability

The study also assessed the learnability of the smartwatch-based CPR aid. The study’s approach to defining a learnability threshold, where both depth and rate effectiveness reach or exceed 75%, provides valuable insights into how quickly rescuers can adapt to these CPR feedback devices. The results indicate that the smartwatch-based feedback aid significantly improved learnability, with most subjects achieving high effectiveness in less than 10 seconds, highlighting how rescuers quickly adapted to its use. More importantly, these results were achieved with minimal instruction and training time before performing the study protocol, highlighting that intuitive application designs and user interfaces may be instrumental in discovering new paths to proficiency. This positive outlook underscores the fact that smartwatch-based feedback devices enhance usability, promote user acceptance, and motivate users to engage with the technology, emphasizing their practical viability. Furthermore, considering the crucial importance of prompt and efficient resuscitation efforts, these findings suggest a potential impact on successful outcomes when using mobile technology that can be swiftly activated at the cardiac arrest scene.

Accessibility (time to first compression)

The utilization of mobile devices as CPR feedback devices, particularly through a smartwatch app accessible on both Android and Apple watches, offers unparalleled advantages in the realm of emergency response. With over 45% of Americans already equipped with some form of smartwatch, the widespread adoption of this technology harnesses a vast network of potential lifesavers.25 The smartwatch application tested in this study has compatibility with widely used operating systems, ensuring that this CPR feedback device can be readily delivered to a massive user base, making it an inclusive and accessible tool for both professional rescuers and laypersons. Furthermore, the prevalence of existing alert networks for cardiac arrests, coupled with the instantaneous nature of smartwatch notifications, enhances the potential for swift mobilization of nearby individuals equipped with the technology, significantly reducing response times. The integration of mobile devices into the cardiac arrest response chain not only capitalizes on the ubiquity of smartwatches but also leverages existing digital infrastructure to create a connected and responsive community of potential first responders.

Some limitations need to be highlighted. Importantly, the present study is a simulation-based study exploring the potential impact on chest compression effectiveness; therefore, how well the improvements seen in this study translate to real clinical scenarios remains to be studied. In addition, all compressions were performed on manikins placed on hard surfaces; thus, the impact of compressing on softer surfaces, such as on mattresses, cannot be accounted for.26 The present study did not include a calculation of recoil, a clinically important factor contributing to the quality of compressions. Furthermore, the subjects in the present study were grouped based on their previous CPR training and profession but were not adequately matched regarding their weight and physical fitness, factors known to affect performance.27 This was most evident in the lay rescuer group, which potentially underperformed compared to the other groups, since this group was composed of only female subjects.27^,28

It is important to note that participant recruitment for this study was conducted on a first-come, first-served basis without regard for sex balance; as a consequence, the control group was composed solely of female participants, while both sexes were represented in the IH and OH rescuer groups. Whereas efforts were made to maintain the validity of the findings, the lack of sex balance may affect the generalizability of the results in the control group, most likely underestimating the effects of the smartwatch-based feedback device.

The authors thank Dr Stan Kachnowski, Chair of Research & Evaluation at HITLAB, and Dr Jason Lazar for their professional support in executing this study at SUNY Downstate.

^AppendixSupplementary data associated with this article can be found in the online version at https://doi.org/10.1016/j.cvdhj.2024.03.006.

Supplementary Data:

1. Virani S.S., Alonso A., Benjamin E.J., et al. Heart disease and stroke statistics-2020 update: a report from the American Heart Association. Circulation. 2020;141:e139–e596. [PubMed] [Google Scholar]

2. Tsao C.W., Aday A.W., Almarzooq Z.I., et al. Heart disease and stroke statistics-2023 update: a report from the American Heart Association. Circulation. 2023;147:e93–e621. [PubMed] [Google Scholar]

3. Girotra S., Nallamothu B.K., Spertus J.A., Li Y., Krumholz H.M., Chan P.S. Trends in survival after in-hospital cardiac arrest. N Engl J Med. 2012;367:1912–1920. [PMC free article] [PubMed] [Google Scholar]

4. Hostler D., Everson-Stewart S., Rea T.D., et al. Effect of real-time feedback during cardiopulmonary resuscitation outside hospital: prospective, cluster-randomised trial. BMJ. 2011;342 [PMC free article] [PubMed] [Google Scholar]

5. Pepe P.E., Duval S., Aufderheide T.P., et al. Optimal combination of compression rate and depth during cardiopulmonary resuscitation for functionally favorable survival. JAMA Cardiol. 2019;4:900–908. [PMC free article] [PubMed] [Google Scholar]

6. Yannopoulos D., Aufderheide T.P., Abella B.S., et al. Quality of CPR: an important effect modifier in cardiac arrest clinical outcomes and intervention effectiveness trials. Resuscitation. 2015;94:106–113. [PubMed] [Google Scholar]

7. Idris A.H., Guffey D., Aufderheide T.P., et al. Relationship between chest compression rates and outcomes from cardiac arrest. Circulation. 2012;125:3004–3012. [PMC free article] [PubMed] [Google Scholar]

8. Vadeboncoeur T., Stolz U., Panchal A., et al. Chest compression depth and survival in out-of-hospital cardiac arrest. Resuscitation. 2014;85:182–188. [PubMed] [Google Scholar]

9. Abella B.S., Alvarado J.P., Myklebust H., et al. Quality of cardiopulmonary resuscitation during in-hospital cardiac arrest. JAMA. 2005;293:305–310. [PubMed] [Google Scholar]

10. Hellevuo H., Sainio M., Nevalainen R., et al. Deeper chest compression - more complications for cardiac arrest patients? Resuscitation. 2013;84:760–765. [PubMed] [Google Scholar]

11. Cheng A., Brown L.L., Duff J.P., et al. Improving cardiopulmonary resuscitation with a CPR feedback device and refresher simulations (CPR CARES Study): a randomized clinical trial. JAMA Pediatr. 2015;169:137–144. [PubMed] [Google Scholar]

12. Lee D.S., Min M.K., Ryu J.H., et al. Cardiopulmonary resuscitation: difficulty in maintaining sufficient compression depth at the appropriate rate. Signa Vitae. 2023;19:79–85. [Google Scholar]

13. Sood N., Sangari A., Goyal A., et al. Do cardiopulmonary resuscitation real-time audiovisual feedback devices improve patient outcomes? A systematic review and meta-analysis. World J Cardiol. 2023;15:531–541. [PMC free article] [PubMed] [Google Scholar]

14. Obling L., Hassager C., Blomberg S.N., Folke F. Inverse association between bystander use of audiovisual feedback from an automated external defibrillator and return of spontaneous circulation. J Am Heart Assoc. 2022;11 [PMC free article] [PubMed] [Google Scholar]

15. Bonnes J.L., Brouwer M.A., Navarese E.P., et al. Manual cardiopulmonary resuscitation versus CPR including a mechanical chest compression device in out-of-hospital cardiac arrest: a comprehensive meta-analysis from randomized and observational studies. Ann Emerg Med. 2016;67:349–360.e343. [PubMed] [Google Scholar]

16. Wang P.L., Brooks S.C. Mechanical versus manual chest compressions for cardiac arrest. Cochrane Database Syst Rev. 2018;8:Cd007260. [PMC free article] [PubMed] [Google Scholar]

17. An M., Kim Y., Cho W.-K. Effect of smart devices on the quality of CPR training: a systematic review. Resuscitation. 2019;144:145–156. [PubMed] [Google Scholar]

18. Ahn C., Lee J., Oh J., et al. Effectiveness of feedback with a smartwatch for high-quality chest compressions during adult cardiac arrest: a randomized controlled simulation study. PLoS One. 2017;12 [PMC free article] [PubMed] [Google Scholar]

19. Lee J., Song Y., Oh J., et al. Smartwatch feedback device for high-quality chest compressions by a single rescuer during infant cardiac arrest: a randomized, controlled simulation study. Eur J Emerg Med. 2019;26:266–271. [PMC free article] [PubMed] [Google Scholar]

20. Grünerbl A., Pirkl G., Monger E., Gobbi M., Lukowicz P. 2015. Smart-watch life saver: smart-watch interactive-feedback system for improving bystander CPR. In Proceedings of the 2015 ACM International Symposium on Wearable Computers, Osaka, Japan, 7–11 September 2015; pp. 19–26. [Google Scholar]

21. Abella B.S., Sandbo N., Vassilatos P., et al. Chest compression rates during cardiopulmonary resuscitation are suboptimal: a prospective study during in-hospital cardiac arrest. Circulation. 2005;111:428–434. [PubMed] [Google Scholar]

22. Houldsworth A. Conscious competence in interprofessional learning in healthcare education. MedEdPublish. 2018;7 [version 1] [Google Scholar]

23. Cheng A., Overly F., Kessler D., et al. Perception of CPR quality: influence of CPR feedback, Just-in-Time CPR training and provider role. Resuscitation. 2015;87:44–50. [PubMed] [Google Scholar]

24. Almeida D., Clark C., Jones M., McConnell P., Williams J. Consistency and variability in human performance during simulate infant CPR: a reliability study. Scand J Trauma Resusc Emerg Med. 2020;28:91. [PMC free article] [PubMed] [Google Scholar]

25. Friend S.H., Ginsburg G.S., Picard R.W. Wearable digital health technology. N Engl J Med. 2023;389:2100–2101. [PubMed] [Google Scholar]

26. Holt J., Ward A., Mohamed T.-Y., et al. The optimal surface for delivery of CPR: a systematic review and meta-analysis. Resuscitation. 2020;155:159–164. [PubMed] [Google Scholar]

27. Kessler D.O., Lemke D.S., Jani P., et al. Caregiver characteristics associated with quality of cardiac compressions on an adult mannequin with real-time visual feedback: a simulation-based multicenter study. Simul Healthc. 2020;15:82–88. [PubMed] [Google Scholar]

28. Peberdy M.A., Silver A., Ornato J.P. Effect of caregiver gender, age, and feedback prompts on chest compression rate and depth. Resuscitation. 2009;80:1169–1174. [PubMed] [Google Scholar]