Hostname: page-component-745bb68f8f-b95js Total loading time: 0 Render date: 2025-02-11T08:13:13.782Z Has data issue: false hasContentIssue false

Development of a Performance Assessment Scale for Simulated Dispatcher-Assisted Cardiopulmonary Resuscitation (Telephone-CPR): A Multi-Center Randomized Simulation-Based Clinical Trial

Published online by Cambridge University Press:  23 July 2021

Daniel Aiham Ghazali*
Affiliation:
Department of Emergency Medicine and EMS, University Hospital Center, Bichat, Paris, France Simulation center Ilumens, University of Paris, Paris, France
Caroline Delaire
Affiliation:
Department of EMS, University Hospital Center, Poitiers, France
Emmanuel Blottiaux
Affiliation:
Department of Emergency Medicine, University Hospital of Rennes, Rennes, France Paris Fire Brigade Emergency Department, Pairs, France
Jean-Yves Lardeur
Affiliation:
Department of Emergency Medicine, Hospital of Chatellerault, Chatellerault, France
Daniel Jost
Affiliation:
Paris Fire Brigade Emergency Department, Pairs, France
Mathieu Violeau
Affiliation:
Department of Emergency Medicine, Hospital of Niort, Niort, France
Cyril Breque
Affiliation:
Simulation Center, Faculty of Medicine of Poitiers, Poitiers, France
Denis Oriot
Affiliation:
Simulation Center, Faculty of Medicine of Poitiers, Poitiers, France Department of Pediatric Emergency Medicine, University Hospital Center, Poitiers, France
*
Correspondence: Daniel Aiham Ghazali, MD, PhD Emergency Department and Emergency Medical Service University Hospital of Bichat 46 rue Henri Huchard, 75018 Paris, France E-mail: aiham@hotmail.com
Rights & Permissions [Opens in a new window]

Abstract

Introduction:

Dispatchers should be trained to interrogate bystanders with strict protocols to elicit information focused on recognizing cardiac arrest and should provide telephone cardiopulmonary resuscitation (CPR) instructions in all cases of suspected cardiac arrest. While an objective assessment of training outcomes is needed, there is no performance assessment scale for simulated dispatcher-assisted CPR.

Study Objective:

The aim of the study was to create a valid and reliable performance assessment scale for simulated dispatcher-assisted CPR.

Methods:

In this prospective, randomized, controlled, multi-centric simulation-based trial (registration number TCTR20210130002), the scale was developed according to the European Resuscitation Council (ERC) and American Heart Association (AHA) Guidelines 2015 and revised by experts. The performance of 48 dispatchers’ telephone-CPR and of 48 bystanders carrying out CPR on a manikin was assessed by two independent evaluators using the scale and using a SkillReporter (PC) software to provide CPR objective performance. Continuous variables were described as mean (SD) and categorical variables as numbers and percentage (%). Comparative analysis between two groups used a Student t-test or a non-parametric test of Mann-Whitney. The internal structure of the scale was evaluated, including internal consistency using α Cronbach coefficient, and reproducibility using intraclass correlation coefficient (ICC) and linear correlation coefficient (R2) calculation.

Results:

The scale included three different parts: two sections for dispatchers’ (32 items) and bystanders’ CPR performance (15 items) assessment, and a third part recording times. There was excellent internal consistency (α Cronbach coefficient = 0.77) and reproducibility (ICC = 0.93; R² = 0.86). For dispatchers’ performance assessment, α Cronbach coefficient = 0.76; ICC = 0.91; R2 = 0.84. For bystanders’ performance assessment, α Cronbach coefficient = 0.75; ICC = 0.93; R2 = 0.87. Reproducibility was excellent for nine items, good for 19 items, and moderate for 19 items. No item had poor reproducibility. There was no significant difference between dispatch doctors’ and medical dispatch assistants’ performances (33.0 [SD = 4.7] versus 32.3 [SD = 3.2] out of 52, respectively; P = .70) or between trained and untrained bystanders to follow the instructions (14.3 [SD = 2.0] versus 13.9 [SD = 1.8], respectively; P = .64). Objective performance (%) was significantly higher for trained bystanders than for untrained bystanders (67.4 [SD = 14.5] versus 50.6 [SD = 19.3], respectively; P = .03).

Conclusion:

The scale was valid and reliable to assess performance for simulated dispatcher-assisted CPR. To the authors’ knowledge, no other valid performance tool currently exists. It could be used in simulated telephone-CPR training programs to improve performance.

Type
Original Research
Copyright
© The Author(s), 2021. Published by Cambridge University Press on behalf of the World Association for Disaster and Emergency Medicine

Introduction

Out-of-hospital cardiac arrest (OHCA) requires initiation of cardiopulmonary resuscitation (CPR) and use of automated external defibrillator (AED) before arrival of Emergency Medical Services. Quality of CPR and AED use impacts the probability of survival after OHCA.Reference Sasson, Rogers and Dahl1,Reference Baekgaard, Viereck and Moller2 The European Resuscitation Council (ERC; Niel, Belgium) and American Heart Association (AHA; Dallas, Texas USA) Guidelines 2015 highlight the critical importance of the interactions between the emergency medical dispatcher and the bystander.3,4 The latter performs CPR following the Basic Life Support (BLS) sequence.Reference Perkinsa, Handley and Koster5 The guidelines say that dispatchers should be trained to interrogate bystanders with strict protocols to elicit information focused on early recognition of cardiac arrest and should provide telephone-CPR instructions in all cases of suspected OHCA.

The use of such protocols can effectively improve OHCA recognition by dispatcherReference Travers, Jost and Gillard6,Reference Stipulante, Tubes and El Fassi7 and provision of telephone-CPR.Reference Castren, Kuisma and Serlachius8,Reference Dami, Fuchs and Praz9 A recent study highlighted the positive effects of the implementation of the assisted telephone-guided CPR program on CPR outcomes.Reference Seyed Bagheri, Sadeghi and Kazemi10 Dispatcher telephone-CPR instructions have been shown to improve bystander CPR rate,Reference Bohm, Vaillancourt and Charette11 reduce the time to start CPR,Reference Stipulante, Tubes and El Fassi7,Reference Tanaka, Taniguchi and Wato12,Reference Rea, Eisenberg and Culley13 increase the number of chest compressions delivered,Reference Ghuysen, Collas and Stipulante14 and improve patient outcomes following OHCA in all patient groups.Reference Stipulante, Tubes and El Fassi7,Reference Bohm, Vaillancourt and Charette11

While an objective assessment of training outcomes is needed, to the best of the authors’ knowledge, there is no performance assessment scale for simulated dispatcher-assisted CPR. Study objective was to create a valid and reliable performance assessment scale for simulated dispatcher-assisted CPR.

Methods

Study Design

This was a prospective, randomized, controlled, multi-centric trial conducted in the simulation center of the University of Paris, the University of Poitiers, and the hospital of Niort (France) from September 16, 2019 through February 14, 2020. The study protocol was approved by the local committee on ethics, research, and informatics (IRB number SL-DG-2020-3.2). The study protocol was registered in Thai Clinical Thai Registry under the number TCTR20210130002. All participants signed written informed consent to the research. All results were kept anonymous.

Creation of the Instrument

The framework introduced by Downing describing the various steps of a validation process for performance assessment was used.Reference Downing15

Content

The different items of the scale were chosen according to the ERC and AHA Guidelines 20153,4 and the literature.Reference Travers, Jost and Gillard6,Reference Deakin, Cheung and Petley16,Reference Mirza, Brown and Saini17 Using a Delphi method, the initial scale was sent to five experts in adult and pediatric emergency medicine. The scale was reviewed and revised, taking into account the comments made by these experts, wherein items were added, removed, or modified until a consensus of at least 65% agreement was reached.Reference Ghazali, Richard and Chaudet18 To analyze the experts’ answers, a seven-point Likert scaleReference Wadgave and Khairnar19 was used for each item: (1) Strongly disagree; (2) Disagree; (3) Slightly disagree; (4) Neither agree nor disagree; (5) Slightly agree; (6) Agree; and (7) Strongly agree. The experts could also make comments and suggestions. For each item, if 65% of the experts rated one or two, the item was removed; if 65% of the experts rated three or four, the item was modified according to the comments from the experts; and if 65% of the experts rated original or modified items the score of six or seven, the items were maintained.

Response Process

The aim of this step was to run a pilot test to control some sources of errors, inaccuracies, and redundancies in the scale. The scale was tested by applying it for the assessment of 10 telephone-CPR simulations prior to the trial’s beginning. Five emergency physicians and five assistants from the academic hospital of Poitiers were asked to carry out phone assistance for 10 medical students from the University of Poitiers who had volunteered to carry out simulated-CPR during a low-fidelity simulation-based training (SBT). Six evaluators experienced in delivering CPR SBT were trained to use the scale during a 30-minute course to explain the study aim and to comment on the scale. Simulations were video-recorded and then assessed by these evaluators. The scale was revised anew, taking into account the comments made by the observers.

Population

The six trained and experimented evaluators were enrolled in the present trial to assess the simulations using the telephone CPR scale. In addition, four novice and unskilled evaluators from the CPR phone assistance process received the same training to use the scale. All the participants were assessed by two experienced independent evaluators and one unskilled evaluator, who each watched the video recordings independently of one another. Evaluators were blinded from the group allocation of bystanders and their experience or not in delivering chest compressions, rescue breaths, and defibrillating. They were also blinded from the professional status of dispatchers (ie, physicians or assistants). Finally, they were blinded from the objective CPR-performance scores of bystanders.

Eighty students from the Medicine and Pharmacy Faculties of Poitiers were invited to participate in the study as bystanders. Among them, a group of 60 students received one month prior a one-day basic CPR training, while 20 students had not yet received the course. Among the 60 trained students, 42 consented to participate to the study and were recruited in the experimental group (trained bystanders). The students who constituted the experimental group were randomized to carry out one of the two scenarios (Scenario 1 and Scenario 2) used in the present SBT. Among the 20 untrained CPR students, 15 accepted to participate and were enrolled in the control group (untrained bystanders). Participants who agreed to participate and to sign the consent form were included. Non-inclusion criteria were the existence of physical constraints preventing participants from performing CPR or ventilation and the non-consent to video recording. Exclusion criteria were missed session and non-operable video recording. All in all, 48 students were enrolled after application of criteria, including 35 students in the experimental group and 13 students in the control group. It was decided to compare the same number of participants in the experimental and control groups. Consequently, in the experimental group, 22 students were randomized to carry out Scenario 1 (Group E1). The 13 other students of the experimental group (Group E2) and 13 students of the control group (Group C) carried out Scenario 2. Dispatchers from the Grande Aquitaine region and Paris (France) with at least two years’ experience in dispatch were also recruited to be randomly paired into the three groups with bystanders to perform the SBT. Non-inclusion criteria were to have received previous assisted telephone-CPR training and non-consent to video recording. Exclusion criteria were missed session and non-operable video recording. Dispatchers were recruited until having the same number as bystanders. Dispatchers included 21 emergency physicians and 27 medical dispatch assistants. Flow chart is given in Figure 1.

Figure 1. Study Design and Flowchart.

Intervention

Bystanders and dispatchers carried out in pairs a 25-30-minute SBT, including five minutes of briefing, 10-15 minutes of simulation, and a 10-minute “plus/delta.”Reference Bajaj, Meguerdichian and Thoma20 The SBT used a Resusci Anne QCPR Manikin (Laerdal; Stavanger, Norway). The manikin was connected with SkillReporter (PC) software (Laerdal; Stavanger, Norway) to provide CPR metrics. Briefing and debriefing were carried out by trained supervisors from the simulation centers of the Faculties of Medicine of Poitiers and Paris. Scenario 1 aimed to assess all the items of the scale (ie, dispatchers’ performance and bystanders’ performance, including chest compressions combined with effective rescue breaths and AED use). This scenario carried out by Group E1 included a combination of chest compression and ventilation for trained bystanders as recommended by the ERC Guidelines. It consisted of an unconscious adult lying still on the floor on one side with no signs of breathing. The victim was a 55-year-old teacher who experienced chest pain before collapsing to the ground. In the hallway where the classroom was located, a defibrillator was present. Trainees were informed of the presence of the defibrillator. The participant was accompanied by a facilitator from the simulation center of the Faculty of Medicine of Poitiers. No other bystander was present during the session. A cellular phone was provided with a possibility to call for rescuers. Dispatchers were in another room, without any view on bystanders’ CPR, with another cellular phone to ask questions to bystanders and provide CPR instructions. Participant and facilitator had to follow dispatcher’s instructions to assess and treat the unresponsive victim according to the BLS sequence. The facilitator could retrieve AED from the hallway at the request of the bystander or the dispatcher. The bystander’s objective CPR-performance scores were downloaded from the Laerdal Recording Resusci Anne SkillReporter Manikin into a computerized database. These data extracted from the recording manikin had been reported in previous research as being reliable.Reference Whitfield, Newcombe and Woolard21,Reference Rivera-Tovar and Jones22 Scenario 2 aimed to provide a common objective to trained and untrained bystanders (Group E2 and Group C, respectively). Since the ERC Guidelines recommended continued compressions at a rate of 100-120/minute without rescue breaths delivered by untrained bystanders, in Scenario 2, the patient presented a facial trauma with no possibility for a trained bystander to give rescue breaths. Consequently, the objectives of this scenario were to rate dispatchers’ performance and trained and untrained bystanders’ performance including chest compressions and AED use.

The simulation ended at the arrival of rescuers planned 10 to 15 minutes after starting CPR. All the bystanders’ and dispatchers’ simulations were video recorded.

Internal Structure

The internal structure of the scale was analyzed based on the evaluations of performances during Scenario 1 which required carrying out all the items on a homogeneous population of 44 trained bystanders and dispatchers.

Relationship to Other Variables

Without any other performance assessment scale found in the literature, the performances in dispatch between dispatch doctors and medical dispatch assistants were compared, as well as the performances in CPR between bystanders with or without CPR training using Scenario 2. A correlation between dispatchers’ performance and bystanders’ objective performance was searched.

Statistical Analysis

Analysis was performed with StatView version 4.5 (SAS Institute Inc.; Cary, North Carolina USA). Statistical analysis was carried out after the second series of simulations (internal structure). Kolmogorov-Smirnov test was used to check normal distribution for assessed measures. Continuous variables were described as mean (SD) and categorical variables as numbers and percentage (%). Comparative analysis between two groups used a Student t-test or a non-parametric test of Mann-Whitney. Comparative analysis between the different groups used Kruskal-Wallis non-parametric test. Internal consistency was analyzed by calculating α Cronbach coefficient. The minimal standards recommended for α Cronbach value are >0.5 and could be considered as excellent when >0.75.Reference Peterson23 Interobserver reproducibility was analyzed by intraclass correlation coefficient (ICC) for the whole scale and each variable, comparison of means, and linear logistic regression. Values of ICC less than 0.5, between 0.5 and 0.75, between 0.75 and 0.9, and greater than 0.90 were indicative of poor, moderate, good, and excellent reliability, respectively.Reference Koo and Li24 Because several observers were included in the assessment, F-test was used to compare variance of scores obtained by the two observers. The correlation between scores in the various groups was analyzed with the Spearman correlation test. A P value <.05 was considered significant.

Results

Population

Ninety-six participants were enrolled, including 48 students and 48 dispatchers. There were 38 (39.6%) males and 58 (60.4%) females. Dispatchers’ level of experience (in years) was 6.6 (SD = 4.0) with no difference in dispatch doctors’ and dispatch medical assistants’ years of experience: 6.2 (SD = 3.5) versus 9.0 (SD = 5.5); P = .13. There was no difference in dispatchers’ level of experience between Group E2 and Group C (ie, included groups to carry out Scenario 2 to compare performance scores): 8.8 (SD = 6.4) versus 6.4 (SD = 3.1); P = .21. The 48 students were 22.8 (SD = 1.3) years old without any difference between the three groups E1, E2, and C: 23.1 (SD = 1.5); 22.4 (SD = 1.2); and 22.5 (SD = 1.2), respectively (H = 3.0; P = .21).

Validation of the Content

The first scale included three different parts: a first part for dispatchers’ assessment with 32 items, a second part for assessment of bystanders’ CPR performance with 15 items, and a third part with time to recognize the cardiac arrest and time to initiate bystander CPR. The final scale (Appendix 1; available online only) also included three parts assessing dispatchers’ and bystanders’ performances, with a maximum score of 70, and times. The dispatcher’s performance included 38 items with a maximum score of 52 and the second part included nine items with a maximum score of 18. An optional and complementary objective CPR-performance assessment including 10 items and carried out with a SkillReporter device was added to the scale. The global objective performance score is given in percent. Finally, different times were recorded: T0 = Beginning of phone call; T1 = Time to recognize cardiac arrest by dispatcher; T2 = Time to dispatch rescuers; T3 = Time to start CPR by bystander; and T4 = Time to start using AED.

Internal Structure

Tested on the whole population, there was excellent internal consistency with α Cronbach coefficient of 0.77 and reproducibility with ICC of 0.93 and linear correlation coefficient R² of 0.86 (Figure 2). For dispatchers’ performance assessment, α Cronbach coefficient was 0.76. The ICC was 0.91 and the linear correlation coefficient RReference Baekgaard, Viereck and Moller2 was 0.84 (Figure 3), indicating excellent reproducibility. There was no statistically significant difference between the evaluators’ assessments of means (P = .74) and SDs (P = .96). For bystanders’ performance assessment, α Cronbach coefficient was 0.75. Assessment of CPR performance showed that reproducibility was good for the unskilled evaluators (ICC = 0.68; RReference Baekgaard, Viereck and Moller2 = 0.42) and excellent for the trained evaluators with ICC of 0.93 and linear correlation coefficient RReference Baekgaard, Viereck and Moller2 of 0.87 (Figure 4). There was no statistically significant difference between the evaluators’ assessments of means (P = .94) and SDs (P = .88).

Figure 2. Linear Regression for Dispatchers’ and Bystanders’ Performance Scores (out of 70) Assessment.

Note: E1 = Evaluator 1; E2 = Evaluator 2.

Figure 3. Linear Regression for Dispatchers’ Performance Scores (out of 52) Assessment.

Note: E1 = Evaluator 1; E2 = Evaluator 2.

Figure 4. Linear Regression for Bystanders’ Performance Scores Assessment (out of 18) by Trained Evaluators.

Note: E1 = Evaluator 1; E2 = Evaluator 2.

The reproducibility for each item is reported in Table 1. Reproducibility was excellent for nine items, good for 19 items, and moderate for 19 items. No item had poor reproducibility. Considering the items for which both evaluators gave the score of zero, the scale highlighted items with a very low completion rate (Table 1). Among dispatchers, in 22 cases, items were completed in more than 50% of the simulations. Seven items were completed in less than one-half of simulations and nine were completed by dispatchers in less than one-quarter of simulations. The items that were least completed were mainly the questions to be asked to the bystanders. It bears mentioning that the requested release sequence during CPR was not indicated to bystanders in 81.8% of simulations. For bystanders, one item (correct electrode pad positioning) was incorrectly done in more than one-quarter of simulations.

Table 1. Metrics of the Scale

Abbreviations: AED, automatic external defibrillator; CPR, cardiopulmonary resuscitation.

a Items’ score of zero for the two evaluators.

Comparison of Performance Scores

The dispatchers’ average score was 32.8 (SD = 4.4) out of 52. There was no significant difference between doctors and medical assistants (33.0 [SD = 4.7] versus 32.3 [SD = 3.2], respectively; P = .70). The average recognition time (seconds) of cardiac arrest (ie, T1-T0) was 48.4 (SD = 16.70). There was no significant difference between doctors and medical assistants (47.7 [SD = 16.34] versus 49.4 [SD = 18.16], respectively; P = .83). Time (seconds) to begin the CPR (ie, T3-T0) was 105.0 (SD = 50.2). This time was significantly lower among trained bystanders than among untrained bystanders (79.3 [SD = 49.9] versus 131.1 [SD = 36.4]; U test = 15.0; P = .003). Bystanders’ average score to follow the instructions was 14.1 (SD = 2.0) out of 18 without any significant difference between trained and untrained bystanders (14.3 [SD = 2.0] versus 13.9 [SD = 1.8], respectively; P = .64). There was no correlation between dispatchers’ and bystanders’ performances (Rho = 0.17; P = .43).

Data details collected from recording manikin and statistical comparisons are reported in Table 2. Overall performance (%) was 60.0 (SD = 18.8); it was significantly higher for trained bystanders than for untrained bystanders (67.4 [SD = 14.5] versus 50.6 [SD = 19.3], respectively; P = .03). The total number of compressions was significantly higher for trained bystanders than for untrained bystanders (339.4 [SD = 122.74] versus 238.4 [SD = 115.0], respectively; P = .01). The average compression rate (per minute) was 93.6 (SD = 26.1) with a significant difference between trained bystanders and untrained bystanders (104.1 [SD = 19.6] versus 81.0 [SD = 27.4]; P = .04). There was no significant difference for the other data: compressions with correct hand position (%), average compression depth (mm), and compressions with full release (%). Even though there was no difference between the two groups for compression depth with less than 50mm, percentage of compressions done with the right depth was significantly higher for trained bystanders than for untrained bystanders (44.7 [SD = 32.3] versus 32.6 [SD = 36.3]; P = .04).

Table 2. CPR Performance Scores Downloaded from the Laerdal Recording Resusci Anne SkillReporter Manikin (mean (SD))

Discussion

A valid, reliable tool for evaluation of performance for simulated dispatcher-assisted CPR, using the framework introduced by Downing, was created. The ERC and AHA Guidelines 2015 highlight dispatcher-assisted CPR and affirm that dispatchers should be trained to interrogate bystanders with strict protocols to elicit information focused on early recognition of cardiac arrest.3,4 Moreover, dispatchers should provide telephone-CPR instructions in all cases of suspected cardiac arrest.3,4 While an objective assessment of training outcomes is needed, currently, there is no other performance assessment scale for simulated dispatcher-assisted CPR. A recent study analyzed the clinical effects of the implementation of the telephone-CPR program on the outcomes of OHCA. The authors found that after implementation of the telephone-CPR program, the number of successful CPR cases and survival rate increased, and brain complications and CPR-related complications decreased. However, there were some limitations since the study lacked objective data such as ambulance arrival time and the experience of dispatchers as well as bystanders.Reference Seyed Bagheri, Sadeghi and Kazemi10 The present scale offers an objective assessment of telephone-CPR.

The scale has excellent validity. The “dispatcher’s assessment” part had excellent internal consistency and reproducibility. No research currently exists in the literature to be compared with these results. The “assessment of CPR performance” part had excellent reproducibility with trained evaluators who watched video recordings. Indeed, it has been proved that video recording may be capable of improving inter-observer reliability.Reference Whitfield, Newcombe and Woolard21,Reference Lester, Morgan and Donnelly25,Reference Donnelly, Lester and Morgan26 Valid CPR performance assessment tools were created to assess CPR performance after CPR training.Reference Whitfield, Newcombe and Woolard21,Reference Lester, Morgan and Donnelly25,Reference Donnelly, Lester and Morgan26 But as the previous scales consisted of tasks to do in a prescribed sequence according to CPR training course and not in relationship with telephone-CPR, they could not be used.

All in all, reproducibility was excellent for 19.2% of items, good for 40.4% of items, and moderate for 40.4% of items. For some items with moderate reproducibility, this could be explained by confusion of interpretation of items by the evaluators, such as: “He/she is in cardiac arrest” and “You must start cardiopulmonary resuscitation;” “Trained bystander?” and “Can you do it?;” “Reinsurance” and “The rescuers have been sent;” and “Ensure the continuity of the resuscitation” and “Repeat encouragement.” For items of accessibility and ability to recall, since the score can be only zero or one, some evaluators could consider that all details were required to validate the item, whereas other evaluators could consider the item as validated if it was indicated by the dispatcher during the simulation. These results suggest that evaluators should be trained to use the scale and to assess trainees prior to their being enrolled in an educational program. Moreover, the present results highlighted that the objectives of the simulation, as well as level of detail, should be clarified before starting the simulation. For items on bystanders’ performance, there could be some confusion because many trained bystanders had already carried out these actions before calling for help. For other items, it might be difficult to assess actions on the manikin (opening the airways, positioning of the arms). In other trials,Reference Whitfield, Newcombe and Woolard21,Reference Lester, Morgan and Donnelly25,Reference Donnelly, Lester and Morgan26 observers were recruited from a wide range of professional backgrounds (from people without any experience in CPR, like research assistants, to CPR instructors), but the observers received training of at least 30 minutes to use their scoring system, with written instructions explaining how to note the different items, and they could watch and evaluate videotapes and discuss inter-observer differences. They all found good to perfect agreement between observers for their scale and each item except “opening the airway” in the trial directed by Lester, et alReference Rivera-Tovar and Jones22 and “visual placement of the hands” for Donnelly, et al.Reference Donnelly, Lester and Morgan26 Further development of the present scale is therefore required to improve the reproducibility of these items. The present scale was only introduced to the observers, and it would be advisable for the next trials to add written instructions and previous training to all the observers reviewing and assessing videotapes to practice scoring, thereby increasing the reproducibility of the scale. Differences of reproducibility when the scale was used by trained (excellent reproducibility) or untrained evaluators (good reproducibility) reinforce this hypothesis. Since internal consistency was excellent and no item had poor reproducibility, the present scale is a valid and reliable phone-assisted CPR assessment tool in simulation. Given a reliable scoring system, comparison of skills between different groups of people, different training courses, or successive dispatcher-assisted CPR attempts become possible. It can also provide feedback, which is considered important in the transfer of knowledge and skills.Reference Rivera-Tovar and Jones22,Reference Vanderschmidt, Burnap and Thwaites27 All the participants highly appreciated reviewing the different items, especially those that were often not taken up by dispatchers or bystanders. As suggested in Table 1, assessment using the present scale highlighted some of technical or non-technical items that SBT should focus on the latter; this is not sufficiently done in the whole population (score of zero for the two evaluators in more than one-quarter of the trainees). Moreover, individual debriefing should focus on items that were not done (ie, score of zero for the two evaluators). The scale was also easily applied in practice and was understood by the independent evaluators. There was no statistically significant difference between the performance and the average recognition time of cardiac arrest between the doctors and the medical dispatch assistants, as was also found in a previous trial.Reference Sigaux28 For the bystanders’ items, the highest ICCs were found for items related to AED use, meaning that they were easier to evaluate. In their scale, Whitfield, et al found excellent agreement for all the items about the use of AED.Reference Whitfield, Newcombe and Woolard21

The present study highlighted the need to assess objective parameters of CPR performance, since some items are not easy to be assessed by evaluators. Moreover, using the present scale, it was found that trained and untrained bystanders were able to follow the instructions. Regarding the objective CPR performance scores, trained bystanders had significantly better scores for most of the items. However, there was no significant difference between some of the items, on which CPR courses should focus. As presented in Table 2, compressions with correct hand position, compression depth, and compressions with full release are crucial items to be improved. By watching the videotapes, the untrained bystanders started CPR after all the instructions had been given by the dispatcher and often stopped CPR while new instructions were provided or when they stopped to do the metronome. This result is comparable to Guysen’s trial.Reference Ghuysen, Collas and Stipulante14 Even if the total average compression rate was close to the recommended rate of 100-120/minute for untrained bystanders, the percentage of compressions done with the right rate was low for both untrained and trained bystanders. However, according to the observers, 75% of the dispatchers did the metronome at the right rate, which was proved to better the compression rate.Reference Deakin, Cheung and Petley16 This item had average reproducibility and the evaluators had no stopwatch to check the rate provided by the dispatchers, a lack which could explain this discrepancy.

The average compression depth was lower than the recommended depth of five to six centimeters, whereas 77.6% of the dispatchers indicated it in the right way and at the beginning of the CPR. Many other trials found the same results; the bystanders were either people without any medical qualificationsReference Ghuysen, Collas and Stipulante14,Reference Deakin, Cheung and Petley16 or health professionals.Reference Brown, Dias and Saini29 In contrast, more than one-half of the compressions were done with full release, while 81.8% of the dispatchers did not request release after the compressions and did not specify full release. There was no correlation between dispatchers’ and bystanders’ performance, meaning that dispatchers and bystander should be trained. Dispatchers should be trained since all bystanders seemed to be able to follow instructions, and bystanders should be trained since instructions are not sufficient to carry out high-quality CPR.

This scale is valid and reliable to evaluate performance for simulated dispatcher-assisted CPR as well as bystanders in training program; the two parts of the scale had excellent validity and reliability.

Limitations

This study is not without limitations. The scale was evaluated in simulations where the influence of stress could be different from real situations. It could explain why the times to recognize cardiac arrest and to initiate CPR were shorter than in other trials assessing the use of protocols on real OHCA.Reference Travers, Jost and Gillard6,Reference Rea, Eisenberg and Culley13

The study population was small since the protocol aimed to enroll in an initial scenario a population of trained bystanders to assess all the items of the scale. In a second scenario, the study protocol aimed to compare untrained and trained bystanders to look for a difference in performance scores. It was not possible to use only the second scenario, since ERC and AHA Guidelines 2015 emphasized that bystanders who were untrained or unwilling to give rescue breaths should not do so. Moreover, the protocol aimed to assess dispatchers’ and bystanders’ performance, since there are currently few studies on which to base such choice more scientifically, all the more they assess bystanders’ performance only.Reference Whitfield, Newcombe and Woolard21,Reference Lester, Morgan and Donnelly25,Reference Donnelly, Lester and Morgan26

Finally, the present scale could be used to assess pediatric CPR by changing ratio of compressions and breaths, depth of chest compression, five rescue breaths first, and hand position. However, it was not validated on a pediatric scenario.

Conclusion

The scale was valid and reliable as a means of assessing performance for simulated dispatcher-assisted telephone-CPR according to the framework introduced by Downing. It is the first scale which assess simulated dispatcher-assisted CPR and CPR performance on a manikin. The use of protocols and dispatchers’ training for dispatcher-assisted CPR are some of the key messages from the ERC and AHA Guidelines 2015. This scale could allow objective measurement of training outcomes and comparison of them across studies.

Conflicts of interest/funding

None of the authors have any financial conflicts of interest in connection with the work to disclose. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Acknowledgments

The authors would like to thank Jeffrey Arsham, an American scientific translator, for having reviewed the English language text.

Supplementary Materials

To view supplementary material for this article, please visit https://doi.org/10.1017/S1049023X21000716

References

Sasson, C, Rogers, MA, Dahl, J, et al. Predictors of survival from out-of-hospital cardiac arrest: a systematic review and meta-analysis. Circ Cardiovasc Qual Outcomes. 2010;3(1):6381.CrossRefGoogle ScholarPubMed
Baekgaard, J, Viereck, S, Moller, T, et al. The effects of public access defibrillation on survival after out-of-hospital cardiac arrest: a systematic review of observational studies. Circulation. 2017;136(10):954965.CrossRefGoogle ScholarPubMed
European Resuscitation Council Guidelines for Resuscitation 2015. Resuscitation. 2015;95:1311.CrossRefGoogle Scholar
American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science. Circulation. 2015;132(Suppl 2):S313S589.CrossRefGoogle Scholar
Perkinsa, GD, Handley, AJ, Koster, RW, et al. European Resuscitation Council Guidelines for Resuscitation 2015: Section 2. Adult Basic Life Support and automated external defibrillation. Resuscitation. 2015;95:8199.CrossRefGoogle Scholar
Travers, S, Jost, D, Gillard, Y, et al. Out-of-hospital cardiac arrest phone detection: those who most need chest compressions are the most difficult to recognize. Resuscitation. 2014;85(12):17201725.CrossRefGoogle ScholarPubMed
Stipulante, S, Tubes, R, El Fassi, M, et al. Implementation of the ALERT algorithm, a new dispatcher-assisted telephone cardiopulmonary resuscitation protocol, in non-Advanced Medical Priority Dispatch System (AMPDS) Emergency Medical Services centers. Resuscitation. 2014;85(2):177181.CrossRefGoogle Scholar
Castren, M, Kuisma, M, Serlachius, J, et al. Do health care professionals report sudden cardiac arrest better than laymen? Resuscitation. 2001;51(3):265268.CrossRefGoogle ScholarPubMed
Dami, F, Fuchs, V, Praz, L, et al. Introducing systematic dispatcher-assisted cardiopulmonary resuscitation (telephone-CPR) in a non-Advanced Medical Priority Dispatch System (AMPDS): implementation process and costs. Resuscitation. 2010;81(7):848852.CrossRefGoogle Scholar
Seyed Bagheri, SM, Sadeghi, T, Kazemi, M, et al. Dispatcher-assisted bystander cardiopulmonary resuscitation (telephone-CPR) and outcomes after out-of-hospital cardiac arrest. Bull Emerg Trauma. 2019;7(3):307313.CrossRefGoogle ScholarPubMed
Bohm, K, Vaillancourt, C, Charette, M L, et al. In patients with out-of-hospital cardiac arrest, does the provision of dispatch cardiopulmonary resuscitation instructions as opposed to no instructions improve outcome: a systematic review of the literature. Resuscitation. 2011;82(12):14901495.CrossRefGoogle ScholarPubMed
Tanaka, Y, Taniguchi, J, Wato, Y, et al. The continuous quality improvement project for telephone-assisted instruction of cardiopulmonary resuscitation increased the incidence of bystander CPR and improved the outcomes of out-of-hospital cardiac arrests. Resuscitation. 2012;83(10):12351241.CrossRefGoogle ScholarPubMed
Rea, TD, Eisenberg, MS, Culley, LL, et al. Dispatcher-assisted cardiopulmonary resuscitation and survival in cardiac arrest. Circulation. 2001;104(21):25132516.CrossRefGoogle ScholarPubMed
Ghuysen, A, Collas, D, Stipulante, S, et al. Dispatcher-assisted telephone cardiopulmonary resuscitation using a French-language compression-only protocol in volunteers with or without prior life support training: a randomized trial. Resuscitation. 2011;82(1):5763.CrossRefGoogle ScholarPubMed
Downing, SM. Validity: on meaningful interpretation of assessment data. Med Educ. 2003;37(9):830837.CrossRefGoogle ScholarPubMed
Deakin, CD, Cheung, S, Petley, GW, et al. Assessment of the quality of cardiopulmonary resuscitation following modification of a standard telephone-directed protocol. Resuscitation. 2007;72(3):436443.CrossRefGoogle ScholarPubMed
Mirza, M, Brown, TB, Saini, D, et al. Instructions to « push as hard as you can » improve average chest compression depth in dispatcher-assisted cardiopulmonary resuscitation. Resuscitation. 2008;79(1):97102.CrossRefGoogle Scholar
Ghazali, DA, Richard, A, Chaudet, A, et al. Profile and motivation of patients consulting in emergency departments while not requiring such a level of care. Int J Environ Res Public Health. 2019;16(22):4431.CrossRefGoogle Scholar
Wadgave, U, Khairnar, MR. Parametric tests for Likert scale: for and against. Asian J Psychiatr. 2016;24:6768.CrossRefGoogle ScholarPubMed
Bajaj, K, Meguerdichian, M, Thoma, B, et al. The PEARLS Healthcare Debriefing Tool. Acad Med. 2018;93(2):336.CrossRefGoogle ScholarPubMed
Whitfield, RH, Newcombe, RG, Woolard, M. Reliability of the Cardiff test of basic life support and automated external defibrillation version 3.1. Resuscitation. 2003;59:291314.CrossRefGoogle ScholarPubMed
Rivera-Tovar, LA, Jones, RT. Effect of elaboration on the acquisition and maintenance of cardiopulmonary resuscitation. J Pediatr Psychol. 1990;15(1):123128.CrossRefGoogle ScholarPubMed
Peterson, RA. A meta-analysis of Cronbach’s Coefficient Alpha. J Consum Res. 1994;21(2):381391.CrossRefGoogle Scholar
Koo, TK, Li, MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155163.CrossRefGoogle ScholarPubMed
Lester, CA, Morgan, CL, Donnelly, PD, et al. Assessing with CARE: an innovative method of testing the approach and casualty assessment components of basic life support using video recording. Resuscitation. 1997;34(1):4349.CrossRefGoogle ScholarPubMed
Donnelly, PD, Lester, CA, Morgan, CL, et al. Evaluating CPR performance in basic life support: the VIDRAP protocol. Resuscitation. 1998;36(1):5157.CrossRefGoogle ScholarPubMed
Vanderschmidt, H, Burnap, TK, Thwaites, JK. Evaluation of a cardiopulmonary resuscitation course for secondary schools. Med Care. 1975;13(9):763774.CrossRefGoogle ScholarPubMed
Sigaux, A. Evaluation de la conformité du conseil téléphonique au centre 15 en cas d’arrêt cardiaque. Ann Fr Med Urgence. 2014;4:1117.CrossRefGoogle Scholar
Brown, TB, Dias, JA, Saini, D, et al. Relationship between knowledge of cardiopulmonary resuscitation guidelines and performance. Resuscitation. 2006;69(2):253261.CrossRefGoogle ScholarPubMed
Figure 0

Figure 1. Study Design and Flowchart.

Figure 1

Figure 2. Linear Regression for Dispatchers’ and Bystanders’ Performance Scores (out of 70) Assessment.Note: E1 = Evaluator 1; E2 = Evaluator 2.

Figure 2

Figure 3. Linear Regression for Dispatchers’ Performance Scores (out of 52) Assessment.Note: E1 = Evaluator 1; E2 = Evaluator 2.

Figure 3

Figure 4. Linear Regression for Bystanders’ Performance Scores Assessment (out of 18) by Trained Evaluators.Note: E1 = Evaluator 1; E2 = Evaluator 2.

Figure 4

Table 1. Metrics of the Scale

Figure 5

Table 2. CPR Performance Scores Downloaded from the Laerdal Recording Resusci Anne SkillReporter Manikin (mean (SD))

Supplementary material: File

Ghazali et al. supplementary material

Ghazali et al. supplementary material

Download Ghazali et al. supplementary material(File)
File 31.7 KB