Introduction
Demand for emergency health care services is increasing in Australia.Reference Tohira, Fatovich, Williams, Bremner, Arendts, Rogers, Celenza, Mountain, Cameron, Sprivulis, Ahern and Finn 1 , Reference Lowthian, Cameron and Stoelwinder 2 St John Ambulance Western Australia (SJA-WA; Belmont, Western Australia) experienced a 29.0% increase in activity in the Perth metropolitan area, the capital city of the state of Western Australia (WA), between the 2006/2007 and 2012/2013 financial years (138,996 to 194,445 cases). 3 Similarly, emergency department (ED) attendances increased by 17.0% between the 2008/2009 and 2012/2013 financial years.Reference Lowthian, Cameron and Stoelwinder 2 These increases are greater than annual population growth (3.3%). 4
Various approaches have been initiated to reduce ED attendances, including alternative treatment pathways rather than default ambulance transportation to ED. Paramedics with extended skills (eg, Extended Care Paramedics [ECP]) have been introduced in other states in Australia. 5 , Reference Malone 6 Similar to the Emergency Care Paramedic role in the UK, 7 ECPs are trained to make autonomous decisions as to whether a patient requires transportation to ED, or care in the community, or can be discharged at the scene. These new roles for paramedics have reduced the number of transportations to ED; however, the appropriateness of these paramedic decisions has not been well-assessed. 8
To help paramedics decide the most appropriate pathway for a patient, the North West Ambulance Service in the UK developed two generic flow-chart-type clinical decision tools (one for medical etiologies and another for trauma), which they called Paramedic Pathfinders.Reference Tohira, Williams, Jacobs, Bremner and Finn 9 The Paramedic Pathfinder first identifies patients who require transportation to ED based on symptoms and vital signs (eg, headache and body temperature). The self-care pathway (SCP) assessment checklists, which are part of the Paramedic Pathfinder, are then used for patients who were not selected for transportation to ED to determine whether a patient can be discharged appropriately at the scene or should be referred to an Urgent Care Centre.Reference Newton, Tunn, Moses, Ratcliffe and Mackway-Jones 10
The SCP assessment checklists are applicable only to lower acuity patients who have been assessed as not requiring transportation to ED, as described above. Checklists are available for two medical conditions (post-ictal state and hypoglycemia) and four types of trauma (minor injuries, minor head injuries, neck injuries, and falls).Reference Newton, Tunn, Moses, Ratcliffe and Mackway-Jones 10 If a patient presents with one of these six etiologies and meets all criteria in the SCP assessment checklist for that etiology, then the patient can be discharged at the scene. Although the overall accuracy of the Paramedic Pathfinder was reported to be around 80.0%,Reference Tohira, Williams, Jacobs, Bremner and Finn 9 the accuracy of the SCP assessment checklists was not described. The accuracy of the SCP assessment checklists is more important than overall accuracy of the Paramedic Pathfinder because paramedics need to make crucial decisions regarding the destination of lower acuity patients (ie, discharge at the scene or transport to an Urgent Care Centre) while paramedics would already transport higher acuity patients to ED. This study aimed to answer the following two questions: (1) Do the SCP assessment checklists accurately identify post-ictal or hypoglycemic patients suitable for discharge at the scene? and (2) Do SCP assessment checklists identify patients suitable for discharge at the scene more accurately than paramedics can, relying only on clinical judgment?
Methods
Setting
This study was conducted in Perth, WA. The estimated population of the Perth metropolitan area was 1.97 million in 2013, and the land area is 5,400 km2. 4 St John Ambulance Western Australia is the sole emergency ambulance service provider in WA. St John Ambulance Western Australia paramedics receive four years of on-the-job and formal training, which includes a three-year University bachelor degree. There are 10 public hospital EDs and one private hospital ED in the Perth metropolitan area. Total annual attendances to the public EDs was 530,237 in 2013.Reference Newton 11
Data
Ambulance data from SJA-WA collected from January 1, 2013 through December 31, 2013 were used to identify the study cohort. All ambulance activities and patient information were collected routinely by paramedics through an electronic patient care record (e-PCR) system. The ambulance data were then subsequently linked with the ED information system (EDIS) data. These EDIS data were available for eight public hospital EDs and they captured 90.0% of all public hospital ED attendances in 2013.Reference Newton 11 The EDIS data were used to obtain the single diagnosis at ED discharge and to identify subsequent ED presentation after the index event. Data from the WA death registry were also linked with the ambulance data to identify subsequent deaths. The linkage of these three data sources was performed using probabilistic matching software (FRIL ver.2.1.5; Emory University and Centers for Disease Control and Prevention; Atlanta, Georgia USA). 12 All ambulance transports to ED were matched to an EDIS record using this probabilistic linkage software supplemented by manual matching.
Self-care Pathway Assessment Checklists
The study retrospectively evaluated two SCP assessment checklists for medical conditions (post-ictal and hypoglycemia). The trauma SCP assessment checklists were not used because the four minor trauma etiologies (minor head injury, minor injury, neck injury, and fall) could not be distinguished accurately from the available information in the ambulance data. The SCP assessment checklists for post-ictal states and hypoglycemia are shown in Table 1. The following two criteria were not used, “4. No evidence of the patient feeling unwell prior to the seizure” in the post-ictal checklist and “10. No irritability or altered behavior” in the hypoglycemia checklist, because of the lack of information in the ambulance data for these two criteria. The effect of these exclusions on the performance of the SCP checklists is discussed in the Analysis.
Table 1 The UK North West Ambulance Service Self-care Pathway Assessment ChecklistsReference Newton, Tunn, Moses, Ratcliffe and Mackway-Jones 10
a These criteria were not used in this study.
Study Cohorts
Two study cohorts, namely post-ictal and hypoglycemic patients, were identified. The study cohorts comprised all adult patients (age >16 years) who had one of these two conditions recorded by paramedics on the e-PCR as the presenting condition and who were assessed as lower acuity. A free-text variable, which described the patient’s condition at the scene, was also examined to identify additional post-ictal or hypoglycemic patients. If the variable included one of the keywords (eg, “post-ictal” or “hypoglycemia”), the description was examined to determine whether the patient could be included in the study cohort. Patients whose blood sugar level (BSL) was less than 4.0 mmol/LReference Jurczyk, Lu, Xiong, Cragan and Correa 13 during initial prehospital assessment were also included in the hypoglycemic cohort, even if the presenting condition was recorded as another condition.
The paramedic-determined prehospital triage level and vital signs were used to select lower acuity patients. The triage level is an ordinal scale ranging from one (requiring immediate care) to five (to be treated within 120 minutes). Lower acuity patients were defined as those who were assigned triage level three (to be treated within 30 minutes), four, or five 14 and did not have one or more abnormal vital signs (systolic blood pressure <90 mmHg; oxygen saturation <95%; Glasgow Coma Scale (GCS) <15 for hypoglycemic and <13 for post-ictal patients; or temperature ≥ 38°C) during the prehospital phase. Patients with GCS =13 or 14 for the post-ictal cohort were included because these patients tend to show decreased verbal and/or motor responses. The cut-off GCS value was arbitrarily set at 13 because there are no reports regarding a GCS that is representative of the post-ictal state. Spaite et al reported that seizure patients with an initial GCS >10 were less likely to receive interventions and be transported to ED than those with GCS ≤ 10.Reference Sprivulis, Grainger and Nagree 15 Hence, the cut-off GCS value of 13 appeared to be conservative compared to that report and should be acceptable for identifying lower acuity, post-ictal patients. Patients who were both post-ictal and hypoglycemic, who were transported from another hospital, or who were transported by appointment were excluded from the study cohort.
Outcome Measures
The primary outcome measure was the patient’s destination from the scene (ie, discharge by paramedics at the scene or transportation to ED). Originally, the SCP checklist determined whether a patient could be discharged at the scene or should be transported to an Urgent Care Centre, which does not exist in WA. Therefore, transportation to ED was considered as equivalent to transport to an Urgent Care Centre. Secondary outcomes included disposition after ED (discharge from ED or admission to hospital) and subsequent ambulance requests; ED presentations; hospitalization from ED; and death within 24 hours, three days, and seven days from the index event. It was also examined whether the subsequent ambulance requests were related to seizure or BSL derangement (eg, hypoglycemia or hyperglycemia) using the paramedics’ documented prehospital medical conditions and whether the subsequent ED attendances were related to seizure or diabetes mellitus (DM) using the ED discharge diagnosis. Prehospital medical conditions were identified by paramedics and recorded using SJA-WA-specific codes. Emergency department discharge diagnosis was recorded using the International Classification of Diseases, Tenth Revision, Australian Modification (ICD-10-AM) code.Reference Spaite, Valenzuela, Meislin, Criss and Ross 16 The ICD-10-AM codes of G40, G41, and R56 were considered as a seizure-related diagnosis, and those of E10, E11, E14, E15, and E16 a DM-related diagnosis.
Analysis
For each of the two patient groups (post-ictal and hypoglycemia), the information extracted from the ambulance data was applied retrospectively to the SCP assessment checklists, and patients who were suitable for discharge at the scene were identified based on the checklists. For the patients who were identified by these checklists, actual destination determined by paramedics (discharge at the scene, transportation to ED, or hospitalization after ED attendance) was examined. The secondary outcomes were also compared between patients who did and did not meet all of the available criteria on the relevant checklist. To assess the performance of the checklists against the destination determined by paramedics, the secondary outcomes were compared between those who were identified as being the suitable for discharge at the scene by the checklist but not actually discharged by paramedics (checklist-fulfilled-but-not-discharged) group and those who were not identified as being the suitable for discharge by the checklist but were actually discharged by paramedics (checklist-not-fulfilled-but-discharged) group. Patients whose destination identified by the checklist agreed with paramedics’ determined destination were not considered in this comparison in order to make the two groups mutually independent for statistical testing.
Sensitivity analyses were conducted to assess the impact of the excluded criterion in the post-ictal checklist (Criterion 4: “No evidence of the patient feeling unwell prior to the seizure”) on the results. Because there is no reported probability of meeting this criterion, three different probabilities (25%, 50%, and 75%) were used. The numbers of patients who did and did not meet all criteria (including Criterion 4) in the post-ictal checklist, and the occurrence of subsequent ambulance request, ED presentation, and hospitalization within 24 hours after the index event, were estimated. The Monte Carlo method with 1,000 iterations was used to compute 95% CIs for the estimates. Sensitivity analysis for the excluded Criterion 10 (“No irritability or altered behavior”) in the hypoglycemic checklist was not performed. It was considered that if Criterion 5 (“Glasgow Coma Scale =15”) of the hypoglycemic checklist was met, then Criterion 10 (“No irritability or altered behavior”) would most likely also be met. Therefore, the results would not change significantly if Criterion 10 was included.
A chi-square test or Fisher’s exact test was used for independent categorical variables, McNemar’s test for dependent categorical variables, and a student’s t-test for quantitative variables at the five percent significance level. All statistical analyses were performed with the IBM SPSS version 21.0 (IBM; Armonk, New York USA). Ethics approval for this study was granted by the Curtin University Human Research Ethics Committee (Bentley, Western Australia, Australia; approval number: HR127/2013).
This study was conducted as part of a project supported by the Western Australian Department of Health Targeted Research Fund (Perth, Western Australia, Australia; reference number: F-AA-00788). The complete study protocol has been published. 17
Results
Post-ictal State
In total, 629 post-ictal, adult patients were attended by SJA-WA paramedics during the study period and met the inclusion criteria (Figure 1). Characteristics of the 629 post-ictal patients are shown in Table 2. The mean age was 40.6 years, with more males (63.6%), and 91 patients (14.5%) met all available checklist criteria (Table 2). Eight out of 10 criteria were met by the majority of patients, but two (known epileptic and attendance of a care giver) were met by only one-half of the patients (Table 2).
Figure 1 Flow Diagram of Post-ictal and Hypoglycaemic Patients Included in this Study. Abbreviations: ED, emergency department; SCP, self-care pathway.
Table 2 Patient Characteristics by Clinical Sub-group
Abbreviation: BSL, blood sugar level.
Among patients who met all post-ictal checklist criteria, 25 patients (27.5%; 95% CI, 18.3%-36.6%) actually were discharged at the scene (Figure 1), and 21 patients (23.1%; 95% CI, 14.4%-31.8%) were admitted to hospital after ED (Table 3). The SCP assessment checklist identified a similar proportion of patients suitable for discharge at the scene to that determined by paramedics (14.5% vs 14.0%; P=.86 [McNemar’s test]). Patients who met all criteria were significantly younger than those who did not, and there was a significant difference in the distribution of triage levels between the two groups (Table 3). Patients who met all criteria were significantly more likely to be discharged at the scene but also subsequently more likely to request a further ambulance service and re-present to ED with a seizure-related diagnosis within three days after the index event (Table 3). There was no difference in the occurrence of any subsequent events at seven days after the index event between those who did and did not meet the checklist criteria; no deaths were recorded (Table 3).
Table 3 Self-care Pathway Assessment Checklist for Post-ictal State (N=629)
Abbreviation: ED, emergency department.
There were 66 patients in the checklist-fulfilled-but-not-discharged group (those who were identified as being suitable for discharge at the scene by the checklist but not actually discharged by paramedics) and 63 patients in the checklist-not-fulfilled-but-discharged group (those who were not identified as being suitable for discharge by the checklist but were actually discharged by paramedics; Figure 1, Table 4). The checklist-fulfilled-but-not-discharged group included more patients with a triage level of three than the checklist-not-fulfilled-but-discharged group, although age and sex distributions were not significantly different (Table 4). There was no significant difference in any subsequent events (Table 4).
Table 4 Comparison between Checklist-fulfilled-but-not-discharged and Checklist-not-fulfilled-but-discharged Groups in Post-ictal Cohort (N=129)
Abbreviation: ED, emergency department.
Hypoglycemia
In total, 609 hypoglycemic, adult patients were attended by SJA-WA paramedics during the study period and met the inclusion criteria (Figure 1). Mean age was 49.6 years, with more males (54.7%), and 37 patients (6.1%) met all criteria in the checklist (Table 2). The majority of patients did not meet three criteria required for discharge at the scene, namely: increased BSL, known diabetic, and BSL >5 mmol/L (Table 2).
Among patients who met all checklist criteria for hypoglycemia, 18 patients (48.6%; 95% CI, 32.5%-64.8%) actually were discharged at the scene (Figure 1), and seven patients (18.9%; 95% CI, 6.3%-31.5%) were admitted to hospital after attending ED (Table 5). Significantly fewer patients were identified as being suitable for discharge at the scene by the checklist than actually were discharged by paramedics (6.1% vs 28.7%; P<.001 [McNemar’s test]). While patients who met all criteria were significantly older than those who did not, the distributions of sex and triage level were not significantly different between the two groups (Table 5). Patients who met all criteria were significantly more likely to be discharged at the scene than those who did not, but there was no significant difference in the occurrence of subsequent events between the two groups (Table 5). Four patients (0.66%) died within three days after the index event, none of which were attributed directly to hypoglycemia.
Table 5 Self-care Pathway Assessment Checklist for Hypoglycemia (N=609)
Abbreviations: BSL, blood sugar level; DM, diabetes mellitus; ED, emergency department.
There were 19 patients in the checklist-fulfilled-but-not-discharged group and 156 patients in the checklist-not-fulfilled-but-discharged group (Figure 1, Table 6). The proportion of patients with a triage level of three was higher in the checklist-fulfilled-but-not-discharged group than the checklist-not-fulfilled-but-discharged group, although age and sex distributions were not significantly different (Table 6). No significant difference was observed in any subsequent events (Table 6).
Table 6 Comparison between Checklist-fulfilled-but-not-discharged and Checklist-not-fulfilled-but-discharged Groups in Hypoglycemia Cohort (N=175)
Abbreviations: BSL, blood sugar level; DM, diabetes mellitus; ED, emergency department.
Sensitivity Analyses
The sensitivity analyses showed significant change in the number of patients who did and did not fulfill the post-ictal checklist (Graph 1 in Figure 2), but no significant changes were observed in the occurrence of the subsequent events (ambulance request, ED presentation, and hospitalization; Graphs 2-4 in Figure 2).
Figure 2 Sensitivity Analyses. Inclusion of Criterion 4 in the post-ictal checklist (“No evidence of the patient feeling unwell prior to the seizure”) significantly reduced the number of patients who met all criteria in the checklist and increased the number of patients who did not meet. The occurrences of subsequent events (ambulance request, ED presentation, and hospitalization) did not significantly change. Abbreviation: ED, emergency department.
Discussion
The predictive performance of the UK North West Ambulance Service SCP checklists was assessed retrospectively for two groups of prehospital patients, namely those with hypoglycemia and those in a post-ictal state. Among those patients who met all criteria in the respective checklist, less than one-half of the hypoglycemic patients and less than one-third of the post-ictal patients actually were discharged at the scene by paramedics; approximately 20.0% of patients were admitted after ED assessment. There was no difference in the occurrence of subsequent events between patients meeting the hypoglycemia checklist criteria and those who did not, but patients meeting the post-ictal checklist showed significantly higher occurrence of subsequent events within three days. Paramedics discharged more hypoglycemic patients than the checklist would have without a significant difference in the occurrence of subsequent events.
The SCP assessment checklists did not identify accurately patients suitable for discharge at the scene. Amongst patients identified as suitable for discharge at the scene by either of the checklists, approximately 20.0% actually were admitted after assessment at ED. The addition of a new criterion may improve the performance of the checklists. For instance, two reviews recommended that a hypoglycemic patient on oral hypoglycemic agents should be transported to hospital because of the risk of prolonged or repeated hypoglycemia events.Reference Finn, Fatovich and Arendts 18 , Reference Fitzpatrick and Duncan 19 However, only a small proportion of patients (6.0% for hypoglycemia) met all of the current checklist criteria, and the addition of another criterion would likely further reduce this proportion, probably making the checklist even less useful.
The performance of the two checklists was no better than paramedics’ decision as to whether a patient could be discharged at the scene when compared against admission or subsequent events rates. The checklists’ poor performance might be explained by the criteria included in the checklists being too restrictive. For example, while all hypoglycemic patients in the checklist-fulfilled-but-not-discharged group met the sixth criterion of the hypoglycemia checklist (6. “BSL >5.0 mmol/L”), only 37 of 156 patients (24.0%) in the checklist-not-fulfilled-but-discharged group met this criterion. Although most patients in the checklist-not-fulfilled-but-discharged group did not fulfill the hypoglycemia checklist, no significant difference was found in the occurrence of subsequent events between the checklist-fulfilled-but-not-discharged vs checklist-not-fulfilled-but-discharged groups. Further, paramedics discharged significantly more patients at the scene than the hypoglycemia checklist would have recommended without increasing subsequent events. The use of the hypoglycemia checklist might lead to an increase in ED attendances because five times fewer patients were identified as suitable for discharge at the scene by the checklist than actual discharges.
The UK North West Ambulance Service SCP assessment checklists for prehospital assessment of post-ictal state and hypoglycemia are similar to other existing guidelines. The Joint Royal Colleges Ambulance Liaison Committee (JRCALC; London UK) guidelines for post-ictal patients states that they may be discharged at the scene “if they are fully recovered and not at risk, and in the care of a responsible adult.”Reference Roberts and Smith 20 The Epilepsy Foundation of USA (Landover, Maryland USA) guideline includes similar criteria to the SCP assessment checklist for the post-ictal state. 21 The JRCALC guideline for hypoglycemia also suggests that a hypoglycemic patient may be left at the scene if a patient is completely alert after treatment, has a BSL >5.0 mmol/L, and is under the care of a responsible adult. 22 These guidelines are based on expert opinion only but have not been validated. The results of this study showed that 72.0% of patients who met the entire checklist criteria for post-ictal discharge actually were transported to ED, of whom 32.0% were then admitted to hospital. Further, post-ictal patients who met post-ictal checklist criteria were significantly more likely to experience subsequent events (ie, ambulance request and ED attendance with a seizure-related diagnosis) within three days after the index event than those who did not. The use of such guidelines could delay initiation of hospital care for some post-ictal patients.
Subsequent requests for emergency health care services may not be an appropriate measure to assess the hypoglycemia SCP checklist. Previous studies have reported that among hypoglycemic patients who were discharged at the scene, 2.7% to 9.0% of the patients subsequently requested an ambulance. 23 - Reference Mechem, Kreshak, Barger and Shofer 27 Even if patients were treated and discharged at the scene by a physician, 7.8% of patients were reported as seeking secondary care within 72 hours.Reference Socransky, Pirrallo and Rubin 28 In addition, there was no significant difference in the occurrence of subsequent events between those who met the SCP checklist criteria and those who did not. Hence, the subsequent request for emergency health care services appeared to be inevitable for some hypoglycemic patients.
Limitations
This study has limitations related to its retrospective design. One criterion from each checklist was excluded because the information for these was not available in the ambulance data. If the excluded criteria are important predictors, the checklists might have had better performance for accurately identifying patients suitable for discharge at the scene but would further reduce the number of patients who met all criteria as observed in the sensitivity analyses (Graph 1 in Figure 2). The sample sizes for the comparisons between the checklist-fulfilled-but-not-discharged and checklist-not-fulfilled-but-discharged groups may not be sufficient to identify important differences. A prospective study with a larger sample size is required for more accurate validation of the checklists.
Conclusions
The UK North West Ambulance Service SCP assessment checklists for post-ictal and hypoglycemia did not identify accurately patients who could be left at the scene within the Emergency Medical Service. Both checklists showed similar occurrence of subsequent events to paramedics’ decision, but the hypoglycemia checklist identified fewer patients who could be discharged at the scene than paramedics actually discharged. Patients who met all criteria of the post-ictal checklist made more subsequent health service requests within three days after the index event than those who did not. Therefore, the sole use of the checklists actually may increase transportations to ED, delay the initiation of appropriate management, and possibly jeopardize the patients’ well-being.
Acknowledgement
Professor Ian Jacobs, who was the chief investigator of this project and Clinical Services Director of the St John Ambulance Western Australia, died before this manuscript was submitted. He was involved in conceiving the study and obtaining funding. His invaluable support and contribution to this study is acknowledged. The authors also thank Mr. Mark Newton, North West Ambulance Service, UK for providing us with the self-care pathway assessment checklists.
