Introduction
In recent years, decision-making has become more complex for healthcare providers and healthcare consumers alike (Chambers, Reference Chambers2017; Epstein & Gramling, Reference Epstein and Gramling2013; Kuziemsky, Reference Kuziemsky2016). This is due to an ever-expanding body of scientific knowledge, combined with increased emphasis on holistic care and patient-centered design of health services. Frequently, decisions involve weighing multiple heterogeneous elements, which can lead to decision fatigue and subsequent susceptibility to systematic errors including implicit bias (Islam et al., Reference Islam, Weir and Del Fiol2014; Kuziemsky, Reference Kuziemsky2016).
An example of this is the triaging of patients to receive specialist palliative care (PC) services. When there is only one bed left in the palliative care unit, or only time for one more home visit, which patient should be attended to first? Is it the patient who is imminently dying or the patient with a pain crisis; the patient with severe anxiety or the patient whose caregiver isn't coping? Patients referred for specialist PC often have disparate needs and goals (Fitzsimons et al., Reference Fitzsimons, Mullan and Wilson2007; Moghaddam et al., Reference Moghaddam, Coxon and Nabarro2016). Furthermore, most triage decisions must be made without direct assessment of the patient but rather are based on information from referring non-PC health professionals or lay caregivers who may under- or overestimate urgency of need. These decisions are not simple and often cause consternation among PC teams as they manage waiting lists (Eagle & de Vries, Reference Eagle and de Vries2005). Meanwhile, a growing body of evidence documents inequity of access to specialist PC, often related to sociodemographic and disease-related factors (Addington-Hall et al., Reference Addington-Hall, Altmann and McCarthy1998; Addington-Hall & Altmann, Reference Addington-Hall and Altmann2000; Grande et al., Reference Grande, Addington-Hall and Todd1998; O'Neill & Marconi, Reference O'Neill and Marconi2001; Walshe et al., Reference Walshe, Todd and Caress2009). Given these factors and with the aim of making resource allocation transparent, efficient, and equitable, an evidence-based systematic approach is needed to guide the complex decision-making involved in PC triage.
Discrete choice experiments (DCEs), a quantitative methodology traditionally used in marketing and economics, are being increasingly recognized as an important healthcare research tool (Clark et al., Reference Clark, Determann and Petrou2014; Farrar et al., Reference Farrar, Ryan and Ross2000; Ryan, Reference Ryan2004; Viney et al., Reference Viney, Lancsar and Louviere2002), and are useful for both exploring and informing complex decision-making. During a DCE, participants are presented with a series of vignettes in pairs or groups, called “choice sets,” and are required to state their preference within each set. The vignettes are described by a finite set of characteristics, or attributes. Response patterns are used to determine how participants’ preferences are influenced by the attributes and which tradeoffs between attributes they are willing to make. Statistical modeling is used to assign weights based on the relative importance of each attribute. Responses are choices rather than opinions, and these choices involve weighing multiple factors simultaneously; hence, DCEs can be used to simulate real-world decision-making.
Several DCEs have been conducted to explore patient and caregiver preferences for cancer care (Casarett et al., Reference Casarett, Fishman and O'Dwyer2008; Herrmann et al., Reference Herrmann, Sanson-Fisher and Hall2018; Kohler et al., Reference Kohler, Lee and Gopal2015; Osoba et al., Reference Osoba, Hsu and Copley-Merriman2006; Mühlbacher et al., Reference Mühlbacher, Lincke and Nübling2008; Wong et al., Reference Wong, Norman and Dunning2014) and in PC specifically (Douglas et al., Reference Douglas, Normand and Higginson2005; Finkelstein et al., Reference Finkelstein, Bilger and Flynn2015, Reference Finkelstein, Malhotra and Chay2016; Gomes et al., Reference Gomes, de Brito and Sarmento2017; Hall et al., Reference Hall, Kenny and Hossain2014; Malhotra et al., Reference Malhotra, Farooqui and Kanesvaran2015; Meads et al., Reference Meads, O'Dwyer and Hulme2017; Molassiotis et al., Reference Molassiotis, Emsley and Ashcroft2012) but none thus far have focused on the views of PC clinicians because they provide complex care within complex health systems. Our investigator team has embarked on a mixed-method program of research ultimately aiming to develop an evidence-based clinical decision-making tool for PC triage. This paper presents a research protocol for the novel application of an international, online DCE—its design, pilot, and planned execution and analysis—as an example of how this sophisticated methodology can be applied to health services research in PC.
Discrete choice experiment design process
The various stages of development of the planned DCE (Figure 1) were undertaken in conformity with international guidelines (Bridges et al., Reference Bridges, Hauber and Marshall2011; Reed Johnson et al., Reference Reed Johnson, Lancsar and Marshall2013). Ethics approval was granted by the St Vincent's Hospital Human Ethics Research Committee [LNR/16/SVHM/42].
Fig. 1. Discrete choice experiment design process.
Identification of attributes of interest
An initial qualitative study was conducted to explore the practices and attitudes of 20 Australian PC clinicians toward triaging PC needs and is reported in detail elsewhere (Russell et al., Reference Russell, Sundararajan and Hennesy-Anderson2018). This study, informed by a literature review, identified six key clinical characteristics or ‘attributes’ that clinicians use to determine urgency of PC needs: (1) physical suffering; (2) psychological suffering; (3) caregiver distress; (4) unmet communication or information needs; (5) discrepancy between care needs and care arrangements; and (6) mismatch between current site of care and desired site of death when imminently dying (Table 1).
Table 1. Attributes and levels used for pilot discrete choice experiment
Creation of pilot DCE
Vignettes were written using the six attributes as descriptors (Figure 2), with care taken to omit confounders such as gender, age, or disease, and to be plausible in all PC settings (inpatient, hospital consultation, and community). For simplicity, pain was chosen as an example of physical suffering, anxiety as an example of psychological suffering, and discussion of prognosis and goals of care as an example of communication and information needs.
Fig. 2. Example choice set from pilot discrete choice experiment.
Levels of intensity were assigned for each attribute (Table 1), using the minimum number of levels required to sufficiently capture the full spectrum of clinically relevant alternatives to increase the design efficiency of the DCE. For example, a patient having “nil” versus “mild” pain was thought unlikely to be a significant differentiating factor in the assessment of urgency; thus, these attribute levels did not warrant separation within the experimental design. Attribute levels were made explicit (e.g., moderate, severe) rather than using clinically authentic but ill-defined terms (i.e., debilitating, niggling, or overwhelming) that would require interpretation and thus potentially confounding results.
The six attributes, four three-level and two two-level (Table 1), gave rise to 324 (3 × 3 × 3 × 3 × 2 × 2) possible unique vignettes and 3242 possible paired choice sets. Inclusion of all these clinical vignettes in the DCE would represent a full factorial design and would enable the estimation of the independent importance of each attribute (main effects) and all correlations between attributes (interactions), but would not be feasible; thus, an efficient orthogonal fractional factorial design was developed based on D-efficiency and using SAS 9.2 (Macro MktEx and ChoicEff). A cognitive burden of 36 vignettes presented in 18 paired-choice sets was hypothesized to be feasible for health professional participants. Participants were not provided an option of skipping any choice sets because this would not reflect real-world clinical practice.
Execution of pilot DCE
PC practitioners who had participated in the earlier qualitative study (Russell et al., Reference Russell, Sundararajan and Hennesy-Anderson2018) were contacted by phone or e-mail and invited to complete the pilot DCE followed by a face-to-face or phone interview to clarify their interpretation of the vignettes and the rationale for their choices. Ten participants were recruited (response rate, 50%), meeting the target sample size of 5 to 15, which has been shown to reveal most critical problems within survey design (Willis, Reference Willis2004). Participants provided written informed consent. Their characteristics are shown in Table 2.
Table 2. Pilot participant characteristics
The cognitive burden of 18 choice sets was acceptable to participants and the median completion time was 14.7 minutes. Interviews revealed that participants sometimes did not select the patient they believed had the most urgent needs if they thought that those needs would be better met by an alternative type of PC service to the one they offered or by referral to an emergency department. Participants also noted that achieving care in the desired site of care was a separate and distinct PC priority from achieving death in the desired site of death, but the two concepts had been conflated as one attribute in the vignettes (“mismatch between current site of care and desired site of death when imminently dying”).
The pilot DCE was also reviewed by two PC physicians from the United Kingdom and two PC physicians from North America to ensure the appropriateness of language and terminology within their differing healthcare systems.
Refinement of final DCE
Based on the findings of the pilot DCE, instructions to participants were expanded to clarify that no alternative health services were available, with the aim of ensuring participants only considered urgency, rather than appropriateness of service type (Figure 3). Additionally the attribute “mismatch between current site of care and desired site of death when imminently dying”’ was separated into two attributes, “mismatch between current site of care and desired site of care” and “imminently dying,” to allow independent analysis of the concepts involved (Table 3). In response to feedback from the international reviewers, minor changes were made to the terminology used in the demographic items preceding the DCE itself.
Fig. 3. Example choice set from final discrete choice experiment.
Table 3. Attributes and levels used for final discrete choice experiment
The final DCE plan therefore incorporates seven attributes, four three-level and three two-level (Table 3), giving rise to 648 (3 × 3 × 3 × 3 × 2 × 2 × 2) possible vignette profiles and 6482 possible choice sets, again making a full factorial design unfeasible. Six pairwise interactions between attributes were hypothesized to be clinically important by investigators (B.R., J.P.) (Table 4); therefore, the number of choice sets needed to be increased to enable statistical estimation of these interactions. SAS 9.2 (Macro MktEx and ChoicEff) was used to develop an efficient orthogonal fractional factorial design that would allow estimation of main effects and the hypothesized interactions. The final DCE will hence consist of 72 choice sets divided into four blocks of 18 choice sets, with each participant randomly allocated to one of the four blocks to maintain acceptable cognitive burden.
Table 4. Hypothesized interactions between attributes
The minimum sample size required for each block was determined using the Orme (Reference Orme2010) equation: n > = 500 × c/(t × a), where t is number of choice sets, a is the number of choices in each choice set, and c is the maximum number of attribute levels. The minimum number of participants in each block was therefore determined to be 42 (n > = (500 × 3)/(18 × 2)), requiring total recruitment of at least 168 participants for the four blocks.
Planned execution of final DCE
The DCE will be open worldwide to doctors, nurses, and allied health professionals who have specialist PC training or who work primarily in PC, in inpatient, hospital consultation, or community PC services. Participants will be required to be proficient in English and have 2 years professional experience in clinical PC.
PC professional organizations internationally (listed in the International Association for Hospice and Palliative Care [2018] Global Directory) will be contacted by phone or e-mail and requested to circulate recruitment invitations to their members and affiliates via e-mail, newsletter, or social media platforms. This will be supplemented by promotion on social media and via personal contacts. An internet address will be provided for potential participants to access further information or to take part. The opening page will include a participant information and consent statement.
The DCE will be an online anonymous questionnaire using the Lime Survey platform and including demographic (age, gender, location) and work-related items (profession, PC setting, years of PC experience, prior consideration of PC triage process) to allow description of the study population and potential analysis of subgroups.
The importance of each attribute and prespecified interactions will be analyzed using a mixed logit model with maximum simulated likelihood (Train, Reference Train2009). All attributes will be assumed to have random coefficients to assess whether each attribute varies among the health professionals in a significant manner. If the interactions do not significantly contribute to the fit of the model, only the main effects model will be presented. Analysis will be performed using user-written package “mixlogit” in STATA 13.1.
Discussion
Decision-making in PC triage is complex, because multifaceted and disparate clinical scenarios compete for finite clinical resources. At a minimum, transparency is required so that rationales for triage decisions can be made explicit and examinable, thus allowing standardization and correction of systematic biases over time. Ideally, efficiency and equity would also be embedded in routine practice, such that the needs of each patient are assessed and compared against those of others in a simple and fair manner. The planned DCE described in this paper allows for consideration of multiple factors simultaneously and thus will be a useful method for exploring how PC clinicians make triage decisions. To the best of our knowledge, this will be the first time that a DCE has been conducted with PC clinicians as participants, but it appears to be a highly appropriate methodology for a discipline in which the tension of providing tailored patient-centered complex care has potential to hamper efforts to improve standardization.
As recommended by current guidelines (Reed Johnson et al., Reference Reed Johnson, Lancsar and Marshall2013), the initial qualitative work (Russell et al., Reference Russell, Sundararajan and Hennesy-Anderson2018) was essential to delineate the factors that clinicians consider important when making PC triage decisions and thus generate the attributes to be tested in the DCE. Without this rigor, key attributes may have been omitted from the DCE, leading to a clinically inauthentic final triage tool and/or trivial attributes that may have been included in the DCE, leading to an unnecessarily large and potentially unfeasible DCE.
The efforts made to carefully craft the language of the DCE vignettes will minimize error because of misinterpretation (Coast et al., Reference Coast, Al-Janabi and Sutton2012) and make the experiment pertinent to all PC settings. Piloting the DCE was crucial, particularly in identifying the problematic nature of combining the concepts of desired site of care and desired site of death. This distinction has been documented in the literature previously (Agar et al., Reference Agar, Currow and Shelby-James2008), but was not made explicit during the qualitative study, yet the interviews revealed how it had potential to confound the DCE results.
An ideal DCE is orthogonal in design, allowing vignettes that combine all of the attributes to be evaluated by study participants. The planned DCE has the recommended maximum of seven attributes, many with only two levels, but because four attributes required three levels, the number of possible vignettes increased exponentially. Hence, a tradeoff had to be made between rigor and feasibility. Here the importance of clinical input was invaluable, to refine and prune levels, query whether each attribute critically captured an independent domain of PC triage, and which interactions between attributes were important to capture within the experimental design. All this needed to be within the maximum acceptable cognitive burden (and potential fatigue) placed upon the study participants, and we are confident this is the case given the pilot data.
DCEs are gaining popularity, but have thus far been underused in PC despite being a discipline in which many decisions and interventions are complex. Although DCEs tend to be internally valid and consistent, external validity may still be a limitation; that is, respondents may make different choices in a real-world situations than the hypothetical scenarios in a DCE (Viney et al., Reference Viney, Lancsar and Louviere2002). DCEs, however, remain an efficient research tool where a “revealed preference” experiment (i.e., observed real-world choices rather than hypothetical choices as in the present “stated preference” DCE) may not be feasible because of logistical factors, as in PC triage. In any case, limitations regarding the study of hypothetical rather than real-world decisions may not be such a concern when participants are experienced healthcare professionals, as evidenced by the previously documented congruence between stated and revealed prescribing preferences of physicians (Mark & Swait, Reference Mark and Swait2004).
In summary, we anticipate the planned DCE will be a rigorous and efficient method by which to inform the development of an evidence-based PC triage tool.
Our future research includes the execution of the planned DCE and the use of the statistical model generated to form a scoring system for items on the triage tool (Table 3), followed by validation of the final tool. It would also be beneficial to conduct a parallel program of research with patients and the public to investigate their priorities for PC triage, and thus develop triage strategies directly informed by those who access, who fund, and who may in future be recipients of, PC services.
Author ORCIDs
Bethany Russell, Jennifer Philip, 0000-0002-3509-0355, 0000-0002-3312-0645.
Acknowledgments
The authors are grateful for the contribution from Professor Anthony Scott. The authors declare no conflict of interest. This work was supported by the Department of Health Victoria; the Bethlehem Griffiths Research Foundation (grant number BGRF1710-1); the St Vincent's Hospital Research Endowment Fund (grant number 81891); and the Australian Government Research Training Program Scholarships.
