INTRODUCTION
Patients with advanced disease often suffer from psychosocial distress (Block, Reference Block2001; Gotze et al., Reference Gotze, Brahler and Gansera2014; Gruneir et al., Reference Gruneir, Smith and Hirdes2005; Mazzocato et al., Reference Mazzocato, Stiefel and de Jonge2000). The reported prevalence of distress varies between 20 and 30% depending on what measurement instrument is employed (Thekkumpurath et al., Reference Thekkumpurath, Venkateswaran and Kumar2008). Psychological distress tends to be underdiagnosed and undertreated in palliative care settings (Thekkumpurath et al., Reference Thekkumpurath, Venkateswaran and Kumar2008). Ultrashort screening tools like the Distress Thermometer (DT) were developed to facilitate its detection (Mitchell, Reference Mitchell2007).
Several studies have examined the validity of screening tools in patients with early-stage disease. However, research and clinical guidance are lacking for treatment of patients with advanced disease (Thekkumpurath et al., Reference Thekkumpurath, Venkateswaran and Kumar2009). The German Association for Palliative Medicine has recommended the Distress Thermometer (DT) and the Hospital Anxiety and Depression Scale (HADS) for basic assessment and identification of psychosocial distress in the context of palliative care (Herschbach & Weis, Reference Herschbach and Weis2008). However, validation of a German version of these instruments in a palliative home care setting has not been yet been accomplished. Validation in a palliative care setting is important, because the properties of an instrument as a screening tool depend partly on the population in which it is investigated (Rouse, Reference Rouse2007). In a former study (Küttner et al., Reference Küttner, Wueller and Pastrana2016), we demonstrated that the DT has a high degree of sensitivity and acceptability in persons at the end of life.
The objective of the current paper was to examine the accuracy of the DT and identify an optimal cutoff score for its German version (Mehnert et al., Reference Mehnert, Müller and Lehmann2006) compared to the HADS in home care patients with palliative care needs.
METHODS
A 15-month cross-sectional study was begun in September of 2013 in Aachen (Germany) with consecutive patients cared for by a palliative home care service. The survey was conducted during the first contact. Patients were excluded if they were under 18 years of age, mentally or physically unable to complete the assessment questionnaires as judged by their healthcare workers, or unable to understand the German language.
Measures
Sociodemographic and medical data were collected. The variables assessed were sex; age; marital status; living situation (alone, with family, or in an institution); care situation; diagnosis; functional status (Karnofsky Performance Status [KPS] score reported by health worker); duration of illness (time from diagnosis to first contact); outcome (death or discharge); and, for patients who died, the total number of days in the study was recorded.
The Distress Thermometer is a single-item self-report measure of distress (Roth et al., Reference Roth, Kornblith and Batel-Copel1998). It was developed as an ultrashort screening tool for patients with a cancer diagnosis by the National Comprehensive Cancer Network (NCCN) (Holland et al., Reference Holland, Anderson and Breitbart2013; Roth et al., Reference Roth, Kornblith and Batel-Copel1998) and has been validated and employed in different settings and patient populations (Donovan et al., Reference Donovan, Grassi and McGinty2014; Ma et al., Reference Ma, Zhang and Zhong2014), including palliative care settings (Gessler et al., Reference Gessler, Low and Daniells2008; Thekkumpurath et al., Reference Thekkumpurath, Venkateswaran and Kumar2009). It has an 11-point range, with endpoints from 0 (“no distress”) to 10 (“extreme distress”). Respondents were instructed to circle the number (0–10) that best described how distressed they were during the previous week. According to the NCCN guidelines, a score of 4 or higher on the rating scale is defined as “significant distress” and indicates the need for professional support (Holland et al., Reference Holland, Anderson and Breitbart2013). On the associated problem list, patients identify the source of their distress during the previous week from five categories (emotional, familial, practical, spiritual, and physical).
The HADS is a 14-item self-rated scale with two 7-item subscales: depression and anxiety (Zigmond & Snaith, Reference Zigmond and Snaith1983). The cutoff scores used for the HADS are HADStotal ≥ 15, HADSanxiety ≥ 8, and HADSdepression ≥ 8 (Herrmann, Reference Herrmann1997). The accuracy of the single-item DT has been compared with HADStotal as the standard measurement. The HADS has been validated in patients with palliative care needs (Mitchell et al., Reference Mitchell, Meader and Symonds2010). Validation of the DT versus HADS has been conducted several times (Gessler et al., Reference Gessler, Low and Daniells2008; Roth et al., Reference Roth, Kornblith and Batel-Copel1998; Thalén-Lindstrom et al., Reference Thalén-Lindstrom, Larsson and Hellbom2013). We employed the same approach in a palliative home care setting.
Sample Size Calculation
The sample size calculation showed that 62 patients were required to ensure that, to achieve 95% coverage probability, sensitivity and specificity equal to 80% with a tolerance of 10% were necessary (nQuery Advisor® 7.0). Sensitivity and specificity of 80% are considered to be indicative of a valid diagnostic measure.
Statistical Analysis
The data were anonymized and collected in a digital database. Statistical analysis was conducted using the Statistical Package for the Social Sciences for Windows (v. 21). A descriptive analysis of the demographic and clinical parameters of patients who completed the study was conducted. The linear relationship of DT score with HADStotal score was investigated using Pearson's correlation coefficient (ρ). In addition, we analyzed the diagnostic quality of the DT using receiver-operating characteristic (ROC) curve analyses. The optimal cutoff score was determined according to the Youden criterion (maximized Y = Se + Sp – 1), also considering that a high sensitivity was important due to the increased risk of death associated with a high level of distress. The accuracy properties of sensitivity (Se), specificity (Sp), positive predictive value (PPV), negative predictive value (NPV), and positive and negative likelihood ratios (+LR and –LR) were calculated for each cutoff of the DT score versus HADStotal.
The ethics committee of the Medical Faculty of RWTH University Aachen approved our study.
RESULTS
During the study period, a total of 286 new individuals were registered for home care services. Of these, 138 were excluded from the study (Figure 1), and 103 of the remaining 148 eligible patients completed the questionnaires (DT and HADS). However, 14 HADS questionnaires were not evaluable, because some questions were disregarded, which left a total response rate of 59.7% (n = 89) (Figure 1). Those who did not complete the HADS questionnaire had higher DT values (t test, p = 0.029). No other statistically significant differences were found between participants and nonparticipants with regard to KPS score, age, sex, or polypathology. The following analysis includes these 89 subjects.
Fig. 1. Flow diagram of participant recruitment.
Most participants were women (55.1%, n = 49). The mean age of patients was 67 years (range = 45–89). The majority of participants were married or living in a formal relationship (65.2%, n = 58). Some 73% of participants lived at home and were mainly cared for by relatives (57.3%, n = 51). The main diagnosis was cancer (92.1%, n = 82), mostly located in the digestive system (29.3%, n = 24), reproductive organs (26.8%, n = 22), and respiratory tract (25.6%, n = 21). Metastases were registered in 60.7% (n = 54) of oncological diagnoses. Participants had an average of four different diagnoses. The mean KPS score was 50.8 (±15.9). For 57.3% (n = 51) of participants, this index was reported to be between 50 and 70, indicating severely impaired functional performance.
The average time between receiving the diagnosis and their first contact with the home care team was 29±45 months (range = 25 days to 25 years). Some 86% of patients died within 10.2 (±12.3) weeks, and 13.6% (n = 12) were discharged (Table 1).
Table 1. Sociodemographic and medical characteristics of the sample
SD = standard deviation.
Prevalence of Psychological Distress
The mean DT score was 6.3 (±2.3), with a range of 0 to 10. Using the DT, 89% (n = 79) of participants scored above the DT cutoff score (≥4). The mean HADStotal score was 17.9 (±7.8), with a range of 1 to 35. Using the HADS, 64% (n = 57) of participants reported distress (HADStotal ≥ 15).
Patients with a DT score ≥4, indicating significant distress according to NCCN guidelines, had a significantly higher probability of dying within 1 to 10 weeks after completing the DT questionnaire (χ2, p < 0.0001).
No statistically significant association was found between HADStotal or DT scores and sociodemographic or medical variables.
Comparison of DT and HADS
The Pearson's correlation between the DT and HADStotal scores was moderately positive (ρ = 0.522, p < 0.0001). Figure 2 depicts a scatterplot that displays DT-versus-HADStotal scores.
Fig. 2. Scatterplot displaying DT score versus HADStotal score.
Overall, DT rating was significantly higher for patients who scored above the recommended HADStotal cutoff of 15 points (t test, p = 0.002) than for patients who scored below the cutoff (HADStotal < 15). The area under the ROC curve (Figure 3) yielded an area under the curve (AUC) of 70% (confidence interval = 0.58–0.82). In other words, the probability is that a randomly selected person defined as a case by HADStotal will score higher on the DT than a randomly selected person defined as a “non-case” (Table 2).
Fig. 3. Receiver operating characteristics curve of DT score versus HADStotal score.
Table 2. Accuracy measure for DT scores according to HADStotal
PPV = positive predictive value; NPV = negative predictive value; –LR = negative likelihood ratio; +LR = positive likelihood ratio.
With a DT cutoff score ≥4, sensitivity was 94.7% and specificity 21.9%, with a PPV of 70% and an NPV of 68.4%, and with 88.8% of participants identified as being distressed. Of those screened positive by the DT, 87.5% were false positives, and, of those screened negatively by the DT, 1.8% were false negatives. The Youden optimal cutoff score on the DT within our study population was identified to be 5, which yielded a similar sensitivity as a DT score of 4 (Se = 93.0%) and a 12%-increased specificity of 34.4%.
Comparison of DT and HADS subscales
On the HADS subscales, 51.7% of participants reported anxiety (HADSanxiety ≥ 8), and 73% reported depression (HADSdepression ≥ 8). The mean HADSdepression score was 10.10 (±4.7), while the mean HADSanxiety score was 7.8 (±4.7). The value of Pearson's correlation coefficient between DT scores and HADSanxiety or HADSdepression was moderately positive, being higher for HADSanxiety (ρ = 4.06, p < 0.0001) than for HADSdepression (ρ = 3.62, p < 0.0001).
Patients with HADSdepression scores ≥8 achieved significantly higher DT scores than those who scored below 8 (similar for HADSanxiety scores ≥8; t test, p < 0.0001 for both). However, no statistically significant difference was found for the sociodemographic and medical variables.
Contingency analysis between being distressed (DT ≥ 4) and testing positive for anxiety (HADSanxiety≥ 8) or depression (HADSdepression ≥ 8) was not statistically significantly coherent (χ2). Contingency analysis between being distressed (DT ≥ 5) and testing positive for anxiety (HADSanxiety ≥ 8) was statistically significantly coherent (χ2, p = 0.003). Depression (HADSdepression ≥ 8) was not associated with being distressed (DT ≥ 4).
Establishment of a DT Cutoff Score
A sensitivity and specificity analysis suggested that a HADStotal ≥ 5 cutoff score was Youden optimal and achieved a sensitivity of 93.0% and specificity of 34.4%, as well as a PPV of 73.3% and NPV of 71.6%. In contrast to the ≥4 cutoff, the specificity and predictive values were increased, while sensitivity was maintained. Although cutoff scores of 4 and 5 suffer from low specificity, resulting in good negative and bad positive likelihood ratios (–LR = 0.24 resp. 0.20 < 0.3; +LR = 1.21 resp. 1.42 < 3), a further increase in cutoff score dropped sensitivity by 20%, in exchange for a 30% increase in specificity. Since we were interested in a sensitive cutoff score, the Youden optimal score was identified as the best choice for the population at hand.
DISCUSSION
The emphasis of our study was placed on examining the accuracy of the DT in identifying distress in persons with palliative care needs in Germany, with the help of the HADS, in order to validate the DT as a palliative screening tool for Germany and to identify an optimal cutoff score for the DT.
High psychosocial distress in palliative patients has been described in the literature (Neuwohner et al., Reference Neuwöhner, Lindena and Bausewein2011; Thekkumpurath et al., Reference Thekkumpurath, Venkateswaran and Kumar2008). Some 84% (n = 75) of all palliative care patients who participated in our study showed clinically significant psychosocial distress as measured by the DT according to the DT ≥ 5 cutoff criterion, and 64% (n = 57) according to the HADStotal score. Distress scores were higher using the DT than the HADS (including the subcomponents). The AUC was 0.7, which indicated a fair accuracy of the DT in our population.
The ideal cutoff for the DT in palliative care patients is a matter of controversy (Mitchell, Reference Mitchell2007). The importance of validating the DT for special populations has already been established, and an adjustment is required (Gunnarsdottir et al., Reference Gunnarsdottir, Thorvaldsdottir and Fridriksdottir2012). In a systematic review, Ma et al. (Reference Ma, Zhang and Zhong2014) recommended a cutoff of 6 in end-of-life situations. We suggest a cutoff of ≥5 for screening and identifying distressed patients in palliative care in Germany based on achieving a sensitivity of 93% along with a reasonable specificity of ~34%. In another palliative patient population in the United Kingdom, Thekkumpurath et al. (Reference Thekkumpurath, Venkateswaran and Kumar2009) achieved similar results (cutoff ≥5). This cutoff showed a PPV of 73.3% and an NPV of 71.6%. These findings indicate that nearly 75% of patients scoring above the cutoff on a screening questionnaire will have diagnosable psychological distress. A lower cutoff score would have greater sensitivity but very low specificity, leading to a high rate of false positives.
The only variable associated with high levels of distress by DT and HADSdepression scores was the occurrence of death within the following weeks. No further sociodemographic or medical variables showed a statistically significant influence on levels of distress. Neither sociodemographic nor medical variables had an impact on levels of distress. Previous studies have reported a positive association between a high distress level on the DT with female gender, younger age, and lower functional status (Waller et al., Reference Waller, Williams and Groff2011). In our setting, distress can be considered a general phenomenon in patients experiencing a palliative situation at home, suggesting that the palliative situation outweighs other possible sociodemographic factors.
CONCLUSIONS
The DT performed satisfactorily compared to the HADS for screening distress in our patient population. A significant proportion of home care patients reported the symptoms of distress.
Due to the high prevalence of distress, we recommend its routine use for screening distressed persons at home with palliative care needs so that adequate support can be provided.
This is particularly important considering that even high-level distress often goes unnoticed by doctors and nursing staff (Sollner et al., Reference Sollner, DeVries and Steixner2001). The findings of our study support the use of the DT for screening persons with palliative care needs and suggest that its use can help create new concepts and structures with which to address the psychosocial needs of home care patients and achieve relief from suffering. The DT should be an integral part of the procedures utilized to detect patients' distress, and it is thus an important tool for effectively targeting adequate support to those in need.
ACKNOWLEDGMENTS
The authors are very grateful to Petra Schaubeck, Bernd Wehbrink, Brigitte Brings, Karin Hack, Karin Schnurbusch, Karin Thielemann, Lothar Rißmayer, Marlies Frohn, and Ute Armbruster for their invaluable support, and to Andrea Peterman-Mayer for her critical input. The authors also wish to thank the NCCN for allowing us to use the DT in our research.
