Significant outcomes
∙ Emotional Thermometers 4 (ET4) could be routinely used in busy neurology clinics to help identify multiple sclerosis (MS) patients suffering from depression, which could transform the under-diagnosis of depression in this group.
∙ ET4 performed well as a rule-out screening tool. Patients scoring above the cut-off of ≥16 will likely need further exploration of depression symptoms.
∙ ET4 performs as well as the Hospital Anxiety and Depression Scale – Depression Subscale, but does not require clinician scoring, and is therefore quicker and easier to use.
Limitations
∙ This was performed in a regional neurology centre MS clinic and so may not be applicable across all other patient populations.
∙ A significant proportion of patients had non-core depressive symptomatology but did not meet criteria for Major Depression.
∙ The Major Depression Inventory used as the gold standard with which to compare screening tools generates a depression diagnosis based solely on self-report, rather than by diagnostic interview.
Introduction
Depression is a very common, serious problem in patients with multiple sclerosis (MS), with a lifetime prevalence of up to 50% in patients attending tertiary neurology clinics (Reference Sadovnick, Remick, Allen, Swartz, Yee, Eisen, Farquhar, Hashimoto, Hooge, Kastrukoff and Morrison1,Reference Minden and Schiffer2). It has an adverse impact on quality of life (Reference D’alisa, Miscio, Baudo, Simone, Tesio and Mauro3), is associated with impaired cognitive performance (Reference Arnett, Higginson, Voss, Wright, Bender, Wurst and Tippin4); reduced concordance with prescribed medication (Reference Mohr, Goodkin, Likosky, Gatto, Baumann and Rudick5); and increased suicide risk (Reference Feinstein6).
Depression in MS is likely to be significantly under-recognised in clinical practice (Reference Marrie, Horwitz, Cutter, Tyry, Campagnolo and Vollmer7) and so a number of depression-screening tools have been evaluated in outpatient settings. Those that have been well validated include the Hospital Anxiety and Depression Scale (HADS) (Reference Mitchell, Baker‐Glenn, Granger and Symonds8), and the Beck Depression Inventory Fast Screen (Reference Mitchell, Baker‐Glenn, Park, Granger and Symonds9). Even these short screening tools take more than 5 min, and require clinician scoring, so can be difficult to incorporate routinely into busy neurology clinics. The Patient Health Questionnaire – 9 (PHQ-9) (Reference Kroenke, Spitzer and Williams10) has also been used in MS depression screening, but use of this scale has been found to result in high false positive rates due to the inclusion of fatigue and cognitive symptoms, which often occur in patients with MS in the absence of depression (Reference Gunzler, Perzynski, Morris, Bermel, Lewis and Miller11).
A very rapid tool based on visual analogue ‘Emotional Thermometers (ET)’ has been proposed as an alternative to these more time-consuming, questionnaire-based screening instruments (Reference Mitchell, Baker‐Glenn, Granger and Symonds8,Reference Mitchell, Baker‐Glenn, Park, Granger and Symonds9). It examines ‘distress’, ‘depression’, ‘anger’ and ‘anxiety’ using four simple thermometers (collectively denoted ‘ET4’). In addition, three additional parameters ‘need for help’, ‘burden’ and ‘duration’ create a seven-domain tool (‘ET7’). These tools were originally developed and evaluated in cancer patients, but they have more recently been successfully applied to other patient groups, including outpatients with epilepsy attending a specialist neurology clinic (Reference Rampling, Mitchell, Von Oertzen, Docker, Jackson, Cock and Agrawal12). The ET4 and ET7 tools incorporate the Distress Thermometer (DT), which has also been evaluated as a single-item, self-report measure of distress which is felt to be comparable to longer measures of psychological distress (Reference Jacobsen, Donovan, Trask, Fleishman, Zabora, Baker and Holland13–Reference Roth, Kornblith, Batel‐Copel, Peabody, Scher and Holland15).
Aims of the study
In this study we aimed to evaluate the diagnostic usefulness of the ET tool in comparison with other validated depression screening tools in the population of patients attending a specialist MS/neuro-inflammation clinic. We hoped to improve the screening for depression in these patients by identifying a very rapid and user-friendly method that could easily be incorporated into everyday clinical practice.
Materials and methods
Patient enrolment
Consecutive patients attending the specialist MS/neuro-inflammation clinic at a regional neurosciences centre were invited to take part in the study. Some were seen by a neurologist while others saw a clinical nurse specialist. Before their appointment participants were given written questionnaires to complete in the waiting-room. The clinician reviewed the completed questionnaires and screening tools as part of their clinical assessment and also recorded additional clinical data regarding each patient using a Clinician Questionnaire (see below).
Records and screening tools
Clinical record sheet
Our clinical record sheet listed demographic variables such as age; gender; employment status; age of disease onset; concurrent antidepressants; and concurrent talking therapy. This information was obtained from routine clinical records.
Major Depression Inventory (MDI)
The World Health Organisation has developed a tool, the MDI, which is a brief self-report questionnaire and diagnostic tool based on the International Classification of Diseases, 10th revision (ICD-10) and Diagnostic and Statistical Manual, 4th edition (DSM-IV) criteria (Reference Cuijpers, Dekker, Noteboom, Smits and Peen16,Reference Bech, Rasmussen, Olsen, Noerholm and Abildgaard17). This study used the MDI as a tool for ICD-10 and DSM-IV diagnosis of depression, providing a gold standard with which to compare the other screening tools.
HADS
We used the version as originally published by Zigmond and Snaith (Reference Zigmond and Snaith18) and previously validated in an MS patient population, by Honarmand and Feinstein (Reference Honarmand and Feinstein19). This is a 14 item self-reported questionnaire screening tool for depression and anxiety. It has seven items each for depression and anxiety, all scored between 0 and 3. It requires clinician scoring after administration. The Hospital Anxiety and Depression Scale – Depression Subscale (HADS-D) was used in this study. The published cut-off score of 8 was used to indicate depression in this study.
ET7
This is a visual analogue tool comprising seven vertical visual analogue ‘thermometers’ graded from 0 to 10 and labelled ‘Anger’, ‘Distress’, ‘Depression’, ‘Anxiety’, ‘Burden’, ‘Duration’ and ‘Need For Help’, as shown in Fig. 1. Patients are asked to mark how they have felt in the last 2 weeks. The first four thermometers comprise the ET4 subset. This does not require any further clinician scoring. The published cut-off score in epilepsy for ET7 is ≥29 (Reference Rampling, Mitchell, Von Oertzen, Docker, Jackson, Cock and Agrawal12). We explored the optimal cut-off for depression in MS in this current study.
Clinician questionnaire
This was completed by the clinician seeing the patient and recorded: clinical diagnosis (including type/stage of MS); whether currently in relapse; details of ongoing disease-modifying treatment; and Extended Disease Severity Scale (EDSS) score which is a widely used disability measure in MS (Reference Kurtzke35).
Data analysis
Microsoft Excel was used for data analysis
The MDI was used to generate ICD-10 and DSM-IV diagnoses of depression. It was used to calculate accuracy parameters for the other screening tools including sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) (using the standard published diagnostic cut-off values for each screening tool). Receiver operating characteristic (ROC) curves were also generated and area under the ROC curve calculated for each screening tool. We used a geometric calculation to calculate the area under the ROC curve.
Power calculation
As this project’s primary aim was to validate a screening tool, sensitivity was prioritised over other parameters in calculating the necessary sample size. The sensitivity of the ET7 tool for detecting depression in a study carried out in a different neurological patient population (those with epilepsy) reported by Rampling et al. (Reference Rampling, Mitchell, Von Oertzen, Docker, Jackson, Cock and Agrawal12), was 0.85, so we used this as an estimate of the test’s likely sensitivity in our population. We carried out a power calculation based on aiming for confidence intervals (CI) of ±0.1 in the sensitivity, an estimated prevalence of depression in the patient population of 30% based on previous studies as reviewed above, and a type 1 error rate of 0.05. We used a standard formula for sample size calculation in diagnostic studies as described by Buderer (Reference Buderer20):
SN is the Sensitivity, W the Maximum acceptable width of CI, P the Prevalence of condition, Z 2 α/2 the percentile of a standard Normal distribution corresponding to the desired type 1 error rate (5%) divided by 2.
Results
Patient characteristics
In total, 190 consecutive patients attending a regional MS clinic were included. Full details of their demographic and basic clinical parameters are shown in Table 1. Over two-thirds of patients were female (71.1%), and the mean age of patients was 44.9 (±12.2 years). The mean age of MS onset was 31.9 (±11.5 years). Over half (53.7%) were not working, and a third (33.2%) were working full time. A majority of patients (72.6%) had relapsing/remitting MS, 15.8% had secondary progressive MS and 7.4% had primary progressive MS. Out of remaining eight patients two had neuromyelitis optica (1.1%), one had vasculitis (0.5%) and five (2.6%) had central neuro-inflammatory conditions of uncertain type. Median EDSS score was 3.5, with an interquartile range of 1.5–6. The vast majority (91.4%) were not in relapse during assessment. More patients were not receiving disease modifying therapy (53.7%) than receiving it (46.3%). The disease modifying treatments used are listed in Table 1.
RRMS, Relapsing remitting multiple sclerosis; SPMS: Secondary progressive multiple sclerosis; PPMS Primary progressive multiple sclerosis; NMO, Neuromyelitis optica; DMT, Disease modifying treatment; EDSS, Extended Disease Severity Scale; IQ, interquartile.
About a third of patients were receiving treatment for depression, at time of assessment. This comprised 47 (25%) who were taking antidepressants, and 9 (4.8%) who were receiving psychological therapy.
Figure 2 shows the profile of disease severity and disability amongst patients with MS included in the study, grouped by type/stage of MS. There was a bimodal distribution of severity, with peaks at EDSS scores of 0.5 and 5.5.
Prevalence of depression using the MDI
In total, 188 patients had adequate MDI data to generate ICD-10 and DSM-IV diagnoses. Of these, 21 (11.2%) met criteria for ICD-10 depression (mild, moderate or severe), and 24 (12.8%) met criteria for DSM-IV Major Depression. As the DSM-IV was more inclusive in this patient population and the primary aim of this project was to identify screening tools, we used DSM-IV diagnosis generated from the MDI as the gold standard with which to compare the performance of the other screening tools. Using the raw total MDI score, 44 patients (23%) scored 26 or above (the cut-off suggested in the original paper in which the MDI was validated, by Bech et al. (Reference Bech, Rasmussen, Olsen, Noerholm and Abildgaard17)). Of these 44 patients, only 23 (52%) met criteria for DSM-IV Major Depression based on their MDI responses. This suggests that a significant number of patients with MS in this population either had atypical or subclinical depression.
HADS-D subscale
In total, 188 patients had adequate data to calculate a HADS-D score. Of these, 59 (31.4%) scored 8 or above, which is the optimal cut-off for identifying patients who may have depression in an MS population (Reference Honarmand and Feinstein19).
ET and cut-offs
In total, 186 patients completed the first four ET (constituting ET4). The additional three ETs (‘Burden’, ‘Duration’ and ‘Need For Help’) making ET7 were completed by 179 patients. Cut-off of ≥4 was used for depression and DTs respectively, ≥16 for ET4, and ≥29 for the full ET7.
Comparison of performance of different screening tools
Table 2 compares the cut-offs, sensitivity, specificity, PPV, NPV and the area under the ROC curve for the different screening tools, relative to MDI-derived diagnosis of DSM-IV Major Depression. The ET for depression and distress were analysed individually as well as forming part of the ET4 and ET7 tools. The sensitivity for the ET4 was 0.91 (0.80–1), and specificity was 0.72 (0.65–0.79). This was similar to ET7 which had sensitivity of 0.83 (0.67–0.98) and specificity of 0.67 (0.59–0.74). Both compared well with HADS-D which was the best performing screening tool: it had sensitivity of 0.96 (0.87–1) and specificity of 0.77 (0.71–0.84). NPVs were very similar across all screening methods used: HADS-D 0.99, distress ET 0.98, depression ET 0.96, ET4 0.98, ET7 0.96. PPVs ranged from 0.27 for ET7 to 0.37 for HADS-D. Distress ET and depression ET had the same PPV of 0.29, and ET4 performed slightly better at 0.32. The ET4 performance was most similar to HADS-D in all parameters.
HAD S-D, Hospital Anxiety and Depression Scale – Depression Subscale; ET, Emotional Thermometer; CI, confidence interval; PPV=positive predictive value; NPV, negative predictive value; ROC, receiver operating characteristic.
Figure 3 shows the ROC curves for the same screening tools, relative to an MDI diagnosis of Major Depression. Specifically, areas under ROC curves were as follows: HADS-D 0.93; distress ET 0.83; depression ET 0.84; ET4 0.85; and ET7 0.84. As a simple geometric calculation was used to establish the area under ROC curve, this precluded the generation of CI for the area under ROC curves.
Discussion
In this study we have compared the performance of a visual analogue screening tool for depression with established questionnaire-based screening tools, in patients attending an MS clinic.
Our objective was to find a very rapid way to identify patients who need further assessment to exclude or confirm a diagnosis of depression (and conversely those in whom no further assessment is needed). It is therefore the NPV of the screening tool that is most critical: if the result suggests that the patient does not have depression, how likely is this to be correct? On this measure, the ET4 form of the visual analogue scale, and the distress ET used on its own perform very well (NPV 0.98, 95% CI 0.96–1), and are comparable to the established questionnaire based screening tool (the HADS-D (NPV 0.99, 95% CI 0.98–1). In addition, they have an advantage over the HADS-D in a busy clinic, as conclusions can be drawn at a glance, rather than requiring formal scoring.
For both the HADS-D and the ET4, the likelihood of a patient with a positive screening result having DSM-IV Major Depression (as defined by MDI) was around one-third in our study (this is the PPV). In other words, for every three patients identified by the screening tools as requiring further assessment, one will be confirmed as having Major Depression. While patients scoring positive on screening with either HADS-D or ET4 require further assessment to establish whether a diagnosis of depression can be made, those who score below the cut-offs are unlikely to have depression. This can help clinicians to prioritise limited clinical time in a way that most appropriately meets the often complex needs of each patient with MS.
The point prevalence of Major Depression in our study population, as determined using the MDI to generate a DSM-IV diagnosis, was 12.8%. This is lower than expected based on previously published studies from both clinic and community MS patient populations where the prevalence of depression has tended to be between 25% and 50% (Reference Sadovnick, Remick, Allen, Swartz, Yee, Eisen, Farquhar, Hashimoto, Hooge, Kastrukoff and Morrison1,Reference Minden and Schiffer2,Reference Patten, Fridhandler, Beck and Metz21–Reference Chwastiak, Ehde, Gibbons, Sullivan, Bowen and Kraft23). This is especially surprising given the often reported increased prevalence of depression in women, and fact that women made up over 70% of this study population. The lower prevalence of depression in this study may be due to lower level of disability and lower number of people in relapse. However, other factors are likely to be important such as the impact of 25% of patients currently taking antidepressants, atypical presentations of depression in neurological disorders, the limitations in the questionnaire-based tools used in this study, and the potential impact of season and vitamin D status, as discussed below.
Differences between populations may account for some of the variation in prevalence of depression in different studies. A number of variables associated with increased risk of depression in patients with MS have been identified (most notably disease severity), and these may vary between different study populations. For example, in Chwastiak et al.’s large community-based study in 2002 (Reference Chwastiak, Ehde, Gibbons, Sullivan, Bowen and Kraft23), which found a prevalence of 29.1% for moderate or severe depression, only 22.8% of the subjects had an EDSS of 0–4 (corresponding to relatively mild disability), compared with 54.2% in our study. They also found a significantly lower prevalence of moderate or severe depression in the EDSS 0–4 group (<20%), and this may have contributed to the lower overall prevalence that we found. It is important to acknowledge that the lower prevalence of depression in this study may have resulted in a higher than expected NPV for all the screening tools used, compared to a more typical population, with a higher depression prevalence.
Gunzler et al. (Reference Gunzler, Perzynski, Morris, Bermel, Lewis and Miller11) in a sophisticated study in which they modified the PHQ-9 scale to adjust for the particular challenges of an MS patient population, have highlighted that inclusion of fatigue and cognitive impairment can reduce the accuracy of depression screening instruments in this context. However, we would argue that as any diagnosis of depression suggested by a screening tool (even a sophisticated one) should be confirmed on the basis of diagnostic interview and mental state assessment, what is needed in clinical practice is a quick and reliable method to flag up when this is needed. The ET are good candidates for this role, and also do not assess fatigue, energy levels or cognitive symptoms so should avoid this problem.
Screening may of course result in harm from false positives, leading to increased anxiety, and false negatives, potentially leading to diagnoses being missed. Indeed on a population level, some commentaries have concluded that mass depression screening is not cost-effective, may result in resources being diverted away from patients most in need, and go towards identifying minor problems that may not be significant (Reference Gilbody, Sheldon and Wessely24). Nevertheless, given that depression in MS could be associated with poor quality of life and reduced functioning and given that it often remains under-recognised, routine screening depression in chronic neurological patients is likely to be beneficial.
A quarter of patients in our study were currently receiving antidepressants, and nearly 5% were receiving psychological treatment for depression. Use of treatments for depression is not reported in most previously published prevalence studies. Depression treatment may contribute to the low point prevalence of depression in this sample, as treated patients’ self-reported symptoms may have improved such that they no longer meet diagnostic threshold.
In previously published studies, a variety of different methods for diagnosis, and definitions of depression have been used, potentially accounting for large variation. Indeed in the meta-analysis of depressive disorder prevalence studies in MS patients by Boeschoten et al. (Reference Boeschoten, Braamse, Beekman, Cuijpers, van Oppen, Dekker and Uitdehaag25) they found a very large heterogeneity between studies of >98%, making the prevalence figure less reliable. Many diagnostic tools in prevalence studies are based on aggregating severity scores for a wide range of depressive symptoms rather than disorder. In contrast, the MDI aims to diagnose Major Depression based on DSM-IV criteria, rather than providing an overall assessment of the symptoms. It requires the presence of specific core symptoms (depressed mood and/or loss of interest or pleasure) no matter the severity of other non-core symptoms.
Vitamin D status and the impact of season may well have had a role to play in the prevalence of depression. Vitamin D levels have been negatively correlated with depression scores in MS populations (Reference Knippenberg, Bol, Damoiseaux, Hupperts and Smolders26), as they have been in some studies of depression alone (Reference Parker, Brotchie and Graham27). However, vitamin D replacement has not been associated with reduction in depressive symptoms in patients with MS (and without deficiency) beyond placebo (Reference Rolf, Muris, Bol, Damoiseaux, Smolders and Hupperts28). Intriguingly, in one study, it was found that levels of light exposure and not vitamin D3 status were inversely correlated with depressive symptoms (Reference Knippenberg, Damoiseaux, Bol, Hupperts, Taylor, Ponsonby, Dwyer, Simpson and van der Mei29). This research is confounded by the fact that vitamin D3 may also be a negative acute phase reactant (Reference Silva and Furlanetto30).
Limitations
It was beyond the scope of our study for patients to undergo a formal psychiatric assessment to establish whether they met criteria for a diagnosis of depression, and we used the MDI to provide a surrogate for this. This represents an important limitation of the study. The MDI used to generate depression diagnoses, while adhering closely to DSM-IV criteria is nevertheless self-rated and is no substitute for clinical diagnostic interview. In field testing the MDI was found to have a sensitivity of 0.90 and specificity of 0.82, in generating a DSM-IV diagnosis of Major Depression (Reference Bech, Rasmussen, Olsen, Noerholm and Abildgaard17). At the time of the study, an MDI related to the new DSM-5 was not available, however, given the lack of significant difference between DSM-IV and DSM-5 criteria for Major Depression, we do not believe this would make a substantial difference to the prevalence (31) or outcome of this study. Furthermore, there is now a preliminary ICD-11 published, but at the time of writing it is still not clinically used and no depression screening tools directly relating to it have been published (32). All such tools are based on self-reported symptoms while ignoring the individual’s context. If used in isolation, this is liable to lead to over-diagnosis and undermines the validity of psychiatric classifications (Reference Horwitz and Wakefield33). Nevertheless, under-diagnosis of depression in MS patients in everyday clinical practice is a significant problem, and effective use of screening tools such as ET4, if combined with further psychiatric assessment in relevant patients, has the potential to allow many more patients to access the variety of effective treatments for depression that are available.
Many patients in our study reported substantial morbidity related to depressive symptoms without meeting criteria for a diagnosis of Major Depression, due to lack of core symptoms. In clinical practice, identifying and addressing these ‘sub-diagnostic’ symptoms is still useful, particularly as depression may present in an atypical way in the context of a neurological illness such as MS. Indeed, it has been shown that non-core depressive symptoms are very useful in diagnosing depression in neurological conditions, such as epilepsy (Reference Mitchell, Ioannou, Rampling, Sajid, von Oertzen, Cock and Agrawal34). The fact that only 52% of the patients in our study with a total MDI score of 26 or more met criteria for Major Depression, compared with 82% in the original study in which that cut-off was defined in a psychiatric clinic population (Reference Bech, Rasmussen, Olsen, Noerholm and Abildgaard17), would suggest that there is more non-core depressive symptomatology in this MS population. However, clinicians should be aware that a high level of depressive symptomatology does not equate directly to a syndromic diagnosis of Major Depression.
As stated above, 25% of patients were on antidepressants at assessment. These patients were not excluded or analysed separately, as we wanted to retain a representative clinical sample of MS patients.
Seven patients out of 186 did not complete the full 7 ET, and reasons for this were not recorded, but they were completed fully by the other 179 (96%). Patient experience of completing different measures was outside the scope of this study.
As this was a single site study of less than 200 patients this does limit generalisability. However, the study was carried out in the routine clinical practice and included consecutive patients attending MS clinic. Study sample was large enough and was supported by a sample size calculation. Ethnicity and English ability were not recorded, but may be relevant in patients’ understanding of screening questionnaires, however, a visual analogue method is less likely to be affected by English language ability than questionnaire based screening tools.
Summary
Our data show that the visual analogue ET tool can be used as a valid initial screening tool for depression in patients attending an MS clinic. It is not only valid, but extremely easy to use, and rapid. We would suggest that the ET4 (using the cut-off ≥16) is the best performing version of the tool in this population, and is preferable to the longer ET7. Alternatively the DT performs remarkably well in isolation if an extremely rapid tool is desired. The HADS-D also performed well in this role, as has been shown before, but is more time-consuming for patient and clinician.
We would encourage clinicians seeing MS patients to consider which screening tool would fit best with their day-to-day clinical practice, and to use it routinely. A major advantage of ET is their simplicity and brevity (for both patient and clinician), while still providing excellent NPV. We found that patients were able to complete these tools easily in the waiting room before clinic appointments, guided by very brief written instructions. We suggest a larger multi-centre study with diagnostic clinical interview as the gold standard to further evaluate the usefulness of ET in the MS population.
It must be remembered that all screening tools provide an initial screening step, are not without potential harm, and are not a substitute for full psychiatric interview when diagnosing depression. Patients identified by a screening tool should be further assessed for evidence of Major Depression by the MS clinician by asking further questions, and if necessary referred for specialist psychiatric assessment.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/neu.2019.1
Acknowledgements
The authors would like to acknowledge the neurology receptionists and MS nurses at St George’s Hospital outpatients who helped us with the screening process. Authors’ Contributions: A.T., R.V., N.A. and C.B. all contributed to conception, design and data acquisition of the study, as well as being involved in drafting, editing and approving the article for publication. A.T., R.S., N.P., N.A., C.B., R.V. all contributed to data analysis and interpretation. They also were involved in critical revisions of the article, and approval for publication. S.W., C.M., P.J. and A.B. contributed to data acquisition.
Financial Support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Conflicts of Interest
None.
Ethical Standards
The use of screening instruments to screen for depression in neurology outpatient clinics was considered by the South West London Clinical Research Ethics Committee to constitute a clinical service development project and not to require ethics committee approval. Subjects were informed that participation was entirely voluntary, and that their responses to the questionnaires and screening tools would be reviewed by their clinician and form part of their clinical assessment. All data were anonymised in this paper, and all procedures were in compliance with relevant laws and institutional guidelines. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.