Introduction
Delirium, an acute confusional state characterized by disturbed consciousness and cognitive function, is common among patients with advanced cancer (Centeno et al., Reference Centeno, Sanz and Bruera2004; Bush et al., Reference Bush, Lawlor and Ryan2018). It causes distress in patients, families, spouses/caregivers, and nurses (Breitbart et al., Reference Breitbart, Gibson and Tremblay2002; Morita et al., Reference Morita, Hirai and Sakaguchi2004) and is associated with poor clinical outcomes (Witlox et al., Reference Witlox, Eurelings and de Jonghe2010). Although the effectiveness of antipsychotics for delirium remains unclear (Neufeld et al., Reference Neufeld, Yue and Robinson2016; Burry et al., Reference Burry, Mehta and Perreault2018), short-term use of small-doses of antipsychotics may be considered only for patients with severe distress or risk of harming themselves or others [American Geriatrics Society Expert Panel on Postoperative Delirium in Older Adults, 2015; Marcantonio, Reference Marcantonio2017; Bush et al., Reference Bush, Lawlor and Ryan2018; National Institute for Health and Care Excellence (UK), 2019].
However, there is no widely utilized prediction model for the course of delirium in patients with advanced cancer receiving pharmacological interventions. Notably, short-term outcomes of delirium (such as delirium status on day 3) could estimate treatment effectiveness (Tahir et al., Reference Tahir, Eeles and Karapareddy2010) and have been widely used (Elsayem et al., Reference Elsayem, Bush and Munsell2010). If clinicians could predict such a short-term outcome of delirium, they could share the information with medical staff or families and allocate nursing care efficiently.
Decision tree, a machine learning algorithm, has been widely utilized for clinical prediction models (Esteban et al., Reference Esteban, Arostegui and Garcia-Gutierrez2015; Brims et al., Reference Brims, Meniawy and Duffus2016; Goodman et al., Reference Goodman, Lessler and Cosgrove2016). It has high interpretability because of its complete visualization of prediction rules. Although other machine learning models, such as random forest, can partially visualize influences of predictors (Kurisu et al., Reference Kurisu, Yoshiuchi and Ogino2019; Roger et al., Reference Roger, Torlay and Gardette2020; Tamune et al., Reference Tamune, Ukita and Hamamoto2020), this complete visualization is specific to a decision tree and enables clinicians to utilize the model without software.
The importance of observational studies using real-world data (RWD) has been recognized because they could complement data from randomized controlled studies (Ligthelm et al., Reference Ligthelm, Borzì and Gumprecht2007; Blonde et al., Reference Blonde, Khunti and Harris2018). The U.S. Food and Drug Administration has also mentioned that real-world clinical data are important for healthcare decisions (U.S. Food and Drug Administration, 2018). However, studies using large-scale RWD are lacking for delirium management in patients with advanced cancer.
Therefore, the present study aimed to develop a decision tree prediction model for a short-term outcome of delirium in patients with advanced cancer receiving pharmacological interventions using large-scale RWD [data from Japan Pharmacological Audit Study of Safety and Effectiveness in Real-World (Phase-R)].
Methods
Phase-R study
The present study is a secondary analysis of Phase-R, a multicenter and prospective observational study (Okuyama et al., Reference Okuyama, Yoshiuchi and Ogawa2019; Maeda et al., Reference Maeda, Ogawa and Yoshiuchi2020, Reference Maeda, Inoue and Uemura2021; Matsuda et al., Reference Matsuda, Maeda and Morita2020; Uchida et al., Reference Uchida, Morita and Akechi2020). Data were collected at 14 palliative care units certified by the Hospice Palliative Care Japan and 9 psycho-oncology settings of tertiary cancer care hospitals or university hospitals across Japan from September 2015 to May 2016. The psycho-oncology setting refers to consultation or liaison with psychiatrists or psychosomatic physicians for patients with cancer admitted to oncology wards. The ethics committee of Osaka University (approval number: 13295) and the institutional review boards at all sites approved the study protocol. According to the guideline by the Ministry of Health, Labor, and Welfare, the requirement for informed consent was waived because the study collected data from records of usual clinical practice (Ministry of Health, Labor, and Welfare, 2008). We used an opt-out method such that patients and families could refuse to participate in the study.
Inclusion criteria were (a) patients with advanced cancer who were diagnosed with delirium according to the Diagnostic Statistical Manual of Mental Disorders, Fifth Edition by trained palliative care physicians or psycho-oncologists (American Psychiatric Association, 2013), and (b) those who received antipsychotics or trazodone for symptom improvement. Trazodone was included because of frequent prescriptions for delirium in Japan (Wada et al., Reference Wada, Morita and Iwamoto2018). Exclusion criteria were (a) patients with postoperative delirium and (b) those with alcohol or drug withdrawal delirium.
The definition of study outcome and participants
In the Phase-R project, the Japanese version of the Delirium Rating Scale Revised-98 (DRS-R98) was used to evaluate delirium (Kato et al., Reference Kato, Kishi and Okuyama2010). It is a 16-item clinician-rated scale with 13 severity items and 3 diagnostic items. The score for each item ranges between 0 and 3; the maximum total severity score is 39. The severity score of 10 is suggested as the cutoff point for the diagnosis of delirium.
The patients were evaluated by trained palliative care physicians or psycho-oncologists using the DRS-R98 severity items at the beginning of the pharmacological intervention (baseline) and 72 h after the intervention (day 3). We extracted patients’ records with a DRS-R98 severity score of ≥10 at the baseline from the Phase-R database. As the study outcome, we defined remission as a DRS-R98 severity score of <10 on day 3.
Predictor variables
The following variables measured at the baseline were considered as potential predictors and used in the model development: age, Eastern Cooperative Oncology Group Performance Status (Oken et al., Reference Oken, Creech and Tormey1982), primary tumor sites, comorbid diseases (diabetes, dementia, brain tumor, and cerebrovascular diseases), oral intake availability, precipitating factors of delirium (Inouye et al., Reference Inouye, Westendorp and Saczynski2014), drugs for delirium management, delirium subtypes, the baseline DRS-R98 severity score, treatment lines of drugs for delirium management (first-, second-, or third-line), and settings (palliative care or psycho-oncology).
The precipitating factors of delirium were estimated by trained palliative care physicians or psycho-oncologists and included opioids, drugs other than opioids, dehydration, non-respiratory infection, respiratory infection, organic damage to the central nervous system, hypoxia, liver failure, renal failure, hypercalcemia, hyponatremia, disseminated intravascular coagulation, and others. The drugs for delirium management were categorized into five groups: typical antipsychotics, serotonin-dopamine antagonists, multi-acting receptor-targeted antipsychotics, aripiprazole, and trazodone. The subtypes of delirium were determined by the Delirium Motor Subtype Scale (Meagher et al., Reference Meagher, Moran and Raju2008).
Data analysis
We used a t-test (Student's or Welch's) or the Mann–Whitney U test to compare the means of continuous variables (such as age) between the remitters and non-remitters after examining variance homogeneity using the F-test and normality using the Kolmogorov–Smirnov test. In addition, we used Fisher's exact tests or the Chi-squared tests to compare the proportions of categorical variables (such as sex) between the groups.
A decision tree model was developed using the beforementioned outcome and variables. The Gini index was used as the splitting metric. First, we randomly split three-fourths and one-fourth of the data into the training and test datasets, respectively. Next, using the training dataset, we developed a decision tree model. We optimized the decision tree model's maximum depth by calculating the area under the curve (AUC) of the receiver operating characteristic curve using 5-fold cross-validation. Because a deeper decision tree is more difficult to interpret and tends to overfit (Molnar, Reference Molnar2019), we selected the minimum point among ranges in which the decision tree was constructed, and the AUC was saturated. We then calculated the model performance measured by AUC using the independent test dataset. A set of sensitivity and specificity that maximized the Youden index was also quantified. Finally, we visualized the decision tree model developed using the whole dataset.
All analyses were conducted using an open-source software R (version 4.0.4; R Foundation for Statistical Computing, Vienna, Austria, 2021) with the package “rpart” (version 4.1-15) and “pROC” (version 1.17.0.1). A p-value < 0.05 was considered statistically significant.
Results
Patient characteristics
A total of 668 records were included, of which 141 (21.1%) had a DRS-R98 severity score of <10 on day 3. Several variables showed significant differences between the remitters and non-remitters (Table 1).
Table 1. Descriptive data of the study participants
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20240220134524831-0404:S1478951521001565:S1478951521001565_tab1.png?pub-status=live)
a Mann–Whitney U test.
b Chi-squared test.
c Fisher's exact test.
SD, standard deviation; DRS-R98, Delirium Rating Scale Revised-98; SDA, serotonin-dopamine antagonists; MARTA, multi-acting receptor-targeted antipsychotics.
Decision tree model
In the 5-fold cross-validation, the AUC was saturated when the tree's maximum depth was ≥3. The model achieved an average AUC of 0.698 with the parameter set at 3. In the independent test dataset, the model achieved an AUC of 0.718 (95% confidence interval, 0.627–0.810), a sensitivity of 0.605, and a specificity of 0.822.
The model developed using the whole dataset is shown in Figure 1. The overall remission rate was 0.21. The model showed that the baseline DRS-R98 severity score was the most important predictor. Patients with a score of ≥15 and <15 had a remission rate of 0.13 and 0.44, respectively. Hypoxia and dehydration as precipitating factors were the second and third important predictors.
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20240220134524831-0404:S1478951521001565:S1478951521001565_fig1.png?pub-status=live)
Fig. 1. The decision tree model for delirium of patients with advanced cancer who had a baseline Delirium Rating Scale Revised-98 (DRS-R98) severity score of ≥10. Remission is defined as a DRS-R98 severity score of <10 on day 3.
Discussion
In the present study, we developed a decision tree prediction model for the DRS-R98 severity score improvement on day 3 using the RWD of patients with advanced cancer receiving pharmacological interventions. The model achieved moderate prediction accuracy and showed that the baseline DRS-R98 severity score, hypoxia, and dehydration were the important predictive factors, in this order.
Patients with a higher baseline DRS-R98 severity score also had a higher score on day 3. The result is consistent with that of a systematic review of prolonged delirium (Dasgupta and Hillier, Reference Dasgupta and Hillier2010). A score of 15 on the DRS-R98 severity scale might be used as the threshold for distinguishing severe and non-severe delirium among patients with advanced cancer who had a DRS-R98 severity score of ≥10. Because both the original and the Japanese versions of DRS-R98 did not investigate the cutoff score for delirium severity determination (Trzepacz et al., Reference Trzepacz, Mittal and Torres2001; Kato et al., Reference Kato, Kishi and Okuyama2010), this result could be a new finding. However, the DRS-R98 severity scale has been suggested as unsuitable for evaluating end-stage patients’ delirium because of unconsciousness or non-communicativeness (Uchida et al., Reference Uchida, Morita and Akechi2020). This inappropriateness might also affect the association between the baseline severity score and that on day 3.
Hypoxia and dehydration as precipitating factors were the second and third important factors. Because delirium is defined as occurring due to physiological or pharmacological factors (American Psychiatric Association, 2013), the importance of precipitating factors could be biologically plausible. Although hypoxia has been consistently reported to be associated with a poor outcome, the result of dehydration differed among previous studies (Lawlor et al., Reference Lawlor, Gagnon and Mancini2000; Morita et al., Reference Morita, Tei and Tsunoda2001; Matsuda et al., Reference Matsuda, Maeda and Morita2020). Matsuda et al. (Reference Matsuda, Maeda and Morita2020) noted that this difference might be explained by the different definitions of dehydration or differences in the baseline condition of study participants. Further studies are required to validate the influence of precipitating factors on the DRS-R98 score improvement.
Notably, drugs for delirium management did not appear in the decision tree model. The result might imply that drug selection is less critical for the course of delirium in real-world clinical settings.
This visually interpretable prediction model could help clinicians easily predict the DRS-R98 scores on day 3, which would help them share the information with medical staff or families and allocate nursing care efficiently. Additionally, the relative importance of predictor variables shown in the model is a new finding and may be useful for clinicians to manage delirium considering these clinical manifestations. Further studies are warranted to compare the prediction ability of this model with that of experienced clinicians to confirm the usefulness of the model.
This study had several limitations. First, because the database included only patients receiving antipsychotics or trazodone, the prediction model could not be applied to those without pharmacological approaches or those receiving other drugs. Second, there were no operational criteria to determine the precipitating factors of delirium (Matsuda et al., Reference Matsuda, Maeda and Morita2020). Third, the analysis did not include interventions for the precipitating factors. Fourth, because the outcome measurement was performed on day 3, the model cannot predict a longer-term outcome. This limitation of the model may be notable because the DRS-R98 severity score may fluctuate after day 3. Fifth, the AUC of the model was 0.698 for cross-validation and 0.718 for the test dataset, which was a boundary between moderate and low prediction accuracy (Swets, Reference Swets1988). Finally, the model requires the baseline DRS-R98 severity score, limiting the situations of its utilization. The DRS-R98 is widely utilized for clinical trials (Meagher et al., Reference Meagher, McLoughlin and Leonard2013) and is considered useful for assessing the severity (Oh et al., Reference Oh, Fong and Hshieh2017). However, other evaluation tools, such as the Confusion Assessment Method-Severity Scale, might be desirable for severity evaluation in future studies (Oh et al., Reference Oh, Fong and Hshieh2017).
In conclusion, we developed an easy-to-use prediction model for the short-term outcome of delirium in patients with advanced cancer receiving pharmacological interventions. The model suggested that the baseline severity of delirium and precipitating factors of delirium were important for prediction.
Acknowledgments
We would like to thank the collaborators of the Phase-R study group. The collaborators include the following: Hirofumi Abo, M.D. (Rokkou Hospital); Nobuya Akizuki, M.D., Ph.D. (Chiba Cancer Center); Koji Amano, M.D. (Osaka City General Hospital); Daisuke Fujisawa, M.D., Ph.D. (Keio University Hospital); Shingo Hagiwara, M.D. (Tsukuba Medical Center Hospital); Takeshi Hirohashi, M.D. (Eiju General Hospital); Takayuki Hisanaga, M.D. (Tsukuba Medical Center Hospital); Satoshi Inoue, M.D. (Seirei Mikatahara General Hospital); Shinichiro Inoue, M.D. (Okayama University Hospital); Aio Iwata, M.D. (National Cancer Center Hospital East); Akifumi Kumano, M.D. (Rokkou Hospital); Yoshinobu Matsuda, M.D. (National Hospital Organization Kinki-Chuo Chest Medical Center); Takashi Matsui, M.D. (Tochigi Cancer Center); Yoshihisa Matsumoto, M.D., Ph.D. (National Cancer Center Hospital East); Naoki Matsuo, M.D. (Sotoasahikawa Hospital); Kaya Miyajima, M.D., Ph.D. (Keio University Hospital); Ichiro Mori, M.D., Ph.D. (Garcia Hospital); Sachiyo Morita, M.D., Ph.D. (Shiga University of Medical Science Hospital); Hiroyuki Nobata, M.D. (National Cancer Center Hospital East); Takuya Odagiri, M.D. (Komaki City Hospital); Toru Okuyama, M.D., Ph.D. (Nagoya City University Hospital); Akihiro Sakashita, M.D. (Kobe University Graduate School of Medicine); Ken Shimizu, M.D. (National Cancer Center Hospital); Yuki Sumazaki Watanabe, M.D. (National Cancer Center Hospital East); Keita Tagami, M.D. (Tohoku University School of Medicine); Emi Takeuchi, M.A. (Keio University Hospital); Mari Takeuchi, M.D., Ph.D. (Keio University Hospital); Ryohei Tatara, M.D. (Osaka City General Hospital); Akihiro Tokoro, M.D., Ph.D. (National Hospital Organization Kinki-Chuo Chest Medical Center); Megumi Uchida, M.D., Ph.D. (Nagoya City University Hospital); Keiichi Uemura, M.D. (Hokkaido Medical Center); Hiroaki Watanabe, M.D. (Komaki City Hospital); Ritsuko Yabuki, M.D. (Tsukuba Medical Center Hospital); Toshihiro Yamauchi, M.D. (Seirei Mikatahara General Hospital); and Naosuke Yokomichi, M.D. (Seirei Mikatahara General Hospital).
Funding
This work was supported by a Grant-in-Aid for Scientific Research from the Practical Research for Innovative Cancer Control from the Japan Agency for Medical Research and Development (AMED) [grant number 15ck0106059h0002].
Conflict of interest
The authors declare that they have no conflict of interest.