BACKGROUND
Delirium, acute brain dysfunction, is recognized as a serious medical problem in the adult critical care population (Barr et al., Reference Barr, Fraser and Puntillo2013). Evaluation and treatment of delirium has only recently garnered attention in the world of pediatrics (Janssen et al., Reference Janssen, Tan and Staal2011; Schieveld et al., Reference Schieveld, Leroy and van Os2007; Silver et al., Reference Silver, Traube and Kearney2012; Smith et al., Reference Smith, Fuchs and Pandharipande2009; Reference Smith, Boyd and Fuchs2011).
The “gold standard” diagnosis for pediatric delirium is an assessment by child psychiatrists using the Diagnostic and Statistical Manual–IV (DSM–IV) criteria. Expert consensus supports presentation of delirium in children over the age of two years as comparable to adult delirium, and clinical diagnosis, based on the DSM–IV criteria, has been found to be valid (Karnik et al., Reference Karnik, Joshi and Paterno2007; Leentjens et al., Reference Leentjens, Schieveld and Leonard2008; Turkel et al., Reference Turkel, Braslow and Tavare2003; Reference Turkel, Trzepacz and Tavare2006). Preverbal children under two years of age and developmentally delayed children may be very difficult to evaluate for alterations in awareness, consciousness, and cognition, leading some to question the validity of diagnosing delirium in this population. At the same time, some report infant presentation of delirium with recognizable deficits in awareness, cognition, and arousal when evaluated within a developmental framework by experienced practitioners (Madden et al., Reference Madden, Turkel and Jacobson2011; Schieveld et al., Reference Schieveld, Staal and Voogd2010; Silver et al., Reference Silver, Kearney and Kutko2010; Turkel et al., Reference Turkel, Jacobson and Tavaré2013). Due to the lack of objective research addressing the consistency of the “gold standard,” especially in infants and children with developmental delays, we conducted a study to test the interrater reliability of child psychiatrists' assessments.
METHODS
Design
Thirty-eight delirium assessments were completed by two psychiatrists, each blinded to the other's diagnosis. They assessed all consented pediatric intensive care unit (PICU) patients present that day as the initial part of a validation study of the Cornell Assessment of Pediatric Delirium (CAPD) (Traube et al., Reference Traube, Silver and Kearney2013). Psychiatrists were also blinded to the nursing CAPD scores. The study was conducted over three weeks during March of 2012 and took place in a 20-bed general PICU in a tertiary-care academic medical center in New York City.
Subjects
All patients were eligible unless they had a sedation score equal to or lower than –3 (deeply sedated or unarousable) on the Richmond Agitation and Sedation Scale (RASS) (Sessler et al., Reference Sessler, Gosnell and Grap2002).
Children of varying developmental abilities were included. They were described as having “significant clinical developmental delay” if developmental problems were the cause of an impairment in a child's age-appropriate ability to communicate (a symptom that could affect psychiatrists' assessment of the child's baseline in relevant symptom domains) just prior to their critical illness.
After informed consent was obtained, two child psychiatrists conducted diagnostic interviews and exams on each subject to evaluate for delirium. Each psychiatrist was blinded as to the other's conclusion. After both assessments were completed, if a child was diagnosed with delirium by either psychiatrist, this was reported to the medical team caring for the child so that appropriate interventions could be made. Individual subjects were assessed up to four times. The study was approved by the institutional review board at Weill Cornell Medical College.
Assessment Measures
Interrater reliability was quantified using Cohen's κ coefficient along with its 95% confidence interval.
Evaluator Training
A two-hour initial training session for the six child psychiatrist evaluators, led by the first author, was completed to establish consistency in concepts and vocabulary among the group. In three subsequent training sessions, “thinking developmentally” about delirium, including keeping normal milestones in mind and a broad range of expectations regarding behavior and cognition for children at different developmental stages during critical illness, was emphasized. Manifestations of alterations in attention, consciousness, and cognition were discussed in order to understand clinical experiences with critically ill children. This framework was discussed in relation to each item on our psychiatric assessment worksheet, which was based on the Delirium Rating Scale (DRS-98) (Trzepacz et al., Reference Trzepacz, Mittal and Torres2001), with the addition of descriptors, expansion of some categories, and addition of an item to denote a change in cognition from baseline.
RESULTS
When a total of 38 assessments were completed, including 17 patients, the psychiatric diagnoses were compared. Our sample included 7 infants under 2 years of age (13/38 assessments, 34%), 4 children aged 2–5 years (11/38 assessments, 29%), 4 children aged 6–12 years (11/38 assessments, 29%), and 2 adolescents aged 13–21 years (3/38 assessments, 8%). Of these, 5 had moderate to severe developmental delay (11/38 assessments, 29%) and 7 were intubated (15/38 assessments, 39%).
Overall (see Table 1), delirium was identified in seven patients, including three infants below two years of age and three patients with moderate to severe developmental delay. Among the three children who underwent multiple assessments and were diagnosed with delirium at least once, two showed a fluctuating delirium course. The child psychiatrists diagnosed delirium in 34% (13/38) of evaluations. The interrater reliability of the 38 separate psychiatric assessments was high, with κ = 0.94 (95% CI = 0.83–1.00).
Table 1. Psychiatrist interrater reliability (Cohen's kappa κ) for pediatric delirium by assessment (N = 38)
Abbreviations: m = months; yr = years; Dx = diagnosis; CI = confidence interval; devel. delay = developmental delay.
* 38 paired assessments were administered across 17 patients, with 65% (11/17) undergoing multiple assessments.
** The discordant assessment occurred in a one-month-old non-intubated patient with age-appropriate development.
When analyzed by age, in children under 2 (13 assessments), Cohen's κ was 0.81 (95% CI = 0.45–1.00). For children over 2 years of age (25 assessments), Cohen's κ was 1.00. Likewise, interrater reliability was high for children with and without developmental delay (κ = 1.00 and 0.90 [95% CI, 0.71–1.00], respectively), and regardless of whether or not they were intubated (κ = 1.00 and 0.89 [95% CI = 0.70–1.00], respectively).
DISCUSSION
Diagnosing delirium in children, particularly infants and children with developmental delay, can be challenging. The ability to diagnose delirium requires recognition of a disturbance in consciousness and cognition (DSM–IV, criteria A and B) with a rapid onset and fluctuating course (criterion C) with a linkage to a physiologic cause (criterion D) (APA, 2000). The list of differential diagnoses includes many presentations in pediatrics and child psychiatry (Smith et al., Reference Smith, Boyd and Fuchs2011) and usually requires input from many observers over a period of time to differentiate delirium from other clinical issues (Esseveld et al., Reference Esseveld, Leroy and Leue2013; Schieveld et al., Reference Schieveld, Leroy and van Os2007; Smith et al., Reference Smith, Fuchs and Pandharipande2009). Delirium has many comorbidities (e.g., pain, premorbid or situational anxiety, behavioral regression) that complicate the clinical picture. However, “thinking developmentally” about infants and children and obtaining specific information about each child's baseline cognition, communication, and behavior make this a valid and useful diagnosis.
Discordant Assessment
The one discordant assessment came very early on in the study and was the case of a one-month-old infant with acute respiratory failure due to respiratory syncytial viral pneumonia. Sleeping excessively, with no differentiation of day and night, she was generally quiet, minimally reactive, and not responding much to her parents. One psychiatrist diagnosed her decreased activity and fluctuating mental status as hypoactive delirium. The second psychiatrist felt that this activity was consistent with a “sick baby” and did not meet the criteria for delirium. By the following day, both psychiatrists independently agreed that she did not meet the criteria for a delirium diagnosis, though each noted in their assessment that the patient may have had “subclinical delirium” and needed to be watched closely. Her deliriogenic medications has been reduced, and within 24 hours her awareness, activity, and reactivity improved. On third assessment, both psychiatrists again agreed that she was not delirious. This case highlights the difficulty that may exist in making a definitive diagnosis of delirium in young infants, hypoactive delirium at any age, and subclinical cases. However, the consideration of a diagnosis of delirium led to a clinical pathway that reduced potentially offending medications, ruled out medical causes of delirium, and implemented positive environmental interventions, all of which may have benefited this child (Schieveld et al., Reference Schieveld, van der Valk and Smeets2009; Smith et al., Reference Smith, Brink and Fuchs2013).
SUMMARY
Although it has been generally accepted that the psychiatric interview is the “gold standard” for diagnosing delirium, reliability for this clinical diagnosis in children has not been previously reported. Given that the diagnosis of delirium in preverbal and developmentally delayed children is challenging, it is reassuring that we were able to show a high interrater reliability for psychiatric diagnosis. With specific attention to normal development in each symptom domain, consulting child psychiatrists and other clinicians can be equipped to diagnose delirium in medically ill children of nearly all developmental ages and trajectories.
