Oppositional defiant disorder (ODD) is defined as a recurrent pattern of defiant, disobedient, and hostile behavior that causes impairment in the child’s personal, social or academic life for at least 6 months, and it is not due to a psychotic or mood disorder episode (American Psychiatric Association, 2000). ODD is one of the most common externalizing disorders in childhood. Along with conduct disorder (CD) and attention/hyperactivity deficit disorder (ADHD), ODD is one of the leading reasons for referral to neuropediatric and psychiatric services (Loeber, Burke, Lahey, Winters, & Zera, Reference Loeber, Burke, Lahey, Winters and Zera2000; Nock, Kazdin, Hiripi, & Kessler, Reference Nock, Kazdin, Hiripi and Kessler2007).
The prevalence rates of ODD range from 2.6% to 15.6% in epidemiological samples, and from 28% to 65% in clinical samples (Boylan, Vaillancourt, Boyle, & Szatmari, Reference Boylan, Vaillancourt, Boyle and Szarmari2007; Ersan, Dogan, Dogan, & Sümer, Reference Ersan, Dogan, Dogan and Sümer2004; Rowe, Maughan, Costello, & Angold, Reference Rowe, Maughan, Costello and Angold2005). The high prevalence rate (11.5 %) reported by Ersan et al. (Reference Ersan, Dogan, Dogan and Sümer2004) is probably the result of using questionnaires versus diagnostic interviews, and the absence of clinician confirmation. On the other hand, a lower ODD prevalence rate (around 3.8%) presented by Emberley and Pelegrina (Reference Emberley and Pelegrina2011) may be a consequence of only including teacher reports, and thus may have underestimated ODD prevalence. In Spain, few studies have been conducted and prevalence rates vary (3.8%–18.5%) depending on whether it is a community or clinical sample, if one or more informants are included, and the age of participants (Emberley & Pelegrina, Reference Emberley and Pelegrina2011; Granero, Ezpeleta, Domenech, & De la Osa, 2008; López-Soler, Castro, Alcántara, Fernández, & López, Reference López-Soler, Castro, Alcántara, Fernández and López2009).
One of the most important factors that influence prevalence rates is the evaluator: teacher, parents, and/or child (Cardo et al., Reference Cardo, Meisel, García-Banda, Palmer, Riutort, Bernad and Servera2009). Different informants offer valuable information about the child´s problems. However, research examining ODD prevalence in children has mainly focused on mother reports. Nevertheless, Owens and Hoza (Reference Owens and Hoza2003) suggest that the presence and absence of ODD may be best identified by teacher ratings.
Many studies have shown low correlation when different informants rate children behavioral problems (Karver, Reference Karver2006). This lack of agreement normally occurs between parents and teachers, in comparison to fathers and mothers that have shown an average agreement level (Achenbach, McConaughy, & Howell, Reference Achenbach, McConaughy and Howell1987). This is probably because father and mother, contrary to teachers, observe the child´s behavior in a similar environment. Home and school environments vary in the extent, activity purpose, and relationship between the child and adult (De los Reyes, Henry, Tolan, & Wakschlag, 2009). Discrepancies between evaluators may be a product of the context in which the rating occurs, but may also be determined by the child´s characteristics, aspects of the evaluators´ perspectives, and measurement error (Drabick, Bubier, Chen, Proce, & Lanza, Reference Drabick, Bubier, Chen, Price and Lanza2011).
Informant disagreements have led to a variety of strategies for addressing parent and teacher symptoms reports (Offord et al., Reference Offord, Boyle, Racine, Szatmari, Fleming, Sanford and Lipman1996). One strategy is to consider information from different informants separately (an informant-specific conceptualization). A second strategy involves cross-informant classification agreement. In this way, informants have to agree on the overall classification of the disorder. These two strategies have been used in the present study. The rationale for choosing these grouping strategies relies on the fact that an informant-specific conceptualization of childhood ODD is suggested as the most adequate approach in several studies (Drabick, Gadow, & Loney, Reference Drabick, Gadow and Loney2007; Munkvold, Lundervold, Lie, & Manger, Reference Munkvold, Lundervold, Lie and Manger2009; Offord et al., Reference Offord, Boyle, Racine, Szatmari, Fleming, Sanford and Lipman1996), and a cross-informant classification agreement method would allow an overview of children who exhibit ODD symptoms in multiple settings (Drabick et al., Reference Drabick, Gadow and Loney2007). A third strategy suggests combining informants based on their joint responses to individual items (“or” rule). In this way, symptoms must be endorsed by either informant. A fourth approach is to combine informants based on individual item agreements (“and” rule). This would involve a cross-informant symptoms agreement (Drabick et al., Reference Drabick, Gadow and Loney2007; Offord et al., Reference Offord, Boyle, Racine, Szatmari, Fleming, Sanford and Lipman1996). In sum, grouping strategy selection should be guided by the question addressed. For instance, the third strategy mentioned would be appropriate to minimize false negatives, whereas to minimize false positives the second and last approach mentioned would be more suitable (Drabick et al., Reference Drabick, Gadow and Loney2007).
In general, parents report higher rates of ODD compared to teachers (Granero et al., Reference Granero, Ezpeleta, Domenech and De la Osa2008; Loeber, Green, Lahey, & Stouthamer-Loeber, Reference Loeber, Green, Lahey and Stouthamer-Loeber1991). Normally, school setting provides greater structure than home, which may contribute to higher levels of ODD symptoms at home than school (Drabick et al., Reference Drabick, Gadow and Loney2007). Moreover, children who exhibit ODD behaviors towards a teacher may be more likely to receive negative responses from peers than children who engage in ODD behaviors towards their parents only (Drabick, Strassberg, & Kees, Reference Drabick, Strassberg and Kees2001). Parents and teachers also show differences when they report ODD in boys and girls. In many studies parents do not report sex differences in ODD (Granero et al., Reference Granero, Ezpeleta, Domenech and De la Osa2008; Lahey et al., Reference Lahey, Schwab-Stone, Goodman, Waldman, Canino, Rathouz and Jensen2000; Michanie, Kunst, & Margulies, Reference Michanie, Kunst and Margulies2007), whereas teachers report more boys with ODD than girls (Loeber et al., Reference Loeber, Green, Lahey and Stouthamer-Loeber1991; Maughan, Rowe, Messer, Goodman, & Meltzer, Reference Maughan, Rowe, Messer, Goodman and Meltzer2004; Serra-Pinheiro, Mattos, & Regalla, Reference Serra-Pinheiro, Mattos and Regalla2008). Nevertheless, several studies suggest that ODD is more common in boys than girls (Boylan et al., Reference Boylan, Vaillancourt, Boyle and Szarmari2007).
The main purpose of this study is to determine the global prevalence rate of ODD in school age children. Other specific objectives are to analyze the variability of the prevalence rates per informant (teachers, fathers, and mothers), according to the sources of information (combination between informants), according to sex and age, and finally to analyze the level of agreement between teachers and parents in the overall detection of the disorder as in each of the items or symptoms assessed. Despite the inconsistencies found in previous studies, our hypothesis are the following: (a) the global prevalence rate of the disorder detected at least by one informant (teachers, fathers, or mothers) will be around 10%, in which parents will show higher rates than teachers; (b) if we considered different sources for the detection of ODD, we will observe that the prevalence rate will decrease as we demand agreement between more evaluators; (c) given that ODD is considered a situational disorder, the degree of agreement will be higher between both parents (since they evaluate the child in the same situation or environment), and lower between teachers and parents (since they evaluate in different environments); (d), the general tendency will be more boys with ODD in comparison to girls, especially in teachers’ reports, and; (e) there will be no differences between academic years or age groups, since all children have similar ages.
Method
Participants
The initial population (29,435 children) included all students between the first and the fourth grades of primary school (6 to 11 years old), in state-funded and partially state-funded schools in the island of Majorca (215, 90% of total schools) during the years 2002 and 2003. This epidemiological study was conducted using a community sample extracted by means of multi-stage stratified sampling according to areas (rural, city and touristy) and schooling (state-funded and partially private).
The sampling parameters were the following:
-
• universe: n (approximate) = 30,000;
-
• stratified sampling by grade (1-4 primary grade) and by clusters (number of classes/grades) proportional to the type of centre and socio-demographic zones;
-
• sample size (n): 1,509. Fraction of total: 1/20;
-
• error type I (Zα = .05) for and expected prevalence rate of 5%; and
-
• precision level: +/- 1.07%.
The original sample consisted of 1,509 children, from which we obtained the scales of teachers, fathers or mothers. The final sample was reduced to 1,295 participants (86% of the initial sample), since we were interested in having subjects who had the three informants` reports available. However, we found no significant differences in socio-demographic factors, sex, grade, mean scores in ODD, or percentage of ODD detection across the three informants, between the selected sample and the excluded participants.
Table 1 shows the subjects distribution of the final sample according to grades and sex, and the age means per grade. There are no significant differences in the distribution of participants according to sex, however, there are significant differences in the mean age inter-groups F(3) = 4303.45, p < .001), in which the Scheffé contrasts present significant differences between the four age groups (p < .05).
Table 1. Sample distribution according to grade and sex, and age per grade
Design
This is an epidemiological and correlational design, with an independent variable, ODD scales for parents and teachers, and different dependent variables: informants (three persons), sources (combination between informants), sex and age (four grades).
Instruments
Oppositional Defiant Disorder Rating Scale based on DSM-IV (ODDRS-IV). This scale is similar to the one proposed by Hommersen, Murray, Ohan, and Johnston (Reference Hommersen, Murray, Ohan and Johnston2006), and to the one used by Molina, Smith, and Pelham (Reference Molina, Smith and Pelham2001) in a previous study. It basically consists in formulating the 8 statements of criterion A of the DSM-IV for the diagnosis of ODD, as questions. Parents and teachers are asked to rate the extent to which each symptom is descriptive of their child/student over the past 6 months on a 4-point rating scale (0 = not at all, 1 = just a little, 2 = pretty much, 3 = very much). A symptom is clinically significant when the score is 2 or higher. A child was considered ODD if for four or more symptoms was clinically significant.
Cronbach’s alpha was calculated to test the scale internal reliability: for teachers the value was .94, for fathers was .83 and for mothers was .85. Teachers scored lower (M = 2.65, SD = 4.34), followed by fathers (M = 4.91, SD = 4.05), and mothers (M = 5.16, SD = 4.20). A repeated measures ANOVA pairwise comparison corrected by Bonferroni showed significant differences between teachers and mothers, t(1292) = −15.84, p < .001, and teachers and fathers, t(1292) = −17.10, p < .001, with moderate effect sizes (.54 and .59 respectively). Comparison between fathers and mothers had lower levels of significance, t(1292) = −3.17, p = .005, with a very low effect size (.06).
Procedure
The sample was selected as follows: from a total of 215 schools, 24 schools were chosen (by stratified and random assignment), 14 were state funded (5 urban, 5 touristy and 4 rural), and 10 were partially state-funded (7 urban, 3 rural) through a proportional cluster random sampling. Two schools refused participation and were substituted by other two, randomly chosen from their category. Fifteen to twenty students were chosen randomly from each of the four grades from the participating schools. An information letter was sent to the parents to obtain written consent (only 2% of subjects had to be substituted). The rating scales were sent to parents (fathers and mothers) and teachers and returned within 10 days. The data of 1,509 subjects was collected, even though the final sample of the present study consisted of 1,295 participants, who had the three informants’ scales.
The presence of ODD was defined according to the conditions established by the DSM-IV. So, if the child had at least 4 of the 8 items of parents and/or teachers ODDRS-IV scales with a score equal or superior to 2 points, then he/she was considered to have the disorder. ODD prevalence is obtained from three informants: teacher, father, and mother. Based upon an informant-specific strategy and a cross-informant classification agreement strategy (see Drabick et al., Reference Drabick, Gadow and Loney2007 and Offord et al., Reference Offord, Boyle, Racine, Szatmari, Fleming, Sanford and Lipman1996), we defined the following sources of detection of ODD: ODD 1 informant, meaning that the criteria has been fulfilled for only one informant (teacher, mother or father), criteria met by both parents (ODD parents), criteria met by the teacher and one of the parents (ODD teacher & father or mother) and criteria met by the three informants (ODD 3 informants). It is important to note that this classification groups were exclusive (a child could only be included in one of the four sources of detection).
The data of the differences in the prevalence rate according to the informant or sources, in relation to sex and grade was analyzed using chi-squared (χ2). The analysis of agreement between informants was done using Kappa coefficient.
Results
The estimated prevalence rate, if we demand that the three informants have to meet ODD criteria, is 1.1% (95% CI [.51, 1.65]). However, if we demand that only one informant has to meet the ODD criteria, the prevalence rate is 9.5% (95% CI [7.88, 11.12]). The source of detection of ODD that bases upon the agreement of both parents presents a prevalence rate of 4.9% (95% CI [3.68, 6.05]), and finally the source that bases upon the agreement between teacher and one of the parents offers the lowest prevalence rate: 0.6% (95% CI [.18, 1.05]). In this case, subjects who were detected by the three informants are not included. If we add the prevalence rates of the four sources of detection, we get 16.1% of children suspects of having ODD, in higher or lesser degree, according to parents and/or teachers (see Table 2 for prevalence according to source and sex).
Table 2. ODD prevalence rate according to source
Note:
1Criteria met only by one informant (teacher, father or mother)
2Criteria met by both parents
3Criteria met by teacher and one of the parents
4Criteria met by the three informants
If we analyze the data separately for boys and girls, we can observe a similar tendency: for boys the ODD detection rate by source ODD 3 informants is 1.6%, whereas source ODD 1 informant presents a prevalence rate of 11.2% (the two other sources show the same prevalence rates that were presented for the total sample). This is relatively similar for girls, except for the rates of source ODD 3 informants (0.5%) and ODD 1 informant (7.6%) that are much lower.
If we compare ODD prevalence rate according to sex depending on the source, we can observe that the rate is higher in boys according to source ODD 1 informant (63% of ODD children are boys), and especially according to source ODD 3 informants (79% of ODD children are boys). The percentages for the other two sources are practically identical (50%) for both sexes.
In total, we observe that the ODD prevalence according to teachers is 5.1% (95% CI [3.88, 6.31]), according to fathers is 9% (95% CI [7.38, 10.54]), and according to mothers is 9.7% (95% CI [8.02, 11.29]). If we examine the sample of boys, we can see that in the case of teachers the prevalence rate is 6.8%, in the case of fathers is 9.3% and in the case of mothers is 10.3%. For girls, the prevalence rate according to teacher is 3.1% (less than half the rate shown in boys), according to fathers is 8.6% and according to mothers is 8.9% (in both cases, very similar to the boys` rates).
If we analyze sex differences according to the informant, as shown in Figure 1, in the case of teachers more than 71% of the ODD subjects are boys, while in the case of fathers and mothers this percentage is around 56%. In relation to teachers, sex differences are statistically significant, χ2(1, N = 1295) = 8.98, p = .003, with a medium effect size (0.38). Nevertheless, in the case of fathers, χ2(1, N = 1295) = 0.18, p = .669, and mothers, χ2(1, N = 1295) = 0.688, p = .407, differences are not statistically significant, and the effect sizes were much lower (0.04 and 0.08 respectively).
Figure 1. ODD prevalence rate by sex according to informant.
In the case of teachers, we found no significant differences between grades, χ2(3, N = 1295) = 1.68, p = .641. ODD prevalence rates fluctuate between 4.3% for first grade to 6.5% for second grade, and the effect size was 0.01. Among all the ODD detected, the participant distribution per grade is similar (between 21.2% for first grade and 28.8% for second grade). In the case of fathers, the prevalence rates vary from 6.8% in third grade to 11.4% in fourth grade. Among all the ODD detected, differences in percentages per grade fluctuate between 19.8% in third grade and 32.8% in fourth grade. Even though the differences are relatively higher in comparison to the ones shown for teachers, they are also non-significant, χ2(3, N = 1295) = 4.63, p = .201, and the effect size is similar (0.06). Finally, in the case of mothers, the prevalence rates vary from 8.2% for second grade to 10.5% for fourth grade. The ODD subject distribution in each grade varies between 19.2% in second grade and 28% in third and fourth grade. Likewise, there are no significant sex differences, χ2(3, N = 1295) = 1.22, p = .748, and the effect size is very similar (0.07).
The agreement level when detecting ODD varied across informants. The Kappa coefficient between teachers and fathers is low (although it is significant), κ = .14, t = 5.34, p < .001, between teachers and mothers is similar, κ = .13 t = 4.96, p < .001, and between fathers and mothers is much higher (κ = .60, t = 21.68, p < .001).
Results referred to the agreement between informants across items show a very low Kappa coefficient in the case of parents and teachers except for item number 8 (“Is often spiteful or vindictive”). These coefficients have fluctuated between .04 and .17. However, in item 8 (“Is often spiteful or vindictive”), teachers and fathers have shown a perfect Kappa coefficient (1), and teachers and mothers have shown a high coefficient as well (.58). In the case of father and mother agreement, the coefficients have been moderately high from .51 for item 7 (“Is often angry and resentful”) to .62 for item 1 (“Often loses temper”).
On the other hand, we also analyzed the most prevalent items reported by each informant (see Table 3). In the case of fathers and mothers, the item prevalence ranking has been the same, with hardly any sex differences. The most prevalent item has been number 6 (“Is often touchy or easily annoyed by others”), being reported approximately in 23% of all participants. The least prevalent item is number 8 (“Is often spiteful or vindictive”) with a percentage of approximately 3.7%. Fathers and mothers rated significantly more boys than girls in item 5 (“Often blames others for his or her mistakes or misbehavior”). Moreover, fathers rated significantly more boys in item 6 (“Is often touchy or easily annoyed by others”), and mothers in item 4 (“Often deliberately annoys people”). Teachers also report that item 6 (“Is often touchy or easily annoyed by others”) is the most prevalent, and item number 8 (“Is often spiteful or vindictive”) the least prevalent. Nevertheless, differences are found in the classification of the other items. Sex differences are more visible in the case of teachers (higher percentage of boys), appearing in all items except in number 2 (“Often argues with others”) and 8 (“Is often spiteful or vindictive”).
Table 3. Prevalence of ODD items according to informants (%)
Note:
* Based on Tables of Contingencies (χ2(1))
Finally, we examined the severity of the disorder by comparing participants that were detected as ODD by one informant (teacher, father or mother) with participants detected as ODD by the three informants. Participants identified as ODD by the three informants show higher mean scores than participants that were detected by only one informant. This finding suggests that children identified as ODD by the three informants present more severity. These differences were significant in the case of participants detected by fathers (t(47) = 3.43, p = .01) and mothers (t(56) = 3.77, p < .001), with high effect sizes (1.09 and 1.15 respectively). Although this comparison did not reach significance in the case of children detected by teachers (t(56) = 1.54, p = .129), the effect size was moderate (0.47).
Discussion
The ODD global prevalence rate (the sum of all the sources) is approximately 16%, which is above the expected. However, if we focus on the percentage of subjects that were detected by only one of the informants (the first defined source) then the rate stands around 9.5%. ODD prevalence reported by mothers and fathers was higher (9%–9.7% respectively) than the prevalence reported by teachers (5.1%). ODD prevalence according to both parents is around 5% (almost half of the rate based on one informant) and the prevalence defined by the agreement between teachers and both parents or one of the parents is around 1.7% (almost eight times less, compared to the rate based on one informant). We observed more agreement between fathers and mothers, in comparison to teachers and parents, as estimated. In terms of sex differences, parents do not show significant differences between boys and girls, whereas 70% of participants considered ODD by teachers were boys. On the other hand, our results show no differences in ODD prevalence according to age/grade.
A high global prevalence rate was observed (16%) in comparison to other studies that included parent and teacher reports (Ersan et al., Reference Ersan, Dogan, Dogan and Sümer2004; Gadow & Nolan, Reference Gadow and Nolan2002). However, we must have in mind that our global prevalence rate stems from a hierarchical strategy in which we add all the subjects that fulfill ODD criteria for three informants, for two informants and for only one informant. Nevertheless, this high rate may be partly the result of only using parent and teacher questionnaires without clinician confirmation (Boylan et al., Reference Boylan, Vaillancourt, Boyle and Szarmari2007). On the other hand, the ODD prevalence rate reported by only one of the different informants (9.5%) is more consistent with the rates described in the Diagnostic and Statistical Manual of Mental Disorders (4th Ed., text rev; DSM-IV-TR; American Psychiatric Association, 2000) for community samples (2–16%), and with the prevalence rates found in previous studies (Ersan et al., Reference Ersan, Dogan, Dogan and Sümer2004; Granero et al., Reference Granero, Ezpeleta, Domenech and De la Osa2008).
In the case of parents’ reports, Offord et al. (Reference Offord, Boyle, Racine, Szatmari, Fleming, Sanford and Lipman1996) found a much lower prevalence rate according to mothers (4.8%) than the one found in our study (although in this study the mother was the informant in most of the cases not in all). Prevalence reported by fathers is rarely presented in other studies, because information is collected usually from the mother, since she is the most consistently available parent to provide information about the child’s behavior (De los Reyes & Kazdin, 2005). In the case of teacher reports, similar ODD prevalence rates were also reported by Ersan et al. (Reference Ersan, Dogan, Dogan and Sümer2004) and Nolan, Gadow, and Sprafkin (Reference Nolan, Gadow and Sprafkin2001). However, other studies showed lower prevalence rates when teachers are the informants (Emberley & Pelegrina, Reference Emberley and Pelegrina2011; Munkvold et al., Reference Munkvold, Lundervold, Lie and Manger2009). Specifically, results from an epidemiological study conducted by Emberley and Pelegrina (Reference Emberley and Pelegrina2011) in Spain show a prevalence rate between 3.77%–3.83% according to teachers. Nevertheless, this is not far from the lower limit of our confident interval (3.88).
Fathers and mothers show an average agreement level, whereas teachers and parents show low agreement as reported in other studies (Achenbach et al., Reference Achenbach, McConaughy and Howell1987; Offord et al., Reference Offord, Boyle, Racine, Szatmari, Fleming, Sanford and Lipman1996). The tendency of teachers reporting less ODD than parents is also indicated in previous studies (Angulo et al., Reference Angulo, Jané, Bonillo, Viñas, Corcoll-Champredonde, González and Carbonés2010; Hart, Lahey, Loeber, & Hanson, Reference Hart, Benjamin, Loeber and Hanson1994; Loeber et al., Reference Loeber, Green, Lahey and Stouthamer-Loeber1991; Serra-Pinheiro et al., Reference Serra-Pinheiro, Mattos and Regalla2008). Moreover, other studies also support the fact that teachers report more ODD in boys compared to girls (Maughan et al., Reference Maughan, Rowe, Messer, Goodman and Meltzer2004; Munkvold et al., Reference Munkvold, Lundervold, Lie and Manger2009; Serra Pinheiro et al., Reference Serra-Pinheiro, Mattos and Regalla2008) and that parents do not report sex differences (Granero et al., Reference Granero, Ezpeleta, Domenech and De la Osa2008; Lahey et al., Reference Lahey, Schwab-Stone, Goodman, Waldman, Canino, Rathouz and Jensen2000; Maughan et al., Reference Maughan, Rowe, Messer, Goodman and Meltzer2004; Michanie et al., Reference Michanie, Kunst and Margulies2007).
Our results show no age differences of ODD in school age children. In the same way, other studies did not find age differences when parents (Lahey et al., Reference Lahey, Schwab-Stone, Goodman, Waldman, Canino, Rathouz and Jensen2000) and teachers (Nolan et al., Reference Nolan, Gadow and Sprafkin2001) were the informants. Anyways it is important to consider that our age range was relatively small (from 6 to 8 years), so in the future our results should be compared with those obtained with an adolescent sample and the same informant procedure.
Our findings also support the idea of high variability in the ODD prevalence rate when an individual is evaluated exclusively using psychometric scales. This suggests the necessity to incorporate other evaluation procedures (psychiatric interviews, diversity of scales and observational methods), in order to make progress in the determination of the real prevalence of ODD (and of course, in the diagnostic protocols).
Moreover, we found that parents detect the disorder more easily than teachers. So, it seems clear that either parents are more sensible to conduct problems or that these behaviors are more frequent at home, perhaps because the environment is less structured or parents have less ability to control them. Another potential explanation can be drawn from the attribution bias context model (De los Reyes & Kazdin, 2005), that states that these informant discrepancies can be related to the divergent perspectives of the evaluators with regard to whether or which of the child’s problems warrant treatment.
In sum, our data suggests that informants from different settings tend to disagree when asked to rate oppositional defiant symptoms in children. This proposes that children may display indeed a different behavior at home and at school. We cannot examine with our design which other mechanisms (such as the type of relation between the informant and the child, personal experiences, different perspectives of what is considered problematic, etc.) might influence the rating in more or less degree. But we can confirm the fact that the context in which the rating occurs is an important factor that contributes to informant disagreement.
In any case, we want to highlight that even though Kappa was significant in all our analysis (due to our large sample) only fathers and mothers show a moderately high value (.60). It is indeed an interesting value, but considering that both parents evaluated the child “in the same context” and that perhaps they did not answer the scales independently (since we did not control this), the Kappa value should be higher. This moderate agreement could be explained by differences in how parents and mothers interpret questions and/or the different thresholds for what is considered problematic (Kramer et al., Reference Kramer, Philips, Hargis, Miller, Burns and Robbins2004). In the future, studies are needed to determine what factors are influencing parent disagreement in assessing ODD. For example, it would be interesting to control both parents´ personality traits and their parenting style.
We presented two grouping strategies to analyze ODD prevalence: an informant-specific strategy (classified by fathers, mothers or teachers separately as ODD) and a cross-informant classification agreement strategy (classified by mothers and fathers, classified by teacher and one of the parents, and classified by fathers, mothers, and teachers). We expected that more agreement between evaluators would result in a lower prevalence rate. Our findings are consistent with these predictions: ODD prevalence according to both parents is around 5% (almost half of the rate based on one informant) and the prevalence defined by the agreement between teachers and one of the parents is around 1.7% (almost eight times less, compared to the rate based on one informant). Even if this drop was predictable, it is much accentuated.
The level of incidence of each of the ODD items was also different between parents and teachers. In the case of fathers and mothers, the ranking of clinically significant items was the same. On the other hand, teachers agree with parents in rating the item 6 as the most prevalent (“Is often touchy or easily annoyed by others”) and item 8 as the less prevalent (“Is often spiteful or vindictive”). In relation to sex differences, parents barely show significant differences, whereas teachers present a higher percentage of boys across most of the items.
In summary, high levels of variability between informants consistently imply that an informant´s report reflects the separate and combined influences of the actual characteristics of the child , the context in which the subject is observed, the perspectives (or biases) of the informant, and the error of measurement, as Kraemer et al. (Reference Kraemer, Measelle, Ablow, Essex, Boyce and Kupfer2003) suggested.
The main conclusion that could be extracted from our data is that the most adequate way to determine ODD prevalence is to use a multi-informant approach and to analyze parent and teacher reports separately in order not to overestimate prevalence rates. Moreover, from a pragmatic point of view, the differences found among evaluators (and across items) suggest that it is important to have the information from the three evaluators (fathers, mothers and teacher) when diagnosing ODD in childhood.
The main strengths of our study were the use of a large epidemiological sample, the inclusion of both parents (fathers and mothers) and teachers, and the consideration of sex and age. It is important to highlight that our study is one of the very few that have included fathers’ ratings separated from mothers’ ratings. Exploring fathers and mothers as independent evaluators, which interact in the “same” environment, allow us to analyze their level of agreement/disagreement when detecting childhood behavioral problems. A relevant clinical implication that stems from these findings is the need to include fathers’ ratings in order to obtain a comprehensive diagnosis of childhood ODD. An important limitation of our study is that that we did not control if fathers and mothers answered the scales independently, without influencing each other´s opinions. Another limitation is that we did not include clinician confirmation.
