Autism spectrum disorder (ASD) is a neurodevelopmental disorder clinically defined by social communication deficits and the presence of restricted and repetitive patterns of behavior (APA, 2013). Early intensive behavioral interventions implemented between 18 and 30 months have yielded positive results (Dawson et al., Reference Dawson, Rogers, Munson, Smith, Winter and Varley2010; Reference Dawson, Jones, Merkle, Venema, Lowy, Faja, Winter and Webb2012; Estes et al., Reference Estes, Munson, Rogers, Greenson, Winter and Dawson2015; Kasari, Gulsrud, Wong, Kwon, & Locke, Reference Kasari, Gulsrud, Wong, Kwon and Locke2010), yet early identification of ASD remains an enduring challenge (Zwaigenbaum et al., Reference Zwaigenbaum, Bauman, Stone, Yirmiya, Estes and Wetherby2015).
Characterizing heterogeneity within the diagnostic phenotype of ASD is a perennial topic of investigation (Georgiades et al., Reference Georgiades, Szatmari, Boyle, Hanna and Duku2013; Happé, Ronald, & Plomin, Reference Happé, Ronald and Plomin2006; Lombardo et al., Reference Lombardo, Lai, Auyeung, Holt, Allison and Baron-Cohen2016; Rutter & Schopler, Reference Rutter and Schopler1987), with more recent work importantly focusing on earlier periods of development (Kim, Macari, Koller, & Chawarska, Reference Kim, Macari, Koller and Chawarska2016; Landa, Gross, Stuart, & Bauman, Reference Landa, Gross, Stuart and Bauman2012). However, less attention has been allocated to characterizing heterogeneity across the diagnostic threshold early in development (but see Messinger et al., Reference Messinger, Young, Ozonoff, Dobkins, Carter and Sigman2013), or interindividual variability that some researchers denote as representative of the typical-to-atypical continuum or normal-to-abnormal continuum (Cicchetti, Reference Cicchetti1984; Cicchetti, Reference Cicchetti1993; Cicchetti & Rogosch, Reference Cicchetti and Rogosch1996; Sroufe & Rutter, Reference Sroufe and Rutter1984; Wakschlag et al., Reference Wakschlag, Estabrook, Petitclerc, Henry, Burns and Briggs-Gowan2015). It could be argued that this source of variability obfuscates categorical diagnostic decisions early in development, yielding the wait-and-see approach that leads some families on ‘diagnostic odysseys’ that are both stressful and costly (Shattuck & Grosse, Reference Shattuck and Grosse2007; Shattuck et al., Reference Shattuck, Durkin, Maenner, Newschaffer, Mandell and Cuniff2009).
To address the challenge of improving early identification efforts, improved characterization of quantitative traits and improved characterization of the dimensionality of early emerging repetitive behaviors warrants more study. However, how one approaches measurement is inextricably linked to how one conceptualizes the phenomenon under study. If one conceptualizes the phenomenon of DSM defined autism spectrum disorder as a natural kind, or a categorical entity that exists in nature, then measurement tools will be designed to augment the identification of categorical thresholds. For example, the Autism Diagnostic Observation Schedule (Lord et al. Reference Lord, Risi, Lambrecht, Cook, Leventhal and Rutter2000) is excellent for informing diagnostic decisions, but it is ill-suited for characterizing variability in social communicative skills or repetitive behaviors below the ASD threshold. Indeed, 70% of the calibrated severity score range (Gotham, Pickles, & Lord, Reference Gotham, Pickles and Lord2009) represents variability within an ADOS-derived ASD classification. Whether severity scores of 1 to 3 represent meaningful dimensionality in nonspectrum children remains an empirical question. But to reiterate, the measure was designed to inform categorical judgement regarding a diagnosis. Constraints of health care systems likely play a large role in motivating this conceptual and methodological approach.
Alternatively, one might consider ASD as a collection of profiles/phenotypes that result from variable aggregation of quantitative traits (Constantino, Reference Constantino2011; Reference Constantino2018; Pruett & Povinelli, Reference Pruett and Povinelli2016). In order to examine variability in the aggregation or consolidation of quantitative traits, reliable characterization of the dimensionality, inherent to complex traits/behaviors, that crosses the typical-to-atypical continuum is necessary. Accompanying the variability in diagnostic outcome is variability in the pathways to that outcome, in this case ASD. Equifinality represents an important principle in the developmental psychopathology framework (Cicchetti & Rogosh, Reference Cicchetti and Rogosch1996; von Bertalanffy, Reference von Bertalanffy1968), and ASD need not represent a natural kind to implicate the concept. While disparate pathways or trajectories could be defined by binary variables (e.g., sex, responding to one's name or not, etc.), variable aggregation of continuously distributed traits (e.g., social responsiveness, IQ, etc.) likely play a substantial role in determining outcome. Quantifying the dimensionality of these key traits and complex behaviors, especially early in development, is critical to improving efforts directed at the early identification of ASD.
There exist a number of informant- and self-report instruments that characterize precisely this type of variability in older children and adults (Baron-Cohen, Wheelwright, Skinner, Martin, & Clubley, Reference Baron-Cohen, Wheelwright, Skinner, Martin and Clubley2001; Constantino & Todd, Reference Constantino and Todd2003), but fewer that have been designed explicitly to characterize the dimensionality of autistic traits/features in toddlers and preschool-aged children, periods vital to understanding the early manifestation of the disorder. The Repetitive Behavior Scales for Early Childhood (RBS-EC; Wolff, Boyd, & Elison, Reference Wolff, Boyd and Elison2016) was designed to overcome two shortcomings of the widely used Repetitive Behavior Scales–Revised (RBS-R; Bodfish, Symons, Parker, & Lewis, Reference Bodfish, Symons, Parker and Lewis2000; Lam & Aman, Reference Lam and Aman2007; Mirenda et al., Reference Mirenda, Smith, Vaillancourt, Georgiades and Duku2010), despite the latter's utility in differentiating 12- and 24-month-old high-risk infant siblings of children with autism who subsequently received a diagnosis themselves (Dimian et al., Reference Dimian, Botteron, Dager, Elison, Estes and Wolff2017; Wolff et al., Reference Wolff, Botteron, Dager, Elison, Estes and Piven2014). Namely, the RBS-EC was designed to (a) capture variability across the typical-to-atypical continuum and (b) incorporate developmentally appropriate items for young children, nominally between 6 months and 8 years of age. An initial demonstration in n = 914 community ascertained children between 17 and 25 months of age provided evidence of strong psychometric properties and preliminary evidence of concurrent criterion validity and discriminant validity (Wolff et al., Reference Wolff, Boyd and Elison2016). Notably, two additional instruments show promise in characterizing variability across the typical-to-atypical continuum in the repetitive behavior domain in toddlers and preschool aged children, the Childhood Routines Inventory–Revised (CRI-R; Evans, Ujarevic, Lusk, Loth, & Frazier, Reference Evans, Ujarevic, Lusk, Loth and Frazier2017) and the Repetitive Behavior Questionnaire–2 (RBQ-2; Arnott et al., Reference Arnott, McConachie, Meins, Fernyhough, Le Couteur and Leekam2010; Leekam et al., Reference Leekam, Tandos, McConachie, Meins, Parkinson and Le Couteur2007; Uljarevic et al., Reference Uljarevic, Arnott, Carrington, Meins, Fernyhough and Leekam2017).
In contrast to the repetitive behavior domain, the dearth of targeted quantitative assessments of social communication that meet the criteria set out above is conspicuous. Marrus and colleagues (Reference Marrus, Glowinskie, Jacob, Klin, Jones and Constantino2015) recently published what they consider a downward extension of perhaps the most widely used informant-report questionnaire in the autism field, the Social Responsiveness Scale (Constantino & Todd, Reference Constantino and Todd2003; Constantino et al., Reference Constantino, Davis, Todd, Schindler, Gross and Reich2003; Kamio et al., Reference Kamio, Inada, Moriwaki, Kuroda, Koyama and Constantino2013). The initial report on the vrRSB revealed greater concordance among monozygotic 18- to 24-month-old twins than among a matched group of dizygotic twins, as well as concurrent criterion validity with the MacArthur-Bates Communicative Development Inventories (Marrus et al., Reference Marrus, Glowinskie, Jacob, Klin, Jones and Constantino2015).
In the current study, we evaluate the criterion validity (Cronbach & Meehl, Reference Cronbach and Meehl1955) of the RBS-EC and the vrRSB in a community-ascertained sample of toddlers at normative epidemiological risk for ASD or other neurodevelopmental disorders. A community sample was targeted for use in the current effort so as to establish the phenomena in children at normative epidemiological risk, with the explicit intention of extending these approaches to children at risk for or diagnosed with ASD in the future. Criterion validity was examined in relation to the Infant-Toddler Social Emotional Assessment (ITSEA; Carter, Briggs-Gowan, Jones, & Little, Reference Carter, Briggs-Gowan, Jones and Little2003). The ITSEA characterizes meaningful variability in typically developing samples. For example, interindividual variability in internalizing and externalizing symptomatology below what would be considered thresholds of clinical impairment is associated with the same domains measured by the CBCL, respectively (Carter et al., Reference Carter, Briggs-Gowan, Jones and Little2003). Further, interindividual variability in social competence measured in early childhood is associated with key developmental precursors in typically developing children, such as joint attention (van Hecke et al., Reference Vaughan Van Hecke, Mundy, Acra, Delgado, Parlade and Pomares2007). The clinical utility of the ITSEA has been demonstrated in samples of young children with autism, suggesting that (a) preschool-aged girls with ASD show evidence of less social competence than boys with ASD matched on chronological age (Carter et al., Reference Carter, Black, Tewani, Connolly, Kadlec and Tager-Flusberg2007) and (b) greater externalizing problems in children with ASD is associated with greater parental stress (Davis & Carter, Reference Davis and Carter2008).
Concurrent and predictive forms of criterion validity were examined, controlling for both developmental and adaptive behavior levels. Additionally, we examined the discriminant validity of these constructs/topographies using separate subscales from the ITSEA. We hypothesized that vrRSB scores would be associated with social competence scores from the ITSEA, demonstrating criterion validity for reciprocal social behavior as measured by the vrRSB. Evidence of discriminant construct validity will be inferred from null findings between the vrRSB and the externalizing subscale from the ITSEA.
Further, results indicating that RBS-EC subscales are not associated with social competence as measured on the ITSEA would provide evidence of discriminant validity for the various repetitive behavior topographies, following evidence from large studies of epidemiologically ascertained twins (Ronald et al., Reference Ronald, Happé, Bolton, Butcher, Price and Plomin2006). Considering that (a) restricted patterns of play and insistence on sameness behaviors may be conceptually related to “just right’ behaviors as described by Evans and colleagues (Reference Evans, Leckman, Carter, Reznick, Henshaw and Pauls1997), (b) just-right behaviors share phenomenology with compulsive forms of anxiety, and (c) two key subscales of the ITSEA internalizing domain include general anxiety and inhibition to novelty, we hypothesized that higher-order repetitive behaviors (i.e., rituals and routines, and restricted interests) would be associated with the internalizing domain of the ITSEA. Further, we hypothesized that all topographies measured by the RBS-EC would be associated with the dysregulation subscale of the ITSEA. This hypothesis was informed by (a) recent evidence indicating associations between sensory-related behaviors (sensory sensitivity is a subscale captured by the dysregulation domain) and four categories of repetitive behavior in toddlers with autism (Wolff et al., Reference Wolff, Dimian, Botteron, Dager, Elison and Piven2019) and (b) evidence of increased rate of sleep problems in young children autism (Reynolds et al., Reference Reynolds, Soke, Sabourin, Hepburn, Katz and Levy2019), as the sleep subscale also contributes to the dysfunction domain on the ITSEA. Lastly, we explored associations between the RBS-EC subscales and the externalizing subscale of the ITSEA without directional hypotheses.
Methods
Participants and Procedures
Parents of n = 205 toddlers (53% female) contributed data to the current investigation, drawn from two separate studies of community-ascertained toddlers. The majority of the children were recruited from the Institute of Child Development Participant Pool, a state-wide research registry that draws primarily from the Twin Cities metropolitan area. Three children were referred to our study through the Autism Spectrum and Neurodevelopmental Disorders Specialty Clinic at the University of Minnesota. All analyses were conducted with and without these three participants, and no substantive results changed. In one study (n = 146, including 76 females; average age at time point 1 = 21.04 (SD = 2.67) months), drawn from a much larger sample (cf. Sifre et al., Reference Sifre, Lasch, Fenoglio, Georgieff, Wolff and Elison2018), the only inclusion criterion was age between 17 and 26 months, although no twins were included. In the second study (n = 59, including 33 females; average age at time point 1 = 18.21 (SD = 0.26) months), the following exclusion criteria were used: major pregnancy or birth complications, gestational age less than 37 weeks, birthweight less than 2000 grams, or the presence of genetic, neurological, or medical problems affecting cognition. Sex ratio did not differ between samples, nor did the vrRSB at time point 1, p = 0.87. Following best practices from the literature (Singer & Ye, Reference Singer and Ye2013), parents first completed questionnaires about their child's development online (Weigold, Weigold, & Russell, Reference Weigold, Weigold and Russell2013). Examples of the best practices include offering a financial incentive to participate, sending an introductory email a few days in advance of the actual email invitation to participate, and following up with parents to encourage completion of the questionnaires. Of the 205 participants, 203 contributed data on 2 occasions separated by approximately 8.3 months. One hundred fifty children participated in a follow-up direct behavioral observation, of which n = 142 provided valid direct behavioral assessment data. Parents of 15 children did not complete the full battery of questionnaire data, leaving a final sample of n = 127 (51% female). Demographic characteristics of the sample can be found in Table 1. The University of Minnesota Institutional Review Board approved studies from which these data were drawn, and parents provided permission and informed consent.
Table 1. Sample description
Note: SD = Standard Deviation; Min = Minimum; Max = Maximum; GA = gestational age. *Those who contributed behavioral data did not differ on family income or race/ethnicity from those who did not. The n = 127 indicates the sample with complete data at time point 2.
Measures
The Video-Referenced Rating of Reciprocal Social Behavior (vrRSB; Marrus et. al, Reference Marrus, Glowinskie, Jacob, Klin, Jones and Constantino2015) is a 48-item parent-report questionnaire, specifically designed to function as a downward extension of the Social Responsiveness Scale (Constantino & Todd, Reference Constantino and Todd2003). One strength of the vrRSB is that for the first 13 items, parents rate the behavior of their child compared with that of a 19-month-old performing complex social communicative behaviors in a 3-minute video (Marrus et al., Reference Marrus, Kennon-McGill, Harris, Zhang, Glowinski and Constantino2018). The remaining questions are designed to capture normative variability in reciprocal social behaviors. A total summary score was derived to index reciprocal social behavior, with higher scores indicating greater impairment, commensurate with the original SRS. The instrument takes approximately 10 minutes to complete. Designed for children 18–30 months old, it was administered to all participants at the first time point and all participants under 30 months at the second time point.
The Social Responsiveness Scale-Revised (SRS-2; Constantino & Gruber, Reference Constantino and Gruber2005) is a well-established and widely used informant-report instrument that characterizes quantitative autistic traits. The preschool form used in this study is designed for use with children from 30 to 54 months, and it typically takes 15 to 20 minutes to complete. Parents of children over the age of 30 months at the time of the follow up visit completed the SRS-2.
The Repetitive Behavior Scale for Early Childhood (RBS-EC; Wolff et. al, Reference Wolff, Boyd and Elison2016) is a 34-item parent-report questionnaire that examines the occurrence and frequency of various forms of repetitive behavior including repetitive motor mannerisms, self-directed or self-injurious behaviors, insistence on sameness behaviors or rituals and routines, and restricted patterns of play and/or circumscribed interests. The measure takes approximately 5 to 10 minutes to complete, and it was designed for children between 6 months and 8 years of age. In the current study, we examined total items endorsed, the total items endorsed across each subscale/topography, and the average interference score across all topographies. The RBS-EC was administered to all participants at the first and second time points.
The Infant Toddler Social-Emotional Assessment (ITSEA; Carter et al., Reference Carter, Briggs-Gowan, Jones and Little2003) is a 168-item parent-report measure that quantifies social and emotional problems and competencies. It is appropriate for children from 12 to 35 months, and includes scales of Internalizing (capturing fear, separation distress, inhibition to novelty, general anxiety, and depression and withdrawal), Externalizing (aggression and defiance, activity and impulsivity, and peer aggression), and Dysregulating behaviors (sensory sensitivity, eating issues, negative emotionality, and sleep issues) as well as Social Competence (comprised of items related to imitation play, mastery and motivation, compliance, empathy, and peer social relations). It takes approximately 20 to 30 minutes to complete.
The Mullen Scales of Early Learning (MSEL; Mullen, Reference Mullen1995) is a standardized developmental assessment validated for children from birth to 68 months of age. It takes approximately 60 minutes to administer to children 18 months and older. While the assessment consists of five subscales for children under 34 months, only the four subscales appropriate for all ages in our sample (which included children over 33 months) were used. Standardized
T-scores across four subscales—visual reception, fine motor, receptive language, and expressive language—were summed to yield an Early Learning Composite (ELC), which reflects overall developmental level.
The Vineland Adaptive Behavior Scales- II (VABS-II; Sparrow, Balla, & Cicchetti, Reference Sparrow, Balla and Cicchetti2005) is a standardized, semistructured functional skills assessment of adaptive behavior. It can be administered from birth to 90 years. It was administered as a parent interview and takes approximately 30 minutes to complete. VABS-II provides an overall Adaptive Behavior Composite (ABC) score, which was used in the current study. Descriptive statistics for all measures at both time points can be found in Table 2.
Table 2. Descriptive data for measures completed at time point 1 (exclusively online questionnaires) and time point 2 (online questionnaires and behavioral measures)
Note: SD = Standard Deviation; Min = Minimum; Max = Maximum.
Analytic Strategy
We first examined longitudinal associations between the vrRSB total score at time 1 and the vrRSB total score at time 2, vrRSB total score from time 1 and the SRS-2 at time 2 (using Pearson correlation), and RBS-EC total and subscale scores at time 1 & 2 (using Spearman rank-order correlation). Next, we investigated the criterion validity (concurrent and predictive) of the vrRSB and the RBS-EC, considering the ITSEA subscales as criterion measures, using a hierarchical regression framework. Based on an examination of the density plots, all RBS-EC subscale item endorsed and composite item endorsed scores from both time points were log-transformed to adjust for skew, with the exception of the repetitive motor and composite items endorsed scores from the first time point. Coefficients and inferential statistics for all regressions were then bootstrapped (B = 5,000) to account for any remaining normality issues, such as bimodality in distributions.
First, we characterized concurrent associations between RBS-EC and vrRSB scores and ITSEA scores (i.e., concurrent criterion-oriented validity). Next, we characterized the predictive criterion-oriented validity of both measures by examining the RBS-EC and vrRSB responses from time 1 relative to time 2 ITSEA scores (separated by ~ 7 months). In both sets of analyses, we entered age at time 2, sex, Mullen ELC, and Vineland ABC scores as step 1, and then we entered the vrRSB or RBS-EC score as step 2 to determine the portion of unique variance accounted for by the predictor of interest. Benjamini-Hochberg corrections for multiple comparisons were used for all regression analyses, considering a family of regressions the number of RBS-EC values that were used to predict the same dependent measure (i.e., 6).
Results
Representing data from time point 1, distributions of vrRSB total scores, RBS-EC total topographies endorsed, and RBS-EC subscale topographies endorsed are presented in Figure 1.
Figure 1. Distrubtions of vrRSB total score and RBS-EC items endorsed (total and by subscale) at time point 1, disaggregated by sex.
Tables 3 and 4 list the zero-order correlation coefficients between all key independent variables. Of note, vrRSB total score at time point 1 was strongly associated with vrRSB total score measured at time point 2 (separated by 7 [SD = 2.04] months), r (80) = 0.677, p < 0.001, 95% CI [0.539, 0.779] (Table 3). Additionally, vrRSB total score at time point 1 was also strongly correlated with SRS total score at time point 2 (separated by 11 [SD = 3.87] months), r (84) = 0.636, p < 0.001, 95% CI [0.491, 0.748] (Table 3). Scores for RBS-EC at time point 1 were significantly associated with RBS-EC scores at time point 2 (separated by 8.2 [SD = 3.1] months), with the higher-order repetitive behaviors showing stronger correlation coefficients: RBS-EC total items/topographies endorsed, r s (136) = 0.566, p < 0.001, 95% CI [0.425, 0.681]; repetitive motor behaviors, r s (136) = 0.489, p < 0.001, 95% CI [0.347, 0.617]; self-directed behaviors r s (136) = 0.349, p < 0.001, 95% CI [0.193, 0.488]; restricted interests r s (136) = 0.594, p < 0.001, 95% CI [0.465, 0.704]; rituals and routines r s (136) = 0.583, p < 0.001, 95% CI [0.445, 0.701]. The confidence intervals for the high-order subscales encapsulate the point estimate for repetitive motor subscale, but not the self-directed subscale. These results suggest that stability for higher-order subscales was greater than that for self-directed behaviors.
Table 3. Correlations between vrRSB and RBS-EC (items endorsed) subscales at first time point and RBS-EC (items endorsed), vrRSB, and SRS at second time point. All are Spearman rho coefficients except those between vrRSB and SRS (Pearson product correlations)
Note: CE = Composite Endorsed, RM = Repetitive Motor, SD = Self-Directed, Re = Restricted, Ri = Ritual, CI = Composite Interference. 1 = n of 205, 2 = n of 82, 3 = n of 86, 4 = n of 138. ***p < .001 **p < .01 *p < .05
Table 4. Spearman rho coefficient correlations between RBS-EC (items endorsed) subscales, vrRSB, and SRS at second time point.
Note: CE = Composite Endorsed, RM = Repetitive Motor, SD = Self-Directed, Re = Restricted, Ri = Ritual, CI = Composite Interference. 2 = n of 82, 5 = n of 54, 6 = n of 140. ***p <. 001 **p < .01 *p < .05.
The association between repetitive motor and self-directed behaviors at time point 1 was 0.52, 95% CI [0.413, 0.621]. This association at time point 2 was 0.315, 95% CI [0.145, 0.469]. The association between the restricted and ritual subscales at time point 1 was 0.697, 95% CI [0.610, 0.770]. This association at time point 2 was 0.652, 95% CI [0.521, 0.764]. The 95% confidence intervals for the lower-order subscale correlations and higher-order subscale correlations at time point 1 partially overlap. The confidence intervals for the lower-order correlations and higher-order correlations at time point 2 do not overlap, indicating that associations between higher-order RRBs are stronger than associations between lower-order behaviors at time point 2.
Concurrent Criterion Validity of the RBS-EC
Separate hierarchical regressions were examined for each RBS-EC independent variable (6) and each ITSEA dependent variable (4). A summary of the key results is provided in Table 5. Here, we focus on whether the step 2 independent variables of interest accounted for a unique portion of variance above and beyond the contribution of age, sex, developmental level, and adaptive behavior level. No RBS-EC score accounted for a significant portion of variance in the ITSEA competence subscale. The total items endorsed accounted for a significant portion of unique variance in ITSEA dysregulation, ΔR 2 = 0.146, p < 0.001, and ITSEA internalizing, ΔR 2 = 0.097, p = 0.002. The RBS-EC average interference across all topographies accounted for a significant portion of variance in ITSEA dysregulation, ΔR 2 = 0.286, p < 0.001, and ITSEA externalizing scores, ΔR 2 = 0.114, p = 0.002. The repetitive motor subscale did not account for a significant portion of unique variance in any ITSEA subscale. The self-directed behavior subscale accounted for a significant portion of unique variance in the ITSEA dysregulation subscale only, ΔR 2 = 0.095, p = 0.002. The restricted interest subscale accounted for a significant portion of unique variance in the ITSEA dysregulation, ΔR 2 = 0.109, p = 0.002, and ITSEA internalizing subscales, ΔR 2 = 0.078, p = 0.004. The rituals and routines subscale accounted for a significant portion of unique variance in ITSEA dysregulation, ΔR 2 = 0.196, p < 0.001, and ITSEA internalizing subscales, ΔR 2 = 0.207, p = 0.001.
Table 5. Results of bootstrapped regressions predicting ITSEA scales concurrently with ln-transformed RBS-EC items endorsed scales (n = 116), with Benjamini-Hochberg corrections for multiple comparisons (5,000 iterations per model). All RBS-EC measures were ln-transformed prior to entry as predictors in regressions.
Concurrent Criterion Validity of the vrRSB
Among the n = 76 children who had a vrRSB score at time point 2, total vrRSB scores accounted for a significant portion of unique variance in ITSEA competence scores, ΔR 2 = 0.082, p = 0.002; the ITSEA dysregulation score ΔR 2 = 0.104, p = 0.003; and the ITSEA internalizing score, ΔR 2 = 0.056, p = 0.031. Among the n = 41 children who had an SRS score at time point 2, total SRS score accounted for a significant portion of unique variance in ITSEA competence scores, ΔR 2 = 0.272, p < 0.001; the ITSEA dysregulation score ΔR 2 = 0.255, p < 0.001; and the ITSEA internalizing score, ΔR 2 = 0.364, p < 0.001. Neither vrRSB or SRS scores accounted for a significant portion of variance in ITSEA externalizing scores.
Predictive Criterion Validity of the RBS-EC
A summary of the key results is provided in Table 6. No RBS-EC score accounted for a significant portion of variance in the ITSEA competence subscale. The total items/topographies endorsed accounted for a significant portion of unique variance in ITSEA dysregulation ΔR 2 = 0.223, p < 0.001; ITSEA internalizing ΔR 2 = 0.069, p = 0.016; and ITSEA externalizing, ΔR 2 = 0.108, p < 0.001. The RBS-EC average interference across all topographies accounted for a significant portion of variance in ITSEA dysregulation, ΔR 2 = 0.115, p < 0.001, but no other ITSEA subscales. The repetitive motor subscale accounted for a significant portion of variance in ITSEA dysregulation, ΔR 2 = 0.081, p < 0.001, but no other ITSEA subscales. The self-directed behavior subscale accounted for a significant portion of unique variance in the ITSEA dysregulation subscale, ΔR 2 = 0.086, p = 0.002, and the ITSEA externalizing subscale, ΔR 2 = 0.062, p = 0.002. The restricted interest subscale accounted for a significant portion of unique variance in the ITSEA dysregulation, ΔR 2 = 0.204, p < 0.001; ITSEA internalizing subscale, ΔR 2 = 0.054, p = 0.016; and ITSEA externalizing subscale, ΔR 2 = 0.090, p < 0.001. The rituals and routines subscale accounted for a significant portion of unique variance in ITSEA dysregulation, ΔR 2 = 0.191, p < 0.001; ITSEA internalizing subscale, ΔR 2 = 0.091, p = 0.011; and the ITSEA externalizing subscale, ΔR 2 = 0.121, p < 0.001.
Table 6. Results of bootstrapped regressions predicting later ITSEA scales with RBS-EC items endorsed scales (N = 127), with Benjamini-Hochberg corrections for multiple comparisons (5,000 iterations per model)
Note: *indicated variables were ln-transformed prior to entry as predictors in regressions
Predictive Criterion Validity of the vrRSB
Among the n = 127 children who had a vrRSB score at time point 1, total vrRSB scores accounted for a significant portion of unique variance in ITSEA competence scores, ΔR 2 = 0.076, p < 0.001; ITSEA dysregulation scores ΔR 2 = 0.091 p < 0.001; and ITSEA internalizing scores, ΔR 2 = 0.056, p = 0.006 at time point 2. Total scores on the vrRSB did not account for a significant portion of variance in ITSEA externalizing scores. See Figure 2 for a graphical depiction of the vrRSB total scores at time point 1 and ITSEA social competence scores at time point 2 (n = 184, all children irrespective of Mullen and Vineland data).
Figure 2. Scatterplot of the vrRSB total scores at time point 1 and ITSEA Social Competence scores at time point 2 (n = 184; zero-order correlation r = –0.583, p < 0.001; without 2 outliers, r = –0.459, p < 0.001). Lower scores on the ITSEA and higher scores on the vrRSB indicate lower social competence and lower levels of reciprocal social behavior respectively.
Discussion
Two parent-report instruments, the vrRSB and the RBS-EC, were examined in relation to subscales of the Infant Toddler Social-Emotional Assessment (ITSEA) to elucidate criterion-oriented and discriminant construct validity (Cronbach & Meehl, Reference Cronbach and Meehl1955) in toddlers at normal epidemiological risk for neurodevelopmental disorders. Scores on the vrRSB were stable over time, and longitudinally associated with SRS scores measured ~ 10 months apart. Accounting for developmental and adaptive functioning levels, the vrRSB and the SRS both accounted for a significant portion of variance in ITSEA social competence scores when measured concurrently, and the vrRSB accounted for a significant portion of ITSEA competence scores when measured predictively, providing evidence of criterion-oriented validity. The social competence domain captured by the ITSEA and the vrRSB (predominantly containing items related to social communication and social interaction) appear to be quantifying aspects of a similar latent quantitative trait. Neither the vrRSB nor the SRS was associated with the externalizing subscale of the ITSEA, providing evidence of discriminant validity. Consistent with recent reports (Hawkes et al., Reference Hawks, Marrus, Glowinski and Constantino2018), we also observed associations between vrRSB scores and the internalizing and dysregulation domains of the ITSEA. Of note, the vrRSB contains questions about ASD-related behaviors that are not strictly social, and therefore the association between dysregulation and the vrRSB could be driven by these items.
In general, higher order repetitive behaviors (rituals & routines and restricted interests or play) as captured by the RBS-EC were more stable than the lower-order repetitive behaviors. Current work in our lab is examining longitudinal change in these subscales over time. Evidence from the extant literature suggests that lower order repetitive behaviors decrease linearly over this time frame, while higher-order repetitive behaviors show an inverted U shape, peaking in early preschool-aged children (Uljarevic et al., Reference Uljarevic, Arnott, Carrington, Meins, Fernyhough and Leekam2017). Increased stability among higher-order repetitive behaviors in our sample could stem from the timing of our data collection. That said, we expect these behaviors to change across development more considerably than reciprocal social behaviors captured by the vrRSB (Evans et al., Reference Evans, Leckman, Carter, Reznick, Henshaw and Pauls1997; Evans et al., Reference Evans, Ujarevic, Lusk, Loth and Frazier2017; Uljarevic et al., Reference Uljarevic, Arnott, Carrington, Meins, Fernyhough and Leekam2017; Wagner et al., Reference Wagner, Zhang, Gray, Abbacchi, Cormier and Constantino2019). No RBS-EC metric was associated with social competence as indexed by the ITSEA, providing evidence of discriminant validity for the RBS-EC topographies. All RBS-EC metrics (with the exception of repetitive motor behaviors measured concurrently) were associated with the dysregulation subscale of the ITSEA, which includes items such as: (a) sensory hyper- or hyposensitivity,(b) rigid food preferences or eating habits, and (c) sleep issues, all of which capture aspects of autistic symptomatology or features associated with ASD. Further, the higher-order repetitive behaviors, rituals & routines and restricted interests/play, were associated with ITSEA internalizing scores at both times points, concurrently and predictively. Taken together, these findings suggest general (cf. dysregulation) and specific (cf. internalizing) forms of criterion validity.
Characterizing repetitive behaviors has proven to be quite useful in disaggregating the early autism phenotype (Elison et al., Reference Elison, Wolff, Reznick, Botteron, Estes, Gu and Piven2014; Ozonoff et al., Reference Ozonoff, Macari, Young, Goldring, Thompson and Rogers2008), even using a parent-report measure that was not specifically designed for this age range (Dimian et al., Reference Dimian, Botteron, Dager, Elison, Estes and Wolff2017; Wolff et al., Reference Wolff, Botteron, Dager, Elison, Estes and Piven2014). Elucidating criterion-oriented and discriminant validity of the RBS-EC subdomains in the current sample represents a strategic decision to first establish and validate that meaningful variability can be captured among typically developing toddlers. Further explication of the utility of the RBS-EC is warranted in high-risk and clinical samples, as the ultimate goal is to develop a measure that quantifies clinically meaningful variability in these domains across the typical-to-atypical continuum.
Complementing the potential to improve characterization of the ASD phenotype, the dimensionality inherent to these measures (see Figure 1) may provide meaningful information regarding the effectiveness of treatment interventions implemented in very young children. Effectiveness of early intensive behavioral interventions has been demonstrated with rather gross indices of developmental functioning (Dawson et al., Reference Dawson, Rogers, Munson, Smith, Winter and Varley2010). Of course, a lot of work has yet to be done to determine the utility of these measures, but as more interventions are developed for very young children (Rogers et al., Reference Rogers, Vismara, Wagner, McCormick, Young and Ozonoff2014), new measurement tools will be needed to characterize intervention efficacy.
Replication of these results in larger, more diverse (both demographically and clinically) samples is warranted. Larger samples would also afford the opportunity to implement more sophisticated analytic approaches. As alluded to above, characterizing longitudinal change in repetitive behavior topographies over a broader period of time is needed to further establish the utility of the RBS-EC. One notable limitation is the exclusive use of parent-report for the key measures reported herein. However, this feature of our study could also be conceptualized as a strength, as the breadth of information and contexts that parents draw from when reporting on their children's behavior does not always coincide with behavior observed in a two-hour clinic visit. Nonetheless, identifying congruence with direct observation represents an imperative future effort.
In summary, complementing recent reports (Hawkes et al., Reference Hawks, Marrus, Glowinski and Constantino2018; Marrus et al., Reference Marrus, Glowinskie, Jacob, Klin, Jones and Constantino2015; Woff et al., Reference Wolff, Boyd and Elison2016), we provide evidence for the criterion-oriented validity of the vrRSB and the RBS-EC, at both concurrent and predictive time points. We also provide evidence of discriminant validity for the repetitive behavior topographies and the construct of reciprocal social behavior. These measures promise to provide meaningful information regarding variability and heterogeneity in behavioral traits and features that cross the typical-to-atypical continuum in early childhood, and may prove quite useful for studies of the emergence and early course of ASD.
Acknowledgments
The authors are indebted to the families who participated in this study.
Financial Support
CL was funded by a National Science Foundation Graduate Student Fellowship, JJW was funded by NIMH K01 MH101653, and JTE was funded by R01 MH104324. CL & JTE had full access to the data and take full responsibility for the integrity of the data and the accuracy of the data analysis. Data is available from JTE upon request. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
