Over the last decade several studies point to the importance of investing in the evaluation of adolescents’ social skills and their interpersonal contexts, aiming at preventive interventions, the promotion of health or the treatment of various psychological disorders (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, Reference Durlak, Weissberg, Dymnicki, Taylor and Schellinger2011). Social skills are understood as classes of social behavior, valued by society and essential for a socially competent performance (Del Prette & Del Prette, Reference Del Prette and Del Prette2009; Reference Del Prette and Del Prette2010). Therefore, its definition is complementary to the concept of social competence (Del Prette & Del Prette, Reference Del Prette and Del Prette2005, p. 33), understood as “the ability to articulate thoughts, feelings and actions according to personal goals and the requirements of any given situation and society, resulting in positive consequences for the individual and his relationship with other people”.
When present in the skill repertoire of adolescents, social skills are protective factors (Del Prette, Del Prette, Oliveira, Gresham, & Vance, Reference Del Prette, Del Prette, Oliveira, Gresham and Vance2012) and have been shown to be associated with healthy relationships with parents (Sabbag & Bolsoni-Silva, Reference Sabbag and Bolsoni-Silva2015), the prevention of courtship violence (Murta, Del Prette, & Del Prette, Reference Murta, Del Prette and Del Prette2010, World Health Organization, 2010), good academic performance (Caemmerer & Keit, Reference Caemmerer and Keith2015, DiPrete & Jennings, Reference DiPrete and Jennings2012, Feitosa, Matos, Del Prette, & Del Prette, Reference Feitosa, Matos, Del Prette and Del Prette2009), ease in making and maintaining friendships (Glick & Rose, Reference Glick and Rose2011) and other positive indicators such as social support and well-being (Beauchamp & Anderson, Reference Beauchamp and Anderson2010; Leme, Del Prette, & Coimbra, Reference Leme, Del Prette and Coimbra2015). On the other hand, social skill deficits are associated with depression (Campos, Del Prette, & Del Prette, Reference Campos, Del Prette and Del Prette2014a, Nilsen, Karevold, Røysamb, Gustavson, & Mathiesen, Reference Nilsen, Karevold, Røysamb, Gustavson and Mathiesen2013), social anxiety (Stravynski, Kyparissis, & Amando, Reference Stravynski, Kyparissis, Amando, Hofmann and DiBartolo2014), substance abuse (Vorobjov, Saat, & Kull, Reference Vorobjov, Saat and Kull2014) and various other psychological disorders (Del Prette, Rocha, Silvares, & Del Prette Reference Del Prette, Del Prette, Oliveira, Gresham and Vance2012). As such, a low social skills skill set constitutes a factor of risk.
An instrument published in Brazil in 2009, the Social Skills Inventory for Adolescents, better known through the acronym IHSA-Del-Prette, has been increasingly used, with additional studies further to those referred to in the manual (Del Prette & Del Prette, Reference Del Prette and Del Prette2009), proving its psychometric properties (Campos, Del Prette, & Del Prette, Reference Campos, Del Prette and Del Prette2014b; Dascanio, Olaz, Fontaine, Del Prette, & Del Prette, Reference Dascanio, Olaz, Fontaine, Rodrigues, Del Prette and Del Prette2012; Del Prette, Rocha et al., Reference Del Prette, Del Prette, Oliveira, Gresham and Vance2012; Del Prette, Teodoro, & Del Prette, Reference Del Prette, Teodoro and Del Prette2014; Leme, Campos, Coimbra, Del Prette, & Del Prette, Reference Leme, Campos, Coimbra, Del Prette and Del Prette2016; Leme et al., Reference Leme, Del Prette and Coimbra2015). Most of these studies, except for the Del Prette et al. (Reference Del Prette, Teodoro and Del Prette2014) study, did not explore the differences of social skills between gender and age-group. In the referred study, female adolescents demonstrated more social skills relating to empathy, assertiveness, social self-confidence, self-control and civility than male adolescents. With regards to age, the younger adolescents showed less social skills relating to social self-confidence and the older ones showed more self-control. Research that encompasses such differences can only be conducted if there is an assurance that the factorial structure and parameters of the items are invariant between the groups. The results of these studies provide evidence confirming the notion that the validation of psychological tools is a continuous and cumulative process, based on new studies (Borsa, Damásio, & Bandeira, Reference Borsa, Damásio and Bandeira2012).
IHSA-Del-Prette is a self-reporting tool for adolescents, with items that assess the social skills required for interpersonal situations with family, friends, people with authority, and affective partner. According to the authors of the manual (Del Prette & Del Prette, Reference Del Prette and Del Prette2009), the instrument brings together social skills such as: empathy, which includes items relating to skills in identifying other people’s feelings and problems, expressing understanding and support self-control, which includes skills relating to reacting calmly in frustrating, uncomfortable, angry and humiliating situations; civility, which includes skills relating to saying goodbye, expressing thanks for favor or praise, assertiveness, which refers to the capability of dealing with situations that require standing up for one’s rights and self-esteem, with the risk of the interlocutor reacting in a negative manner; affective approach, which brings together the conversational skills for relationships of friendship, sexual intimacy and expressions of affection; social self-confidence, which includes skills in presenting group work and talking to authority figures.
Given the relevance of studies on prevention and health promotion and the quality of life of adolescents (Murta et al., Reference Murta, Del Prette and Del Prette2010), a greater investment in large-scale evaluations is justified. For this, it is essential to have valid and reliable instruments that can also be quickly applied and processed, as is the case of brief scales. According to Bolsoni and Zuardi (Reference Bolsoni and Zuardi2015), these are important tools for the early identification, detection and prevention of positive and negative indicators of people’s quality of life and health. And, there is also the added advantage of the brief scales enabling fast data collection, with a low monetary cost and little difficulty in answering by the respondent, further to the quick tabulation of the data and, consequently, less time until publication and the information reaching the scientific community.
The development and/or refinement of brief scales from broader instruments is consistent with the current trend of producing “leaner” scales to enable cohort studies and problem tracking (Bolsoni & Zuardi, Reference Bolsoni and Zuardi2015). This requires investing in psychometric studies that allow for the identification of short scales, with acceptable validity and accuracy indicators. It is understood that, as with the complete instruments, the analysis of these properties must involve both factorial structure (exploratory and confirmatory) and reliability in terms of temporal stability and invariance, in other words, if the short scale evaluates in the same manner or if it skews the results when taking into account groups separated by gender and age-group.
Despite the importance and need for valid and reliable brief scales, the study of the psychometric qualities of such scales is still infrequent in foreign and domestic literature (Bolsoni & Zuardi, Reference Bolsoni and Zuardi2015). As such, when considering the psychometric qualities already presented by the original IHSA-Del-Prette version, it is understood that a brief version of this instrument is necessary to foster relevant directions for new research in this area. Based on the previous considerations, the present study aimed to evaluate the psychometric properties of a short version of IHSA-Del-Prette. Specifically, for the short version produced here, the following are presented: (1) Exploratory Factor Analysis - EFA; (2) Confirmatory Factor Analysis - CFA; (3) Reliability; (4) Multi-group Analysis - AMG (invariance).
Method
Participants
Participants in the investigation were Middle and High School students (N total = 2291), separated into two sample groups, who were assessed over an average period of five years. Group 1 (collected 2006–2007) refers to the data described in the original validation study of the IHSA-Del-Prette (Del Prette & Del Prette, Reference Del Prette and Del Prette2009), consisting of 1172 adolescents (ages between 12 and 17 years, M = 14.56; SD = 1.65), of both genders (50.30% women). Students were attending the 5th (n = 10, 0.9%), 6th (n = 176, 15.0%), 7th (n = 212, 18.1%) and 8th (n = 164, 14.0%) years of Middle Education and the 1st (n = 252, 21.5%), 2nd (n = 217, 18.5%) and 3rd (n = 141, 12.0%) years of public and private High Schools in cities of the states of São Paulo, Minas Gerais and Paraná. With regards to the socio-economic level Footnote 1 , most of the students were grouped in classes C (25.1%), B2 (22.8%) and B1 (19.5%).
Group 2 (collected 2011–2012) consisted of 1119 students, of which 581 (51.9%) were female with ages between 1 and 17 years (M = 14.11, DP = 1.31). The adolescents attended 6th (n = 138, 12.3%), 7th (n = 318, 28.4%) and 8th (n = 186, 16.6%) years of Middle Education and the 1st (n = 309, 27.6%) and 2nd (n = 168, 15%) years of public High Schools in cities of the states of São Paulo and Minas Gerais. With regards to socio-economic level, most of the students were grouped in classes C (24.3%), B2 (38.4%) and B1 (27.1%).
Instruments
Social Skills Inventory for Adolescents (IHSA-Del Prette). It is a self-reporting tool, developed by Del Prette and Del Prette (Reference Del Prette and Del Prette2009), which evaluates the social skills of adolescents aged between 12 and 17 years based on everyday situations of social interaction. The original version of the instrument consists of 38 items that take into account skills according to six subscales: (1) empathy (e.g., “When noticing that a colleague is sad or has a problem, I offer my support”); (2) self-control (e.g., “I react in a calm manner when things don’t turn out as I would like”); (3) civility (e.g., “When someone sets a good example, I praise them”), (4) assertiveness (e.g., “When someone asks me to do something I think is abusive, I refuse”), (5) affective approach (e.g., “When I don’t want to stay with a boy/girl, I refuse, even if he/she is very insistent”) and (6) social self-confidence (e.g., “At school or at work I make oral presentations whenever I’m asked”). Answers to the items are given according to a Likert five-point scale (varying between 0 = “0–2 times” to 4 = “9–10 times”). In the original study of the instrument, Del Prette and Del Prette (Reference Del Prette and Del Prette2009), using Principal Component Analysis, found satisfactory and adequate internal consistency indexes for the subscales of: empathy: α = 0.82; self-control: α = 0.73; civility: α = 0.75; assertiveness: α = 0,68; affective approach: α = 0.70; social self-confidence: α = 0.61. The temporal stability analysis (precision) produced positive and significant test-retest correlations (r = 0.84; p < .001).
Procedure
The data for both sample groups was collected in the classrooms. The periods of application lasted on average 30 minutes. At the beginning of each application, the objective of the investigation was announced, data confidentiality guaranteed and the voluntary nature of participation divulged. All parents or legal guardians of the students signed and returned an Informed Consent Form. The investigations were approved by the Ethics Committee of the university where the study took place.
Data Analysis
To report on the objectives related to the latent structure study of the IHSA-Del-Prette, Exploratory Factor Analysis, Confirmatory Factor Analysis and Invariance Analysis were carried out.
Exploratory Factor Analysis
Exploratory Factor Analysis was conducted based on polychoric correlation matrices. These matrices are more suited to ordinary scales, seeing that they focus on the frequency of the answers and not on the average of the item (Lorenzo-Seva & Ferrando, Reference Lorenzo-Seva and Ferrando2006). Item parameters were estimated by means of Unweighted Least Squares (ULS) and Promax oblique rotation.
Selection of factors
Parallel analysis was used for the extraction of the number of factors. Parallel analysis was conducted using 500 estimated random matrices with real data permutation (Buja & Eyuboglu, Reference Buja and Eyuboglu1992). The results of exploratory factor analysis were based on the consolidation of the reduced version of the IHSA-Del-Prette.
Confirmatory Factor Analysis and Internal Consistency
The reduced IHSA-Del-Prette structure was also evaluated using structural equation modeling (confirmatory factor analysis). The parameters were estimated based on the polychoric correlation matrix, using the Robustus Weighted Least Squares (WLSMV) method, recommended by Muthén, du Toit, and Spisic (1997) when the data is categorical (arranged on a Likert type scale, which characterizes it as ordinal). The overall fit of the hypothesized model was evaluated based on the following set of indices and respective reference values for an acceptable fit: χ2/gl <5 (Arbuckle, Reference Arbuckle2008); Bentler Comparative Fit Index - CFI> 0.95; Tucker-Lewis Index - TLI> 0.95; Root Mean Square Error of Approximation - RMSEA <0.05 (Byrne, Reference Byrne2012). The reliability of the IHSA-Del-Prette was evaluated by means of Composite Reliability (CC) and Average Extracted Variance (AVE). The analyses were carried out using the software Factor (version 10.3) and Mplus (version 7.1, Muthén & Muthén, 1998/2012).
Invariance Analysis
In order to verify to what extent the short version of the IHSA-Del-Prette maintained the same subscale structure with regards to temporal stability, age-group and gender, invariance analyses of the model were undertaken using structural equation modeling. As such, observing the recommendations made by Byrne (Reference Byrne2012), three models were tested in which the number of items and factors, (configural invariance), factor loadings (metric invariance) and thresholds (scalar invariance), were progressively constrained to be equal across groups. The differences between the adjustments of the most restricted and least restricted models were evaluated using Δχ2 (difference in chi-square value), ΔCFI, ΔCFI, Gamma-Hat (difference of the absolute Gamma-Hat indicator), ΔNCI (difference of McDonald’s Non-Centrality Index), as indicated by Cheung and Rensvold (Reference Cheung and Rensvold2002).
Results
Exploratory Factor Analysis
Exploratory Factor Analysis was conducted based on sample 1 of the present study. Before reducing the number of items, the preliminary subscale models were tested using this sample. In the first preliminary model, with all items, parallel analysis indicated that the percentage of variance explained by the fourth dimension with random data was higher than the one explained with empirical data (empirical data: 28.1%, 7.3%, 5.2% and 4.5%, random data: 5.6%, 5.4%, 5.1% and 5.0%). To elaborate the second preliminary model, four items with the highest factorial loads in a single dimension (i.e., without cross loadings), and higher communalities, were chosen. As such, the second model presented 12 items contemplating the dimensions of Self-control, Affective Approach and Empathy, but excluded the Assertiveness dimension. This subscale has been associated with important indicators of positive adjustment in adolescents, such as self-esteem and well-being (Sarkova et al., Reference Sarkova, Bacikova-Sleskova, Orosova, Geckova, Katreniakova, Klein and van Dijk2013). Thus, a final model was tested, in which four items with the higher loadings of the Assertiveness subscale were reintroduced into the analyses. The factor loadings, communalities and eigenvalues of the final exploratory factor analysis are presented in the supplementary Table, in the appendix.
Confirmatory Factor Analysis and Internal Consistency
Confirmatory factor analyses were performed using structural equation modeling. All models were estimated based on sample 2 of the present study. Initially, an attempt was made to evaluate the fit of the final model with four dimensions and 16 items, as well as the preliminary model with three dimensions and twelve items. Both models were configured with correlated first-order factors and presented the following fit indicators: Three factor model (12 items), χ2(gl) = 215.14 (51), TLI = 0.95, CFI = 0, 96, RMSEA (90% CI) = 0.05 (0.05–0.06); four factor model (16 items), χ2(gl) = 353.85 (98), CFI = 0.96, TLI = 0.95, RMSEA (90% IC) = 0.05 (0.04–0.05).
It should be noted that the fit indicators of the two models cannot be compared, since they are non-nested models with different numbers of items. Generally, however, the results indicate that both models are plausible. Additionally, the introduction of four items and one dimension was not accompanied by a drastic reduction in the quality of fit of the model. Thus, it was decided to keep the four-dimensional model for the reduction of the IHSA-Del-Prette. For model identification purposes, the variances of latent variables were set at one, so that all factor loadings could be freely estimated. Consequently, standardized and non-standardized loadings resulted in the same values. Item parameters are shown in Table 1.
Table 1. Synthesis of the Confirmatory Factor Analysis Results (n = 1119)
Note: a = determination coefficients (i.e., squared loading) are presented in italics; β = standardised and non-standardized coefficient (latent variable variance was set at 1); SE = standard error of estimate; δ = threshold between categories; AVE = average variance extracted.
The loadings average (standardized regression coefficients between items and factors) was 0.63 (SD = 0.09). However, ten factor loadings presented values lower than 0.70, reducing the reliability of the latent scores. The correlations between the factors were moderate, indicating that a significant part of score variance is shared among the factors.
Regarding the endorsement of the categories, the thresholds, presented in Table 1, point to relatively easy items. For most items, participants, with a latent score only a standard deviation above the average, tend to endorse the higher Likert category. In Table 1, the values for average variance extracted (AVE) are also shown. It was found that the variances shared between the dimensions (determination coefficients) are smaller than the AVE. These results evidence the discriminant validity of the four factorial scores.
AVE was also used to evaluate internal consistency. Considering it is a reduced instrument that has few items, the AVE was expected to be greater than 0.49. With regards to the empathy and assertiveness factors, the higher factorial loads supported AVE values near the cut-off point. With regards to the self-control and affective approach factors, the AVE values are minimally acceptable, even though lower than expected, seeing that the factorial load averages of these factors were 0.58 and 0.60, respectively.
Still with regards to internal consistency, the composite reliability (CC) values were: 0.66 for self-control scores; 0.69 for affective approach; 0.78 for empathy; and 0.75 for assertiveness. These values, close to or higher than 0.70, also show evidence of internal consistency and scores reliability, when considering the reduced number of items, seeing that CC tends to result in single values due the few items.
Invariance Analysis
Based on the evidence of the four-dimensional structure of the reduced version of the inventory, it was sought to evaluate the invariance of the parameters among the different databases (consisting of samples from different years), age-group (12 to 14 years and 15 to 17 years) and gender, by means of multi-group analyses. For this, the total database, consisting of the two samples of the present study (N = 2291), was used. The results for the multi-group analyses are shown in Table 2.
Table 2. Invariance Analysis - Multi-Group Confirmatory Factor Analysis (N = 2291)
Note: α = thresholds of items 10 to 15 freely estimated between genders male and female; b = differences between Partial Scalar and Metric models.
* (p < .05).
With regards to the groups divided by databases and age groups, the parameters of the items presented invariance. The fit differences, evaluated by the chi-squared test, were statically significant between the Metric model and the Scalar model, however the differences in CFI were small and lower than 0.01. Additionally, the differences of the McDonald’s index (ΔNCI) were lower than 0.02, which also denotes the invariance of the parameters. Though, Gamma-hat differences were slightly higher than 0.001. Thereby, the fit indices yielded diverging conclusions: CFI and NCI indicated invariance, Gamma-hat presented borderline results, and the chi-squared test rejected the model in which the thresholds were defined as invariant between the groups. It should be noted, however, that the chi-squared test is sensitive due to sample size. Additionally, when comparing the scalar and configural model adjustments, the critical difference ratio is less than 2 (for the database, Δχ2 /Δgl = (907.9 – 828.5) / (252 – 196) = 1.42; for the age-group, Δχ2 /Δgl = (885.5 – 784.0) / (252 – 196) = 1.81). In summary, there is evidence of the invariance of the parameters for the time (database) and for the age-group.
Regarding the gender variable, the multi-group analysis did not indicate any invariance of the thresholds, seeing that the scalar model presented a lower fit to the data when compared to the metric model. To expand this analysis, it was verified that only items 10 and 15 showed differences above 0.30 between the thresholds of the male and female groups (item 10, female: δ1 = –0.01, δ2 = 0.43, δ3 = 0.97, δ4 = 1.41; item 10, male: δ1 = –0.38, δ2 = 0.04, δ3 = 0.52, δ4 = 1.06; item 15, female: δ1 = –1.34, δ2 = –1.02, δ3 = –0.58, δ4 = –0.04; item 15, male: δ1 = –1.42, δ2 = –1.00, δ3 = –0.40, δ4 = 0.23). Considering that the thresholds are associated with the probability of endorsement, the results showed that girls require a latent score greater than the boys to endorse the high categories of item 10, and a lower latent score to endorse the higher categories of item 15. It should be noted that the differences are greater for item 10 (Δδ > 0.3) and smaller for item 15.
Discussion
Research has shown that social skills prevent social-emotional difficulties and psychopathological disorders during adolescence and other phases of the life cycle, as well as being associated with higher levels of self-esteem, well-being and academic performance (Beauchamp & Anderson, Reference Beauchamp and Anderson2010; Caemmerer & Keit, Reference Caemmerer and Keith2015; Durlak et al., Reference Durlak, Weissberg, Dymnicki, Taylor and Schellinger2011; Feitosa et al., Reference Feitosa, Matos, Del Prette and Del Prette2009; Leme et al., Reference Leme, Del Prette and Coimbra2015). On the other hand, there is a lack of instruments containing evaluated psychometric properties that seek to investigate the repertoire of social skills during adolescence. In this context, there is still no brief scale relating to the social skills of adolescents in Brazil. This type of instrument, under-investigated in a national and international context, makes it possible to collect data more quickly and economically in order to identify resources and behavioral deficits (Bolsoni & Zuardi, Reference Bolsoni and Zuardi2015). The present study evaluated the psychometric properties of a short version of the Social Skills Inventory for Adolescents - IHSA-Del-Prette (Del Prette & Del Prette, Reference Del Prette and Del Prette2009), whose full scale possesses satisfactory psychometric indicators.
The data obtained during the present study indicates the feasibility of the IHSA-Del-Prette brief scale, with good psychometric properties. It can be emphasized that the first preliminary model and the parallel analysis do not make the structure of the original instrument and its factors impracticable, and they could be resumed at a later stage with smaller subsamples, both in intervention and in more detailed evaluations.
The EFA and CFA results confirmed the theoretical model underlying the short version of IHSA-Del-Prette. The reduced instrument consisted of 16 items that took into account important social performance factors during adolescence, namely: self-control, affective approach, empathy and assertiveness. The items of the civility and social resource factors, present in the original instrument, are somehow contemplated in the dimensions of the brief scale. These dimensions are relevant for a healthy family relationship and help deal with peer pressure and bullying situations, thus promoting conflict resolution in a school setting (Feitosa et al., Reference Feitosa, Matos, Del Prette and Del Prette2009; Leme et al., Reference Leme, Del Prette and Coimbra2015; Sarkova et al., Reference Sarkova, Bacikova-Sleskova, Orosova, Geckova, Katreniakova, Klein and van Dijk2013).
Nevertheless, the results suggest some limitations and point to future research in improving the short scale and its more effective use. With regards to factorial load averages of the reduced version, it was verified that 10 items produced values lower than 0.70, which reduce the reliability of the latent scores estimation. Such results were already expected, seeing that reduced scales tend to shrink the reliability and are not suitable for clinical diagnosis, but rather for rapid assessment and screening in wide ranging surveys. Additionally, the reliability presented on the present study is compatible with those presented in previous full-scale studies (α between 0.62 and 0.82 in Del Prette & Del Prette, Reference Del Prette and Del Prette2009, α between 0.63 and 0.82 in Campos et al., Reference Campos, Del Prette and Del Prette2014b). Therefore, it is also suggested that, during research that seeks to compare groups or relate social skills to other variables, latent modelling should be used, seeing that, in that analysis set, the imprecision of the scores is weighted during parameter estimation of the relationships between variables.
It should be noted that the brief scale, by itself, has not yet been applied. In the two samples of the present study, only the full scale was used and, therefore, the comparison of results obtained with the original scale and the brief scale on the same sample of respondents has not yet been tested. In this sense, although unlikely, it is important to assess whether the items presented only in the brief structure maintain the same structure or infringe on the assumption of local independence.
Database and age-group invariance was verified. These results indicate that the scale presented with a stable structure over time (group 1: 2006–2007, group 2: 2011–2012), considering that the period between data collections was five years. Additionally, parameter invariance by age-group allows for the comparison between scores of adolescents of different ages.
However, the results indicated that there was no invariance with regards to gender, especially for items 10 (girls <boys) and 15 (boys <girls). Item 10 relates to the social skill (affective approach) of adolescents in expressing an interest in a person when they have the opportunity. Item 15 refers to the social skill (assertiveness) of adolescents in refusing a person’s insistent request to “go out with” them. It can be assumed that item 10 is more difficult to be observed in girls and item 15 more difficult in boys, due to the influence of the gender roles reproduced in our culture. In general, women are induced to be passive in romantic relationships (Murta et al., Reference Murta, Del Prette and Del Prette2010; Schäfer, Reference Schäfer2008), while men are encouraged to be active and not very discriminate, i.e., they must seek romantic situations, even though they are unwilling, so as not to invalidate their masculinity before their peers (Straus, Reference Straus2011).
As such, future studies can compare the social skills of boys and girls and, if possible, consider the difference in endorsement difficulty. Otherwise, any differences may be attributed, in part, to the differences between how answers are provided in the scale and not to the real differences between boys and girls. However, it should be noted that the differences found in relation to gender were relatively small and limited to two items, which does not, in all probability, invalidate the analyses of differences between genders already made in the literature. In conclusion, the results showed that the IHSA-Del-Prette brief scale demonstrated very satisfactory psychometric properties, by which its use is recommended in future studies relating to the social skills of adolescents. Several researchers have pointed out the importance of parents, teachers and psychologists favoring the development of social skills in order to prevent interpersonal and socio-affective difficulties or to promote mental health in adolescents (Del Prette, Rocha et al., Reference Del Prette, Del Prette, Oliveira, Gresham and Vance2012; Durlak et al., Reference Durlak, Weissberg, Dymnicki, Taylor and Schellinger2011).
Appendix
Supplementary Table
Exploratory Factor Analysis and correlations between IHSA-Del-Prette (n = 1172)
