Introduction
To understand discourse or a written text the comprehender needs to – among other things – establish sentence meaning based on knowledge of syntactic cues such as word order and case marking. In a sentence such as John sees her, the preverbal position of John and the accusative personal pronoun her signal that John is doing the seeing. It is well established that it takes some years for children to acquire the syntactic cues in oral language (Ambridge & Lieven, Reference Ambridge, Lieven, MacWhinney and O'Grady2015), but there has been less focus on possible individual differences in such syntactic knowledge once children progress beyond the first years of schooling. Yet, such individual differences may have relevance for children's possibilities for benefitting from school activities, both when engaging in spoken discourse, and perhaps especially when they are expected to learn from written texts of increasing complexity more or less on their own. Persons with developmental language disorder are known to struggle with acquiring and using syntactic knowledge at school entry (Owen & Leonard, Reference Owen and Leonard2006), into adolescence (Nippold, Reference Nippold2017), and in adulthood (Poll, Betz & Miller, Reference Poll, Betz and Miller2010). But it is less clear whether there are individual differences in syntactic knowledge of practical importance beyond what can be found in the minority of students with developmental language disorder. Our goal was to investigate syntactic knowledge as a construct in two ways: first, through demonstrating the convergent and divergent validity of two syntax measures compared to measures of other cognitive constructs; second, by investigating the relationship between syntax and reading comprehension as an indication of the practical relevance of individual differences in a syntactic knowledge construct in ordinary classrooms.
The role of syntactic knowledge in reading comprehension
According to the influential Simple View of Reading, reading comprehension relies on two partially separable components: decoding and language comprehension (Gough & Tunmer, Reference Gough and Tunmer1986; Hoover & Gough, Reference Hoover and Gough1990). Whereas decoding is narrowly defined as identifying words on the basis of letters, language comprehension is a broader cover term, which includes language components that are shared between reading and listening comprehension, such as vocabulary, morphology, and inferencing. Syntactic knowledge, like vocabulary and morphological knowledge, can be viewed as a lower order comprehension skill, which is used for establishing propositional meaning. Higher order skills integrate propositional meanings with background knowledge: for example, through inferencing (Kintsch & Rawson, Reference Kintsch, Rawson, Snowling and Hulme2005; Perfetti, Landi & Oakhill, Reference Perfetti, Landi, Oakhill, Snowling and Hulme2005). From the perspective of linguistics, syntactic knowledge clearly plays a role in the comprehension process by signaling the propositional relationship between word meanings. But it is not clear whether individual differences in syntactic knowledge in practice constitute a separable or important bottleneck in children's reading comprehension. Beyond the first years of language acquisition, differences could be too small to have relevant impact on real world activities such as reading. (cf. Perfetti et al., Reference Perfetti, Landi, Oakhill, Snowling and Hulme2005, pp. 237–238 for discussion).
Recent studies have found sizable correlations between sentence-level tasks and reading comprehension (Brimo, Apel & Fountain, Reference Brimo, Apel and Fountain2017; Deacon & Kieffer, Reference Deacon and Kieffer2018; Language and Reading Research Consortium & Logan, Reference Logan2017; Poulsen & Gravgaard, Reference Poulsen and Gravgaard2016; Silva & Cain, Reference Silva and Cain2015; Sorenson Duncan, Mimeau, Crowel & Deacon, Reference Sorenson Duncan, Mimeau, Crowel, & Deacon and H2021). To the extent that sentence-level tasks measure syntactic knowledge, this suggests that the individual differences are sufficiently large to have practical importance. But sentence-level tasks are complex, and the relation with reading comprehension could be confounded with several sources of individual differences other than syntactic knowledge: for example, morphological knowledge, vocabulary, working memory, and background knowledge and expectations. Indeed, LARRC and Logan (Reference Logan2017, pp. 460–461) suggested that their grammar latent variable, which was based on sentence-level tasks and word morphology, was a strong predictor of reading comprehension exactly because the indicator tasks drew on a wide range of language skills from the word to the text level. The present study was specifically concerned with isolating syntactic knowledge from other levels of language knowledge and working memory in the prediction of individual differences in reading comprehension.
Assessment of syntactic knowledge
Researchers have used different terms to cover various notions that children may differ in the ability to process sentence-level information. This article focuses on the concept syntactic knowledge – that is, knowledge about how meaning relations are signaled by word order and some inflectional morphology (e.g., case and verb morphology) according to learned conventions of a particular language, including the use of this knowledge to comprehend and produce sentences (e.g., Brimo, Lund & Sapp, Reference Brimo, Lund and Sapp2018). Syntactic knowledge can be distinguished from syntactic awareness: the ability to reason about sentence structure: for example, in word order correction tasks. Brimo and colleagues found indications in their meta-analysis that syntactic knowledge is a stronger correlate of reading comprehension difficulties than syntactic awareness. Furthermore, syntactic knowledge may be relevant in a broader range of communicative activities, where reflecting on the structure of language is rarely applied.
A number of different test formats have been used to measure syntactic knowledge: for example, sentence repetition, multiple choice sentence-picture matching, sentence-animation verification, and multiple choice comprehension questions (e.g., Bishop, Reference Bishop2003; Brimo et al., Reference Brimo, Apel and Fountain2017; Frizelle, Thompson, Duta & Bishop, Reference Frizelle, Thompson, Duta and Bishop2019; Poulsen & Gravgaard, Reference Poulsen and Gravgaard2016; Semel, Wiig & Secord, Reference Semel, Wiig and Secord2006). One central difficulty with interpreting children's performance on these assessment tools is that performance may rely on other factors than syntactic knowledge. Frizelle and colleagues (Frizelle, O'Neill & Bishop, Reference Frizelle, O'Neill and Bishop2017; Frizelle et al., Reference Frizelle, Thompson, Duta and Bishop2019) recently pointed out the heavy processing load in sentence-picture matching tests such as the TROG where the child needs to decide which of four different pictures represents the correct mapping of the syntactic structure of a sentence.
Another frequently used method is sentence repetition, where children are asked to repeat back sentences of varying difficulty (e.g., Semel et al., Reference Semel, Wiig and Secord2006). It is well established that the task is solved by comprehension into a meaning structure followed by regeneration of the sentence from this structure (Kidd, Brandt, Lieven & Tomasello, Reference Kidd, Brandt, Lieven and Tomasello2007; Lombardi & Potter, Reference Lombardi and Potter1992; Polišenská, Chiat & Roy, Reference Polišenská, Chiat and Roy2015; Nick G. Riches, Reference Riches2012; N. G. Riches, Loucas, Baird, Charman & Simonoff, Reference Riches, Loucas, Baird, Charman and Simonoff2010). However, it is still not clear to what extent sentence repetition is also dependent on more general memory resources. LARRC and Logan (Reference Logan2017) reported that sentence repetition loaded equally high on grammar and memory latent variables, suggesting that there is a sizeable memory component to sentence repetition.
To separate general task demands from syntactic demands, Poulsen and Gravgaard (Reference Poulsen and Gravgaard2016) developed a self-paced sentence comprehension task that required the participants to read sentences and answer two-choice comprehension questions addressing literal information encoded by the syntactic structures in the stimuli. The materials consisted of semantically matched basic and difficult syntactic constructions: for example, active vs. passive. They found that performance on the sentences with difficult syntax predicted variance in reading comprehension beyond performance on basic syntax (on which there was little variance), decoding, vocabulary, and verbal working memory. Thus, the correlation between the syntactically demanding items and reading comprehension could not be explained by the general task demands or the other control measures. This result supports the idea that sentence-level tasks measure a construct that is important for reading comprehension for reasons that cannot be reduced to, for example, vocabulary or working memory. An important limitation was that the self-paced format of the task could have allowed the test to be sensitive to individual differences in how children strategically solve problems: for example, by rereading difficult items. The present study aimed to replicate the results using an oral, non-self-paced format of the task to reduce the possible confounds of decoding skills and strategic rereading.
Any task designed to assess syntactic knowledge will potentially draw on other cognitive resources as well. This situation raises two questions of validity: is there substantial common variation between different task formats that can reasonably be said to measure syntactic knowledge (convergent validity)? And is this common variance separable from variance in tasks that do not draw on syntactic knowledge (discriminant validity)? With regard to convergent validity, the correlations between syntax measures vary considerably. Frizelle et al. (Reference Frizelle, O'Neill and Bishop2017) reported a correlation of .08 between a TROG style sentence-picture matching task and a parallel sentence repetition measure in a sample of five-year-olds. Hjetland et al. (Reference Hjetland, Lervag, Lyster, Hagtvet, Hulme and Melby-Lervag2019) reported a correlation of .30 in a sample of seven-year-olds between TROG-2 and sentence repetition, and The Language and Reading Research Consortium, Jiang, Logan, and Jia (Reference Jiang, Logan and Jia2018) reported cross-sectional correlations between .52 and .66 from pre-kindergarten to Grade 3. Aside from the fact that different samples may have differing baseline correlations, another explanation for the differences across studies might be differences in task characteristics, possibly including differences in demands for syntactic knowledge relative to other task demands. Frizelle et al. (Reference Frizelle, O'Neill and Bishop2017) assessed various types of relative clauses using the same target sentences in the two formats, whereas Hjetland et al. (Reference Hjetland, Lervag, Lyster, Hagtvet, Hulme and Melby-Lervag2019) employed a sentence memory task with increasing sentence length, but with no apparent syntactic selection criteria (see examples in Klem et al., Reference Klem, Melby-Lervåg, Hagtvet, Lyster, Gustafsson and Hulme2015). Finally, Recalling Sentences from the CELF test battery (Semel et al., Reference Semel, Wiig and Secord2006) was used in the LARRC study. In Recalling Sentences, sentence length, syntactic complexity, and vocabulary are confounded, and it is unclear how exactly syntactic knowledge is involved.
To sum up, there is evidence that there are individual differences in school-aged children's ability to extract information from sentences, but it is still unclear whether this is due to differences in syntactic knowledge or in other cognitive constructs involved in specific sentence-level tests.
The present study
The goal of the present study was to investigate the construct validity of the notion of syntactic knowledge, whether it differs from verbal working memory, and to what extent the correlation between syntactic knowledge and reading comprehension can be explained by working memory in middle-school aged children. The relationship with reading comprehension serves as an indication of whether individual differences in syntactic knowledge are sufficiently large to have practical relevance. The study evaluated the correlational properties of two syntactic knowledge tasks designed to maximize syntactic load and minimize dependence on decoding, vocabulary, morphology, and problem solving. It is difficult to remove memory load from verbal syntactic tasks since syntactic comprehension involves processing and integrating multiple elements over time. Instead, the study controlled for verbal working memory with tasks that – like the syntax tasks – require the participants to store and process multiple linguistic units, but without syntactic cues to guide encoding and retention.
The syntactic knowledge tasks were sentence repetition and an oral version of the multiple-choice sentence comprehension task used by Poulsen and Gravgaard (Reference Poulsen and Gravgaard2016). Both tasks used syntactic stimuli that were deemed challenging at least into adolescence based on previous research (Dabrowska & Street, Reference Dabrowska and Street2006; Ferreira, Reference Ferreira2003; Kristensen, Engberg-Pedersen & Poulsen, Reference Kristensen, Engberg-Pedersen and Poulsen2014; Slobin, Reference Slobin, Ferguson and Slobin1973), such as passives and non-canonical word order. The inclusion of two syntactic knowledge tasks allowed analyses of their convergent and discriminant validity compared to the other measures of decoding, vocabulary, and verbal working memory.
More specifically, the research questions were:
1) whether the two syntax measures were more highly correlated with each other than with working memory and vocabulary (convergent and discriminant validity).
2) to what extent working memory explained variance in the syntax measures.
3) to what extent the two syntax measures predicted variance in reading comprehension beyond measures of verbal working memory. Controls for decoding and vocabulary were also included to evaluate the independence of syntactic knowledge from these constructs.
4) whether comprehension accuracy on difficult items in the sentence comprehension task would predict unique variance beyond control variables including performance on basic items.
The last question was included as replication of Poulsen and Gravgaard (Reference Poulsen and Gravgaard2016). A positive answer to the last question would indicate that the ability to handle syntactic load plays a role in reading comprehension.
Method
Participants
One hundred and twenty-one grade 6 students from two schools in relatively affluent neighborhoods in the Copenhagen area, Denmark, participated in the study. Some of these were excluded from the analyses, 10 due to missing assessment of reading comprehension and an additional 24 to remove confounds concerning second-language learning. One student had cochlear implant and difficulties with completing the receptive language tasks. This student was also left out of the analyses. The remaining 86 children (32 girls, 54 boys) had a mean age of 12;10 (SD = 0;4). They all spoke the majority language, Danish, at home. Parents were required to give written consent before their child could participate.
Measures
Syntactic knowledge
Syntactic knowledge was assessed with two tasks: sentence comprehension and sentence repetition. In the sentence comprehension task, students listened to prerecorded sentences and answered questions concerning the syntactically coded information in the sentences. The task was very similar to the sentence comprehension task in Poulsen and Gravgaard (Reference Poulsen and Gravgaard2016), but target and question sentences were presented orally rather than orthographically. Materials were prerecorded in a sound studio, and the task was administered on laptop computers using the OpenSesame version 3.2.5 experiment software (Mathôt, Schreij & Theeuwes, Reference Mathôt, Schreij and Theeuwes2012). In each trial, the student first heard a sentence, which was automatically followed by a question. During the question, two written response options were positioned in the left and the right side of the screen. The students were asked to choose a response by pressing one of two buttons, each corresponding to one of the responses (e.g., left button for left response option).
To manipulate and isolate syntactic difficulty, items were constructed in sets of four around a verb, a set of referents, and one of three syntactic alternations: active vs. passive, premodified noun vs. embedded relative clause, and subject-verb-object word order vs. object-verb-subject word order. This is exemplified in Table 1. For example, 1A and 1B convey the same proposition, but either through an active or a passive construction. 1C and 1D reverse the roles of the participants in 1A and 1B. The reversal items were included to counterbalance any involuntary stereotypic background expectations that could (mis)guide interpretation of the sentences (Traxler, Reference Traxler2014). The passives and embedded relative clauses are often used as challenging constructions in tests of syntactic knowledge (Bishop, Reference Bishop2003; Poulsen & Gravgaard, Reference Poulsen and Gravgaard2016), but experience from previous studies is that, even with these constructions, ceiling effects can be expected with older children. Non-canonical constructions with object-verb-subject (OVS) word order were included in the present study because they are known to cause difficulty from early childhood into adulthood (Kristensen et al., Reference Kristensen, Engberg-Pedersen and Poulsen2014; Thomsen & Poulsen, Reference Thomsen and Poulsen2015). The canonical word order of Danish is SVO, like in English. However, objects and adverbials are allowed in the preverbal position, in which case the subject moves to the position immediately following the finite verb. SVO/OVS structure is signaled by the order of postverbal NP's relative to non-finite verbs and certain adverbials. Case markings on personal pronouns (as in English) also mark subject/object status, but in the present study, only word order was used to signal syntactic structure.
Table 1. Approximate Translations of Example Stimuli From the Sentence Comprehension Task
Note. SVO vs. OVS structure is signaled by the order of the non-finite verb and the adverbial relative to the second noun phrase.
The materials of the sentence comprehension task included five sets of four items with each syntactic alternation for a total of 60 items. The horizontal position of the correct answer was balanced within item sets. The order of items was fixed in the same quasi-random order for all participants. Items from the same sets were spread out over the list, and the same construction was never repeated twice in a row. The score was the proportion of correct answers. Cronbach's alpha was .80 in the present sample.
In the sentence repetition task, the students were asked to repeat 30 sentences verbatim. The sentences systematically included the same syntactic constructions as the sentence comprehension task, but the sentence repetition task was not designed to vary difficulty parametrically. Table 2 displays the included constructions and examples of items. The sentences in the sentence repetition task were longer than in the sentence comprehension task in order for them to draw on syntactic skills rather than just rote verbal short-term memory. The task was administered on a laptop computer with the prerecorded items in a fixed, quasi-randomized order. The same syntactic construction never appeared twice in a row. The score was the proportion of correct answers. Cronbach's alpha was .86 in the present sample.
Table 2. Approximate Translations of Stimuli From the Sentence Repetition Task Sorted by Syntactic Construction
Working memory
The students completed three working memory tasks: Backward digit span, Number sequencing, and Letter-number sequencing from WISC-V, Danish version (Wechsler, Reference Wechsler2017). In Number and Letter-number sequencing, the students were orally given a string of numbers and numbers and letters, respectively, and had to repeat them in the correct numerical and alphabetical order. In Backward digit span, students were asked to repeat a string of numbers in reverse order. In all three tests, the items gradually increased in length. The number of correct answers was transformed to scaled scores. The manual reports reliability estimates between .72 to .93 for the three tests in the relevant age group.
Vocabulary
Expressive vocabulary was assessed with a task developed for research purposes (Gellert & Vang Christensen, Reference Gellert and Vang Christensen2012). The students were asked to name 84 color photos of objects. The score was the proportion of correct answers. Cronbach's alpha was .84 in the present sample.
Decoding fluency
Decoding fluency was assessed with Elbros ordlister (“Elbros word lists”) (Elbro, Reference Elbro1990). The students were asked to read two lists of 20 words and two lists of 20 nonwords. Each list contained (non)words of increasing difficulty from two to eight letters. Responses were recorded and scored for accuracy and speed by trained assistants. The score for each list was the number of correct responses divided by the time to complete the list. Average scores for words and nonwords were computed separately. As a measure of reliability, the correlation between the word and nonword fluency was .90 (spearman-brown corrected) in the present sample.
Reading comprehension
Reading comprehension was assessed with Tekstlæseprøve 8 (“Text reading test 8”) (Møller, Reference Møller2013). The text is a connected exposition on a single subject with 43 maze items (four options each) and 11 two-choice recall items. The assessment was group-administered, and the students were allowed 30 minutes to complete the assignment. This was 5 minutes more than allowed by the test manual to minimize the effect of decoding fluency on the test results. The score for the test was the proportion of correct answers out of all items. 74% of the students completed all items within the time limit. Cronbach's alpha was .84 in the present sample.
Results
One outlier student scored 3.45 SD below the mean in reading comprehension. The reading comprehension score for that student was replaced with a value corresponding to the second lowest scoring student (2.44 SD below the mean). All variables had skew values under 1.
Table 3 presents the descriptive statistics of the individual study variables.
Table 3. Descriptive Statistics
To simplify further analyses, we computed composite scores for decoding fluency, memory, and syntactic knowledge as averaged z-scores. This was unproblematic for decoding fluency and syntactic knowledge because the correlations between the averaged measures were high (word and nonword fluency, r = .81, syntactic knowledge, r = .62). In contrast, the correlations between the separate working memory variables were low (r between .24 and .46). These low correlations put into question the reasonableness of the composite score. However, the working memory composite correlated more highly with reading comprehension and the syntactic measures than the separate working memory scores. Thus, this composite measure is the most conservative working memory control measure.
Table 4 presents the zero-order correlations between the study variables. The table includes both the composite and the separate measures of syntactic knowledge. Both measures of syntactic knowledge correlated significantly with reading comprehension. In general, sentence comprehension and sentence repetition correlated with the other measures at about the same level, with a weak tendency for sentence repetition to correlate more highly with other measures.
Table 4. Zero-Order Correlations Between Central Study Variables
Note. r-values above .21 are significant at .05 level.
The first research question was whether the two syntax measures correlated more highly with each other than with working memory and vocabulary. A substantial correlation would indicate convergent validity, and a higher correlation between syntactic measures than between syntax and working memory would indicate discriminant validity (Campbell & Fiske, Reference Campbell and Fiske1959). Table 4 shows that the two syntax measures were more highly correlated with each other (r = .62) than with any other measure. This correlation was indeed significantly higher than the correlation between sentence comprehension and composite working memory (r = .37), t = 2.69, p < .01 (Steiger, Reference Steiger1980), but it was not higher than the correlation between sentence repetition and the working memory composite (r = .43), t = 1.98, p = .051. However, we conclude that the pattern is in favor of the conclusion that the syntax measures tap into a set of skills that are at least partially separate from working memory. Furthermore, both syntax measures correlated more highly with each other than with vocabulary (both p's < .01). This indicates that the efforts to minimize the vocabulary load on the syntax tasks largely succeeded, and it further supports the interpretation of the syntax measures tapping into their own construct, syntactic knowledge.
The second research question was how much variation in the syntax measures can be explained by working memory. This information can be derived from Table 4. The working memory composite explained 13.7% variance in sentence comprehension, 18.5% variance in sentence repetition, and 20.3% variance in the composite measure of the two.
The third research question was to what extent the syntax measures correlate with reading comprehension because of shared variance with working memory. Tables 5–7 show multiple regression and commonality analyses with reading comprehension. Table 5 shows that in Model 1 with sentence comprehension and the working memory composite, sentence comprehension explained a total of 17% variance in reading comprehension. 12% is unique, while 5% is shared with the working memory composite. Model 2 additionally includes decoding and vocabulary as independent variables. In this model, sentence comprehension explained 5.4% unique variance, while 11.5% was common with combinations of decoding, vocabulary, and working memory. Table 6 and 7 show parallel analyses with sentence repetition and the composite syntax measure. The results are largely parallel, except that sentence repetition did not explain significant unique variance beyond decoding, vocabulary, and working memory (p = .053).
Table 5. Multiple Regression and Commonality Analyses of Sentence Comprehension With Reading Comprehension as Dependent Variable
Note. ** p < .01, *** p < .001.
Table 6. Multiple Regression and Commonality Analyses of Sentence Repetition With Reading Comprehension as Dependent Variable
Note. *** p < .001, + p < .10.
Table 7. Multiple Regression and Commonality Analyses of the Syntax Composite With Reading Comprehension as Dependent Variable
Note. ** p < .01, *** p < .001.
The fourth research question was whether accuracy on difficult items in the sentence comprehension task would predict unique variance beyond control variables including accuracy on basic items in sentence comprehension. This largely turned out to be the case. Sentence comprehension accuracy was significantly lower on difficult (M = .69, SD =.16) than on basic items (M = .93, SD = .06), Wilcoxon signed rank p < .001. Basic (r = .33, p < .001) and difficult sentence comprehension (r = .39, p < .001) correlated with reading comprehension at approximately the same level. Table 8 presents a hierarchical regression analysis with reading comprehension as the dependent variable. Basic sentence comprehension explained 2% variance in reading comprehension after controlling for working memory, decoding, and vocabulary, but this was not significant (p = .08). Difficult sentence comprehension explained a significant 4% unique variance in step 3 after control variables including basic sentence comprehension (p = .023). Thus, although both basic and difficult sentence comprehension correlates with reading comprehension at approximately the same level, the correlation between basic sentence comprehension and reading comprehension is largely explained by other factors. These results suggest that when the syntactic load on the sentence task is increased, the sentence comprehension task becomes a stronger unique predictor of reading comprehension.
Table 8. Hierarchical Regression Analyses With Reading Comprehension as Dependent Variable
Note. * p < .05, *** p < .001, + p < .10
Discussion
The study evaluated the convergent and discriminant validity of two researcher-developed measures of syntactic knowledge, and whether syntactic knowledge is related to reading comprehension in Grade 6 children. The tests were specifically designed to reduce dependence on other cognitive constructs such as problem solving and demanding vocabulary. The analyses showed that the syntax measures, sentence comprehension and sentence repetition, were more highly correlated with each other than with decoding, vocabulary, and working memory. Although the correlation between sentence repetition and sentence comprehension was only marginally stronger than between sentence repetition and working memory (p = .051), the pattern did not differ substantially from the sentence comprehension results. We cautiously interpret these results to support the convergent and discriminant validity of the syntactic knowledge measures and the syntactic knowledge construct.
Furthermore, the present study replicated previous studies that have found unique contributions from sentence-level measures to reading comprehension across several languages, including English, Cantonese, and Danish (Brimo et al., Reference Brimo, Apel and Fountain2017; Deacon & Kieffer, Reference Deacon and Kieffer2018; Language and Reading Research Consortium & Logan, Reference Logan2017; Poulsen & Gravgaard, Reference Poulsen and Gravgaard2016; Tong & McBride, Reference Tong and McBride2017). The present results extend the previous results in a number of ways. First, working memory was moderately correlated with the syntactic composite. This supports the notion that sentence comprehension is (partially) dependent on more domain-general working memory capacity, as has been suggested in the sentence processing literature (cf. Schwering & MacDonald, Reference Schwering and MacDonald2020 for a relevant review). But a lot of the variation in the syntax tasks was not explained by working memory, and working memory was not substantially responsible for the correlation between the syntactic measures and reading comprehension: commonality analyses showed that shared variance between the syntax composite and working memory explained 5.7% variance in reading comprehension, compared to 15.6% uniquely explained by the syntax composite. Thus, for the practical purposes of reading comprehension, there is more to sentence comprehension than just working memory. It should be noted that we used a definition of syntactic knowledge that includes the ability to process syntactic information (MacDonald & Christiansen, Reference MacDonald and Christiansen2002). Other conceptualizations may consider processing the domain of the working part of working memory (Crain & Shankweiler, Reference Crain, Shankweiler, Davison and Green1988; Smith, Macaruso, Shankweiler & Crain, Reference Smith, Macaruso, Shankweiler and Crain1989). The theoretical delimitation of working memory cannot be solved in the present paper. However, from an educational point of view, it is useful to know that some children struggle with acquiring and/or using domain-specific syntactic knowledge rather than with more domain-general working memory, as a domain-specific problem may call for more specific interventions.
Second, the present design and results strengthen the evidence that it is syntactic knowledge demands rather than other language comprehension demands that are responsible for the correlation between sentence-level tasks and reading comprehension. The sentence-level tasks were designed to maximize dependence on children's ability to use syntactic knowledge, and to minimize other sources of language comprehension difficulty that have been suggested to explain the correlation between sentence-level tasks and reading comprehension, e.g., from vocabulary or morphology (Language and Reading Research Consortium & Logan, Reference Logan2017). The low correlation between the syntax tasks and vocabulary suggests that the design efforts were successful to some degree.
Third, the results partially confirmed the result from Poulsen and Gravgaard (Reference Poulsen and Gravgaard2016) that performance on the difficult sentences in the sentence comprehension task explained significant variance beyond performance on basic sentences. Performance on the basic sentences was near ceiling, indicating that the task format in itself is not very challenging. However, in contrast to Poulsen and Gravgaard, accuracy on the basic structures was significantly correlated with reading comprehension. But this correlation was accounted for by the other control variables, suggesting that it was due to other task demands. Performance on the difficult sentences, on the other hand, explained unique variance in reading comprehension even after controlling for performance on basic items. This supports the possibility that the ability to handle syntactic load specifically is related to reading comprehension.
However, these results appear to contradict recent results from a study of English speaking grade 5 students that variance on basic – rather than difficult constructions – uniquely predict reading comprehension (Sorenson Duncan et al., Reference Sorenson Duncan, Mimeau, Crowel, & Deacon and H2021). The study used a sentence comprehension task similar to the present, but with generally lower accuracy scores, below chance level for many students on the difficulty constructions. The constructions were largely comparable to the ones in the present study, but task difficulty was higher, as measured by mean accuracy scores, probably because of an additional distractor and because sentences often were nonsensical. Despite the differences in results, both studies indicate that syntactic knowledge is important for the sentence comprehension tasks: even beyond the first years of schooling, children find some syntactic structures difficult, and this is correlated with reading comprehension after controlling for working memory. The conflicting results suggest that the correlation with reading comprehension depends more on whether the task difficulty is sensitive to the relevant range of skills in the sample, rather than specific constructions. Thus, the problem for some children probably is not that they simply do not know specific difficult constructions that mostly occur in written text, but rather that their syntactic knowledge is not sufficiently robust to be applied with consistent success when the demands are high.
Contrary to Poulsen and Gravgaard, the students in the present study and in Sorenson Duncan et al. (Reference Sorenson Duncan, Mimeau, Crowel, & Deacon and H2021) did not have the opportunity to strategically reprocess the stimuli through rereading. This reduces the possibility that the difference between basic and difficult items reflects individual differences in more general problem solving when faced with difficult tasks, in the sense that some students could be generally more inclined to apply strategic behavior than others.
Together, the present and previous results strengthen the case that individual differences in syntactic knowledge play a role in practice in ordinary classrooms when it comes to reading comprehension. The results support the inclusion of syntactic knowledge as a component of language comprehension in models that explain individual differences in reading comprehension, such as the Simple View of Reading (Gough & Tunmer, Reference Gough and Tunmer1986; Hoover & Gough, Reference Hoover and Gough1990) and newer conceptions in that tradition (Catts, Reference Catts2018; Perfetti et al., Reference Perfetti, Landi, Oakhill, Snowling and Hulme2005).
Limitations
The present study used researcher-created measures of syntax instead of traditional standardized test such as the TROG or tasks from the CELF. The reasoning was that the traditional tests may introduce non-syntactic factors to avoid ceiling effects, especially at the higher difficulty levels. The present results suggest that the researcher-created tasks successfully avoided heavy influence from vocabulary or memory, although the sentence repetition format may be more heavily influenced by working memory. However, since the study did not include traditional measures, it is not possible to make direct comparisons to see whether the researcher-created measures were indeed purer measures of syntactic knowledge than traditional measures. An important goal for future studies is to make such comparisons.
The present study did not clearly address the issue of the separability of vocabulary and syntax. Influence from vocabulary was minimized in the construction of materials from the syntactic tasks, and the correlation between the syntax measures and vocabulary were weak in this sample of middle-school aged children. It is unlikely that this was due to any simple sense of low reliability or validity of the vocabulary measure, since the reliability was high, and vocabulary was the strongest correlate of reading comprehension. But the lack of multiple indicators of vocabulary prevents modelling of the role of measurement error in the separation of syntax and vocabulary. Lervåg, Hulme, and Melby-Lervåg (Reference Lervåg, Hulme and Melby-Lervåg2018) in a Norwegian study found that their grammar construct did not predict variance in reading comprehension beyond other language measures when modeling measurement error. Their grammar construct included both morphological and syntactic tasks, and the other language constructs included indicators, such as listening span and inferencing, which presumably depended on syntactic processing of some sort. Further studies are needed to ascertain whether clearer independent contributions to reading comprehension can be found if the different language comprehension sub-constructs use indicators that are more clearly differentiated.
It should be noted that the idea that individual differences in syntactic knowledge are important for reading comprehension does not depend strongly on the separability of individual differences in vocabulary and syntax. There are both theoretical (Ambridge & Lieven, Reference Ambridge, Lieven, MacWhinney and O'Grady2015; Goldberg, Reference Goldberg1995; Tomasello, Reference Tomasello1992) and empirical (Language and Reading Research Consortium, Jiang et al., Reference Jiang, Logan and Jia2018) reasons to believe that the development of syntax is tightly linked to the development of vocabulary, and that the processing of different language components are intertwined (cf. Schwering & MacDonald, Reference Schwering and MacDonald2020). A relatively strong correlation between syntactic knowledge and reading comprehension still indicates that children who struggle with reading comprehension also struggle with assigning meaning based on syntactic cues, irrespective of whether these children have other language difficulties or not. However, indications of separability of syntax and vocabulary strengthen the case that syntactic difficulties are a barrier for text comprehension that cannot be explained away by failure to understand certain words. Ultimately, the issue of causality cannot be solved by regression studies. Intervention studies are needed.
Conclusion
The sentence comprehension and sentence repetition measures of the present study appeared to measure a construct separate from basic verbal working memory. Although the question of separability of individual differences in sub-constructs of language comprehension is still unclear, the present study provides evidence that students with poor reading comprehension also have a tendency to misinterpret the literal, syntactically coded meaning of sentences with easy vocabulary, especially if the syntax is challenging. Thus, what superficially appears as poor memory of a text – and maybe spoken messages – could instead stem from ineffectual encoding of propositional meaning based on word-order cues. The size of the correlations indicates that individual differences in syntactic knowledge could be as important as decoding and vocabulary to the variation in reading comprehension in ordinary classrooms. Some studies have shown promise in improving syntactic knowledge of children with developmental language disorder (Balthazar, Ebbels & Zwitserlood, Reference Balthazar, Ebbels and Zwitserlood2020). But larger scale studies of interventions applicable in ordinary classrooms with practical outcome measures – such as reading comprehension – are needed. Such studies would also provide more direct evidence of causality.
Acknowledgements
This research was supported by grant 8091-00021B from the Independent Research fund Denmark. We are also indebted to the student assistants who helped with data collection, and the schools and students who participated in the study.
