Productive versus receptive vocabulary in L2 acquisition

Both first language (L1) and L2 vocabulary knowledge can be broken down into two types: receptive and productive knowledge. Receptive knowledge is the ability to recall the meaning of a word when its form (written or spoken) is presented; productive knowledge is the ability to produce the correct form (written or spoken) to express a given meaning (Laufer, Reference Laufer1998; Nation, Reference Nation2013). Comprehension typically precedes production in L1 vocabulary acquisition, and even in adult L1 speakers, receptive vocabulary knowledge exceeds productive vocabulary knowledge (Clark, Reference Clark1993, Reference Clark2009). These patterns are mirrored in L2 acquisition, with the gap between receptive and productive knowledge being even more exaggerated, particularly at lower proficiency levels (Schmitt, Reference Schmitt2014; Webb, Reference Webb2008).

Despite the plethora of studies that have been conducted on the gap between receptive versus productive vocabulary size (see Schmitt, Reference Schmitt2014) and a general awareness that low productive vocabulary knowledge impedes learning in the L2 classroom, there has been little research on instructional techniques to improve productive vocabulary knowledge. Some previous studies have compared output-based training tasks to receptive-based tasks. Webb (Reference Webb2005) contrasted a sentence-production task, in which participants had to write a sentence containing the target word, with a sentence-reading task (see also Keating, Reference Keating2008; Pichette et al., Reference Pichette, De Serres and Lafontaine2012). Input about the target word’s meaning was provided by presenting the target word with an L1 translation equivalent. De La Fuente (Reference De La Fuente2002) conducted a similar comparison, but input about the target words was contextualized via L2 labels rather than L1 translation equivalents. While the learners in these studies showed greater improvement in the output-based training conditions, the cued forward recall tasks used to assess learning have notable limitations as measures of productive vocabulary knowledge. In these tasks, the learner must produce the L2 words following a prompt such as a picture of the L1 translation equivalent. This shows knowledge of the basic form–meaning link, which is a central component of vocabulary knowledge (Nation, Reference Nation2013); however, the ability to produce words in context, even in a constrained context such as category association (as in the current study), likely requires a depth of knowledge that goes beyond this basic link (Meara, Reference Meara2009).

These intervention studies also all focused on novel word learning. The target items were selected to be completely unfamiliar to the participants. However, when investigating productive vocabulary knowledge, it is important to consider not only the acquisition of new lexical items but also the development of productive knowledge for words that are already known receptively (Meara, Reference Meara, Schmitt and McCarthy1997). To our knowledge, Lee (Reference Lee2003) is the only study to have tracked the progression from receptive to productive knowledge for specific items following a training intervention (Lee & Muncie, Reference Lee and Muncie2006, conducted a similar study but did not measure the receptive knowledge of specific items). Lee investigated the productive use of vocabulary in thematically constrained written compositions. She found that after reading a text passage containing the target vocabulary, the learners correctly produced only 13% of the target words they knew receptively. After a reading comprehension exercise plus explicit vocabulary instruction, the learners correctly produced 64% of the receptively known words in their compositions. In addition, they correctly produced 43% of the target items, which had not been known receptively at pretest. There was a small decline in their productive knowledge at a delayed posttest 3 weeks later. This study provides evidence that productive knowledge for new, as well as previously known, words can be gained and maintained following a targeted intervention, but to date this is an understudied topic in L2 vocabulary research.

The concreteness effect in vocabulary learning

When investigating any method of vocabulary instruction, it is important to account for word-level variables, as certain word types pose additional challenges to L2 learners. One such variable is concreteness, which describes the extent to which a concept can be perceived by the senses, as well as how easy it is to create a mental image of the concept in question. Concrete words (e.g., ambulance) score higher on concreteness and imageability than abstract words (e.g., emergency). Consistent differences have been observed between abstract and concrete words in L1 and L2 acquisition and processing. In the L1, concrete words are acquired earlier (Schwanenflugel, Reference Schwanenflugel and Schwanenflugel1991; Spätgens & Schoonen, Reference Spätgens and Schoonen2018) and are processed more quickly than abstract words (de Groot, Reference de Groot1989; Schwanenflugel & Shoben, Reference Schwanenflugel and Shoben1983). In L2 lexical processing, concrete words are translated faster and more accurately than abstract words (de Groot, Reference de Groot, Frost and Katz1992), and in L2 vocabulary acquisition they are learned more quickly (de Groot & Keijzer, Reference de Groot and Keijzer2000) and retained better (van Hell & Mahn, Reference van Hell and Mahn1997; see Altarriba & Basnight-Brown, Reference Altarriba and Basnight-Brown2012, for discussion of further distinctions between emotion words and abstract words). These advantages of concrete over abstract words are collectively termed the “concreteness effect.”

Several theories have been developed to explain this effect. The dual coding theory suggests that abstract words are encoded with only verbal information, while concrete words are encoded with both verbal and visual information (Paivio, Reference Paivio1971). The context availability theory posits that concrete words have stronger associations to contextual information than abstract words (Schwanenflugel, Harnishfeger, & Stowe, Reference Schwanenflugel, Harnishfeger and Stowe1988). In addition, concrete words are thought to have more readily generated semantic features than abstract words (Jones, Reference Jones1985; Plaut & Shallice, Reference Plaut and Shallice1991). While the theories cited here differ in their particulars, and this is by no means an exhaustive list of theories, they all highlight key differences between abstract and concrete words that explain persistent difficulties in the acquisition and retrieval of abstract words.

Although the concreteness effect is well documented in the L1 and L2 psycholinguistic literature, few studies that focus on instructional techniques for L2 vocabulary acquisition have considered concreteness as a variable. Where this factor has been investigated, studies have focused on receptive vocabulary knowledge. Van Hell and Mahn (Reference van Hell and Mahn1997) manipulated concreteness as a factor in their comparison of rote rehearsal versus the keyword method on receptive recall in the L2. In both training conditions, translation accuracy on the recall task was significantly higher and speed of recall was faster for concrete versus abstract words, particularly at a 2-week delayed posttest. Zhao and Macaro (Reference Zhao and Macaro2016) similarly assessed learning outcomes for teacher-provided L1 translations versus L2-only definitions with concrete and abstract words. The treatment group, which received L1 translations of the target words, scored better on receptive recall of both concrete and abstract words than the comparison group, which received L2 definitions; however, as the concrete and abstract items were not matched on lexical characteristics, the authors could not compare the differential effects of the two learning conditions on item type. In addition, these studies focused on novel word learning, meaning that the participants had no previous knowledge of the target items. In contrast, the persons with aphasia tested in the studies by Sandberg and colleagues (Kiran et al., Reference Kiran, Sandberg and Abbott2009; Sandberg & Kiran, Reference Sandberg and Kiran2014) and the L2 learners in the current study had (at least partial) prior receptive knowledge of the items in the training.

To our knowledge, only Pichette et al. (Reference Pichette, De Serres and Lafontaine2012) have specifically investigated how different training techniques can influence the productive vocabulary knowledge of concrete versus abstract words. They found that an output-based learning condition (writing sentences containing the target item) led to better productive recall of both concrete and abstract words over a receptive learning condition (reading sentences containing the target item) at immediate and 1-week delayed posttests. Recall of concrete items was higher than of abstract items across both learning conditions, particularly at immediate posttest, while recall accuracy of all items declined from immediate to delayed posttest. Though these studies show that direct training increases the short-term knowledge of abstract words, these gains typically decline posttreatment, and the learning of abstract words lagged behind that of concrete words.

Aphasia treatment as a model for vocabulary training

A defining characteristic of aphasia is anomia, or the inability to retrieve words from the mental lexicon. Treatments for anomia often focus on strengthening the semantic system to increase activation of the target concept, and thus improve target word selection. One widely used therapy is semantic feature analysis (SFA), first applied to persons with aphasia by Boyle and Coelho (Reference Boyle and Coelho1995). The idea is that training the semantic features of a concept activates the network surrounding the target via spreading activation (Collins & Loftus, Reference Collins and Loftus1975). This increases the strength and specificity of the activation of the target concept, which in turn improves the likelihood of choosing the correct lexical label for the target concept (Coelho, McHugh, & Boyle, Reference Coelho, McHugh and Boyle2000). While this technique is very effective with directly trained items, results are mixed regarding the transfer of benefit to untrained items, hereafter referred to as generalization (Boyle, Reference Boyle2010).

One promising approach to promote generalization is to combine the basic procedure of SFA with stimulus selection based on the complexity account of treatment efficacy (CATE; Thompson, Shapiro, Kiran, & Sobecks, Reference Thompson, Shapiro, Kiran and Sobecks2003). Initially, Thompson et al. (Reference Thompson, Shapiro, Kiran and Sobecks2003) found that training more syntactically complex structures (e.g., object relative clauses) results in improvement for not only the directly trained structures but also simpler, related structures (e.g., object clefts) in persons with aphasia. Kiran and Thompson (Reference Kiran and Thompson2003) extended CATE to the semantic realm by training atypical items (e.g., ostrich, artichoke) in natural categories (e.g., birds, vegetables). In a series of studies using not only natural but also well-defined (e.g., shapes) and ad hoc (e.g., things at a garage sale) categories, Kiran found that persons with aphasia exhibited generalization to typical category exemplars when atypical exemplars were trained (see Kiran, Reference Kiran2008, for a review).

More pertinent to the current study, Sandberg and Kiran applied CATE to anomia treatment in aphasia using concreteness as a mode of complexity (Kiran et al., Reference Kiran, Sandberg and Abbott2009; Sandberg & Kiran, Reference Sandberg and Kiran2014). Abstract words are classified as the more complex items because they are less accurate, take longer to process, have fewer semantic features, have less contextual information associated with them, and are more difficult to conjure an image for than concrete words (Jones, Reference Jones1985; Paivio, Reference Paivio1971; Plaut & Shallice, Reference Plaut and Shallice1991; Schwanenflugel et al., Reference Schwanenflugel, Harnishfeger and Stowe1988). The word-finding training protocol used by Sandberg and Kiran (Kiran et al., Reference Kiran, Sandberg and Abbott2009; Sandberg & Kiran, Reference Sandberg and Kiran2014) is an adaptation of SFA, combined with the application of CATE via the use of abstract words as the more complex items. Specifically, the retrieval of abstract words (e.g., truth) within a context-category (e.g., courthouse) is trained through the selection and discussion of semantic features (e.g., is generally considered positive) for each target abstract word. Subsequent performance for the target abstract words is measured via a verbal fluency task within the context-category. Critically, predetermined target concrete words within the same context-category (e.g., jury) are not directly trained, but are still tracked to measure generalization to concrete words. In their studies, Sandberg and Kiran have found generalization to concrete words even when only abstract words are trained, but not vice versa.

The success of this training protocol in persons with aphasia merits exploration of applications to other populations, such as L2 learners. It is not uncommon to compare L2 learning and L1 relearning in aphasia (e.g., Cornelissen et al., Reference Cornelissen, Laine, Renvall, Saarinen, Martin and Salmelin2004; López-Barroso & de Diego-Balaguer, Reference López-Barroso and de Diego-Balaguer2017; Wray, Reference Wray2009). For example, Wray (Reference Wray2009) compared the use of formulaic language across different groups and found that both people with aphasia and L2 learners—particularly those L2 learners who struggled to learn target L2 vocabulary—relied on formulaic chunks and imitation during initial stages of vocabulary learning. López-Barroso and de Diego-Balaguer (Reference López-Barroso and de Diego-Balaguer2017) identified similar possible compensatory mechanisms between L2 learning and language rehabilitation in persons with aphasia, in that both populations recruit either the dorsal or the ventral speech processing streams to support success during language-related tasks.

An important parallel between the two populations is that creating form–meaning connections is a crucial element of vocabulary learning. In persons with aphasia, the connections between previously known forms (the orthographic and acoustic representations of a word) and semantic meanings must be reestablished (Jefferies & Lambon Ralph, Reference Jefferies and Lambon Ralph2006), while in adult L2 learners, meanings from the L1 semantic network must be connected to new L2 forms (Jiang, Reference Jiang2000; Kroll & Stewart, Reference Kroll and Stewart1994). While not identical processes, they share many characteristics, such as the interplay of already known and new information. A further parallel is that these connections may not be established equally at the receptive and productive levels. For L2 learners there is a gap between receptive and productive knowledge (Schmitt, Reference Schmitt2014) that may not be fully closed even with direct training (Lee, Reference Lee2003). Similarly, in several aphasic syndromes comprehension is relatively spared while lexical retrieval deficits in production are pronounced (Papathanasiou & Coppens, Reference Papathanasiou and Coppens2017).

The current study

The current study extends the training protocol from Sandberg and Kiran (Reference Sandberg and Kiran2014) to second language acquisition. As in a lot of research on persons with aphasia, this study was carried out using a multiple-baseline, single-subject research design. In single-subject research design, intervention is provided at the individual participant level, data are analyzed at the individual participant level, and each participant serves as her own control through the collection of baseline data prior to introducing the intervention (i.e., manipulation of the independent variable). The dependent (outcome) variable is repeatedly measured within and across different phases (e.g., baseline and intervention phases) to determine the effect of manipulating the independent variable on the dependent variable. When the manipulation of the independent variable (e.g., introduction or withdrawal of treatment) results in a change in the dependent variable, the intervention is said to have an effect. Each time the introduction (or withdrawal) of an intervention phase results in changes in the dependent variable, whether within or across participants, the effect of intervention is said to be replicated. It is this replication that establishes methodological rigor. This design is ideal for understanding the specific participant characteristics and environmental conditions under which an intervention may or may not be effective (Kratochwill et al., Reference Kratochwill, Hitchcock, Horner, Levin, Odom, Rindskopf and Shadish2010).

This study design also addresses a typical shortcoming in L2 vocabulary acquisition studies, which is that learning outcomes are often measured only one or two times, and frequently only immediately following treatment. Consequently, any claims about longer term learning are speculative at best (Schmitt, Reference Schmitt2010). The design of the current study circumvents this issue, as the generative naming task continues to be administered as a probe even after training in a particular context-category has ended and training in the next context-category begins. Therefore, it is possible to assess the retention of productive vocabulary knowledge across a medium-term time span after the introduction of new information via the training paradigm. This is analogous to a typical L2 classroom, in which learners ideally maintain previously acquired vocabulary while being continuously confronted with new topics and vocabulary.

In persons with aphasia, training the retrieval of abstract words (e.g., truth) in a specific context-category (e.g., courthouse) by training their semantic features (e.g., is generally considered positive) results in both a direct training effect, in which retrieval of the trained abstract words improves, and a generalization effect, in which the retrieval of related but untrained concrete words (e.g., jury) also improves. Conversely, training the retrieval of concrete words results in a direct training effect but no generalization effect to untrained abstract words in that context-category. If this training works similarly in L2 acquisition, we predict

1. 1. Training abstract words in a context-category will improve production of the trained abstract words and generalize to the untrained concrete words within the same context-category, but will not promote improvement of words in untrained context-categories.

2. 2. Training concrete words in a context-category will improve production of the trained concrete words, but will not show generalization to the untrained abstract words within the same context-category.

To test the effectiveness of the training paradigm from Sandberg and Kiran (Reference Sandberg and Kiran2014) with L2 learners, in Experiment 1 we trained American English L2 learners of Spanish on abstract Spanish words in two context-categories (restaurant and university) and tracked their productive vocabulary knowledge for abstract and concrete words in these two categories prior to, during, and after training. In Experiment 2, we trained a new group of American English L2 learners of Spanish either on abstract words or concrete words in the context-category restaurant to replicate the findings from Experiment 1 and to investigate whether training concrete words rather than abstract words leads to generalization (cf. Kiran et al., Reference Kiran, Sandberg and Abbott2009). In Experiment 3, we trained Spanish L2 learners of English on abstract words in the context-category restaurant, to further replicate findings from Experiment 1 with a larger population with a different L2; we also included a translation recognition task prior to training to systematically investigate how preexisting receptive L2 vocabulary knowledge impacts the effectiveness of this training paradigm.

Experiment 1

Method

Design

This experiment used a single-subject design, with a baseline phase, two training phases, a posttesting phase, a delayed posttest, and a control condition. The order of training phases was counterbalanced across participants. Each participant served as his own control, with the opportunity for replication within and across participants.

Participants

Three American English L2 learners of Spanish (all female) were recruited from intermediate-level Spanish content courses at Penn State University. All participants completed a language background questionnaire (Li, Zhang, Tsai, & Puls, Reference Li, Zhang, Tsai and Puls2014). All participants started learning Spanish after age 7 at school. See Table 1 for complete biographical information.

Table 1. Biographical and language background information for all participants

Materials

Context-Category Selection

Context-categories and the vocabulary for each category were selected from survey data previously collected from L1 Spanish speakers. Each L1 Spanish speaker generated as many abstract and concrete words as possible within eight categories that represent different contexts in which a variety of abstract and concrete words are thematically related. These speakers also provided a rating of the cultural relevance of each category. For this study, we selected context-categories (a) with a high cultural relevance rating, (b) for which there were a large number of abstract and concrete words produced by at least two survey respondents, and (c) that are normally included in Spanish language learning curricula. We selected restaurant and university as trained context-categories and soccer as the control context-category.

Vocabulary Selection

For each context-category (restaurant, university, and soccer), we selected 15 abstract and 15 concrete target words (see Appendix A, Table A.1 for a full list of target words in each context-category). Target words were selected from the L1 Spanish survey data and from Spanish textbooks, in collaboration with L1 Spanish speakers from the Spanish Department at the same university from which the participants were recruited. Abstract words were defined as existing only in the mind or those that are theoretical. Concrete words were defined as existing in reality, or concepts that can be experienced by the senses. Abstract words had a concreteness rating of 400 or lower and concrete words had a concreteness rating of 500 or higher. Within each context-category, abstract and concrete words significantly differed in imageability and concreteness (ps < .001) but were balanced on frequency and familiarity (ps > .1). In addition, each context-category was balanced on concreteness, imageability, familiarity, and frequency (ps > .2). All norms were obtained in English from the MRC Psycholinguistic database (Coltheart, Reference Coltheart1981), except frequency, which was obtained from the SUBTLEX databases for American English (Brysbaert, New, & Keuleers, Reference Brysbaert, New and Keuleers2012) and Spanish (Cuetos, Glez-Nosti, Barbón, & Brysbaert, Reference Cuetos, Glez-Nosti, Barbón and Brysbaert2012). There were no differences between the English and Spanish frequencies within each context-category (ps > .3).

Feature Selection

A total of 45 semantic features were used in training for each participant. The majority of these features were developed by Sandberg and Kiran (Reference Sandberg and Kiran2014). They created 17 general features using dictionary definitions of abstract and concrete (e.g., exists outside the mind; is an object) and perceptual characteristics (e.g., can be touched; can be tasted). Fifteen distractor features (e.g., is poisonous) were taken from a previous study (Kiran, Reference Kiran2008). Alongside these 32 standard features, 13 individualized, category-specific features were generated by each participant during the first training session for each context-category.

Procedure

Prior to training, participants completed three baseline vocabulary probe sessions (approximately 15 min in length), at least 1 day apart, but no more than 2 weeks apart. After baseline, participants completed six training sessions for each trained context-category (each approximately 30- to 45-min long), at least 2 days apart, but no more than 1 week apart.Footnote ¹ At each training session, participants completed five different steps, outlined in greater detail below. During training in the first context-category, the second context-category continued to be baselined. Participants switched to the second trained context-category after they had completed six training sessions in the first context-category. During training of the second context-category, the first context-category continued to be posttested. Vocabulary probes were administered at the beginning of each training session, as well as at delayed posttest, which was approximately 2 weeks after the conclusion of all training for the first two participants and approximately 4 weeks after all training for the third participant.

Vocabulary Probes

Productive vocabulary in the three context-categories (restaurant, university, and soccer) was tested through a generative naming task. Participants were instructed to name as many concrete and abstract words as they could within each context-category in 2 min. Responses were divided into four categories: (a) target concrete words (e.g., camarero “waiter,” mantel “tablecloth”) (b) target abstract words (e.g., satisfecho “satisfied,” calidad “quality”) (c) other concrete words (e.g., vela “candle,” propina “tip”), and (d) other abstract words (e.g., orden “order,” felicidad “happiness”). Responses were counted as correct if they were clear and intelligible productions of the target word or a semantic variant of the target word (e.g., satisfacción “satisfaction” for satisfecho “satisfied”). Other abstract and other concrete words were words that were appropriate members of the particular context-category but were not one of the target items. Any responses that did not fit into the four categories listed above were not counted. All sessions were audio recorded for reliability.

Training

Each training session consisted of five steps: category sorting, feature selection, yes/no feature questions, synonym generation and concreteness judgment, and free generative naming. These steps were carried out using Qualtrics survey software.

Category sorting

Each participant was presented with 30 (15 abstract and 15 concrete) words from the trained context-category (restaurant or university) and the control context-category (soccer) on the left-hand side of the computer screen in a random order. The participant then sorted the words into the two context-categories by using a mouse to drag the word from the list to a box under the appropriate category heading. If a word was sorted incorrectly, the examiner brought it to the attention of the participant, who self-corrected the error. Feedback was not needed after the first session.

Feature selection

For each trained word, the participant was presented with 45 features (17 general features, 13 category-specific features generated by the participant in the first training session, and 15 unrelated distractor features) in random order at the left-hand side of the computer screen. The participant went down the list and selected the first 6 most descriptive features for each word by using a mouse to drag the feature to a box located under the target word on the right-hand side of the screen.

Yes/no feature questions

The experimenter asked each participant 15 questions about each abstract word using the 45 features. Five questions were taken from each of the feature categories (e.g., ¿Esto está asociado con un precio más alto? “Is this associated with a higher cost?”). Care was taken to present each feature an approximately equal number of times for each word across sessions.

Synonym generation and concreteness judgment

Each participant was prompted to recall each word, provide a synonym for the word, and judge whether the word was abstract or concrete.

Free generative naming

Each participant was prompted to name as many words as she could in the trained context-category, with no time limit. During this step, participants were prompted to try to recall the words they had trained that day, as well as any other words they could think of.

Training Effect Size Calculation

Effect sizes (ES) for both trained and untrained (generalized) items were calculated for each participant using a variation of Cohen’s d statistic as proposed by Beeson and Robey (Reference Beeson and Robey2006). The mean of the baseline probe scores was subtracted from the mean of the posttreatment probe scores, and then divided by the standard deviation (SD) of the baseline probe scores. In cases where the SD was 0, the smallest SD for that participant was used. The resulting ES were interpreted based on guidelines by Beeson and Robey (Reference Beeson and Robey2008) for trained (small > 6.5, medium > 8, large > 9.5) and untrained (small > 2, medium > 5, large > 8) items. Although subject-level ES is the primary measure in single-subject research design, the average ES across participants was also calculated to estimate overall efficacy.

Reliability

Interrater reliability was performed on at least 25% of the probes. In Experiment 1, the exact percentage was 37%. A research assistant who did not provide the training, and who spoke both English and Spanish, listened to the audio recordings and scored the probes. A third party then compared the original probe scores with the reliability probe scores and calculated percent agreement. In Experiment 1, the percent agreement was 95%.

Results

The results for each context-category for each participant are presented in Figure 1 and Table 2 provides all ES for each participant. As seen in Figure 1, participants generally produced more concrete words (light gray lines) than abstract words (dark gray lines) across all three context-categories in the baseline phase. Critically, across both trained context-categories (i.e., restaurant and university) there was little sustained improvement in either concrete or abstract word production during the baseline phase, prior to training. After the onset of training, there is a significant increase in target abstract word production across all three participants in both context-categories, with smaller increases in concrete word production. Of the three participants, two (P1 and P2) showed large ES for the trained abstract words in both trained context-categories (restaurant, university), and one (P3) showed a small ES in the trained context-category university. Of the two participants who showed training effects in the context-category restaurant, one (P2) showed a small generalization effect to untrained concrete words. All three participants showed training effects in the context-category university, and one (P1) showed generalization to untrained concrete words. Across participants, there were large average ES for the trained abstract words in both trained context-categories (13.6 and 10.2) and a small generalization ES for concrete words in restaurant (3.6). Of interest, both P1 and P2 showed improvement for abstract words in the untrained context-category soccer, and all three participants showed improvement for concrete words in the context-category soccer. Across participants, there was a small average generalization ES for abstract words in soccer (4.7) and a medium generalization ES for concrete words in soccer (7.5). All three participants also showed maintenance of the training effects 2–4 weeks after training ended, with only a 0%–13% drop in accuracy across participants.

Figure 1. Graphs of participant performance across phases for each context-category in Experiment 1. Each column represents data from a single participant. The top graph is the context-category that was trained first, the middle graph is the context-category that was trained second, and the bottom graph is the control context-category. Solid lines indicate the start and end of training. The dotted line separates the final posttest from the delayed posttest. Performance on abstract words is shown in dark gray, and performance on concrete words is shown in light gray. B = baseline phase; T = training phase; P = posttesting phase; DP = delayed posttest.

Table 2. Effect sizes for target items in each context-category for each participant in Experiment 1

Note: *Small effect size. **Medium effect size. ***Large effect size.

Experiment 2

Results

The results for the context-category restaurant for each participant are presented in Figure 2 and Table 3 provides all ES for each participant. As in Experiment 1, participants generally produced more concrete words (light gray lines) than abstract words (dark gray lines) in the baseline phase, and there was little sustained improvement in either concrete or abstract word production during the baseline phase, prior to training. After the onset of training, there is a significant increase in target word production.

Figure 2. Graphs of participant performance across phases for the trained context-category restaurant in Experiment 2. Graphs on the left are from the participants who were trained on concrete words; graphs on the right are from the participants who were trained on abstract words. Solid lines indicate the start and end of training. Performance on abstract words is shown in dark gray, and performance on concrete words is shown in light gray. B = baseline phase; T = training phase; P = posttesting phase.

Table 3. Effect sizes for target items in each context-category for each participant in Experiment 2

Note: *Small effect size. **Medium effect size. ***Large effect size.

All four participants who were trained on abstract words in the context-category restaurant (P1, P3, P4, P5) showed medium to large direct training ES. Two participants (P3, P4) also exhibited small generalization effects for concrete words in the context-category restaurant. Across participants, the average ES for the trained abstract words was large (15.6), while the average generalization ES for concrete words in the trained category approached the small threshold (1.8). In contrast, although all three participants who were trained on concrete words in the context-category restaurant showed some improvement, only one participant (P6) achieved a medium direct training ES. None of these participants showed generalization to abstract words in the trained context-category. The average ES across participants did not reach the small threshold for trained concrete words (5.9) and was zero for untrained abstract words.

Turning to the control categories, P1, who improved on trained abstract words in the context-category restaurant, also showed a small effect for untrained abstract words in the unexposed control context-category university and a large effect for untrained abstract words in the exposed control context-category soccer. P3 and P5, who also improved on trained abstract words in the context-category restaurant, showed small effects for untrained concrete words in both the unexposed and the exposed control categories. P2, who did not improve on trained concrete words in the context-category restaurant, and P6, who did, both showed small effects on untrained concrete words in the exposed control context-category, soccer (see Appendix A, Figures A.1 and A.2 for the corresponding figures of the untrained context-categories university and soccer for each participant). Across all participants, average ES did not reach the small threshold for either abstract (1.0) or concrete (0.9) words in university, and just reached the small threshold for both abstract (2.0) and concrete (2.2) words in soccer.

Experiment 3

Results

Translation recognition

Participants’ accuracy on concrete and abstract restaurant words at baseline and posttest are presented in Table 4. Wilcoxon signed-rank tests revealed no significant difference in participants’ recognition of abstract versus concrete words at either baseline or posttest (baseline: Z = –0.99, p = .324, d = 0.41; posttest: Z = –0.63, p = .527, d = 0.40). However, they exhibited significant improvement in their recognition of both abstract and concrete words from baseline to posttest (concrete: Z = –2.54, p = .011, d = 1.79; abstract: Z = –2.20, p = .028, d = 1.40).

Table 4. Results of Wilcoxon signed-rank tests

Note: IQR, interquartile rank. CI, confidence interval.

Training outcomes

The results for the context-category restaurant for each participant are presented in Figure 3, and Table 5 provides all ES for each participant. As seen in Figure 3, participants generally produced more concrete words (light gray lines) than abstract words (dark gray lines) in the baseline phase and there was little sustained improvement in either concrete or abstract word production during the baseline phase, prior to training. After the onset of training, there was a significant increase in word production in the trained context-category (restaurant) for the majority of the participants, with an average direct training ES of 9.4 (bordering on large), and an average generalization ES of 8.5 (large). Specifically, 7 of the 10 participants (P1, P3, P4, P5, P6, P7, P10) showed medium to large ES for the trained abstract words. All 7 participants who exhibited direct training effects also exhibited small to large generalization effects. The 3 participants who did not show improvement on the trained abstract words (P2, P8, P9) nonetheless showed improvement for the untrained concrete words in the context-category restaurant. While this pattern is not generalization per se, as it is not accompanied by a direct training effect with abstract words, it may indicate some general enhancement that occurred during training.

Figure 3. Graphs of participant performance across phases for the trained context-category restaurant in Experiment 3. Solid lines indicate the start and end of training. Performance on abstract words is shown in dark gray, and performance on concrete words is shown in light gray. B = baseline phase; T = training phase; P = posttesting phase.

Table 5. Effect sizes for target items in each context-category for each participant in experiment

Note: *Small effect size. **Medium effect size. ***Large effect size.

As seen in Table 5, only two participants (P3, P9), one of whom showed no direct training effects (P9), showed small ES in the unexposed control context-category university (see Figure A.3 in Appendix A for the corresponding figures for each participant). Across participants, the average ES for abstract and concrete words in the unexposed control context-category were 0.0 and 0.5, respectively. In comparison, all 10 participants showed improvement for concrete words in the exposed control context-category soccer, with an average ES of 6.0 (medium). Seven participants (P1, P3, P4, P5, P7, P9, P10), one of whom showed no direct training effects (P9), showed improvement for abstract words in the exposed control context-category as well, and the average ES across all 10 participants was 5.6 (medium; see Figure A.4 in Appendix A for the corresponding figures for each participant). At the same time, the mean slopes of improvement for abstract and concrete words in the exposed control context-category soccer (abstract slope: M = 0.65, SD = 0.49; concrete slope: M = 0.75, SD = 0.39) were descriptively lower than those of the trained context-category restaurant (abstract slope: M = 0.93, SD = 0.98; concrete slope: M = 1.14, SD = 0.41). The steeper slope in the trained context-category suggests not only a larger improvement (as also evidenced by the larger ES) but also a more immediate effect.

General Discussion

In all three experiments, the majority of participants showed the predicted pattern of improved production of the trained abstract words within the trained context-category and at least small generalization effects to the untrained concrete words in the same context-category. This effect emerged in both American English L2 learners of Spanish and Spanish L2 learners of English, demonstrating that these effects are not unique to a specific population. In Experiment 2, when concrete words were trained, generalization to abstract words was nonexistent. In Experiment 3, results from a translation-recognition test revealed that participants had receptive knowledge of the trained items prior to training, and that receptive vocabulary for both abstract and concrete words improved in posttraining measures. Each of these findings will be addressed in detail below.

Paralleling studies of word-finding therapy in aphasia (Kiran et al., Reference Kiran, Sandberg and Abbott2009; Sandberg & Kiran, Reference Sandberg and Kiran2014), across all three experiments, training abstract words led to improved production of the trained abstract words and this generalized to improved production of untrained concrete words. Those participants directly trained on concete words in Experiment 2 exhibited no generalization effects to the subsequent production of abstract words, also paralleling previous research on persons with aphasia (Kiran et al., Reference Kiran, Sandberg and Abbott2009). Further, it is worth noting that while all three participants who were trained on concrete words in Experiment 2 made some improvement, only one participant achieved a significant direct training ES for concrete words, suggesting that even direct training effects are smaller when participants are trained on concrete versus abstract words. Together, these results are in line with CATE (Thompson et al., Reference Thompson, Shapiro, Kiran and Sobecks2003), in that training more complex items (i.e., abstract words) results in generalization to less complex, related items (i.e., concrete words), but not vice versa.

The application of the CATE model to abstract word learning relies on the assumption that concepts are stored within a semantic network with weighted connections to other, related concepts, and that when one concept in the network is activated during language comprehension or production, related items are also activated through spreading activation (Collins & Loftus, Reference Collins and Loftus1975). Within the semantic network, abstract words generally have fewer semantic features (Plaut & Shallice, Reference Plaut and Shallice1991), and are more semantically diverse (i.e., are found in more contexts and have more variability in meaning) than concrete words (Hoffman, Lambon Ralph, & Rogers, Reference Hoffman, Lambon Ralph and Rogers2013). This semantic diversity means that abstract concepts are weakly connected to a large number of other concepts within the network, with more spreading activation, while concrete concepts have strong and specific representations, with less spreading activation. These stronger and more specific representations for concrete concepts lead to faster and more accurate retrieval of lexical forms relative to abstract concepts during language production, even among nonclinical populations (Newton & Barry, Reference Newton and Barry1997). However, in terms of generalization, the specificity of concrete words, which limits spreading activation, may also limit generalization. While generalization has been observed after concrete word training, it appears to be limited to concrete words that share semantic features with the trained items (e.g., Kiran, Reference Kiran2008).

Though the underlying lexical–semantic network likely differs between the L1 in persons with aphasia and the L2 in adult language learners, there may be similarities that lead to parallel findings between L1 aphasia research and the results of the current study. In aphasia, concepts are not lost, but retrieving the accurate lexical form during language production is difficult due to brain injury. In L2 learners, established L1 concepts have acquired a new L2 lexical mappings that are often less stable, especially at lower proficiency levels (Kroll & Stewart, Reference Kroll and Stewart1994). In both cases, regardless of potential differences in the underlying organizational structure, the connections between lexical forms and their corresponding concepts are weak. The training paradigm developed by Kiran and Sandberg (Kiran et al., Reference Kiran, Sandberg and Abbott2009; Sandberg & Kiran, Reference Sandberg and Kiran2014) targets these lexical–conceptual connections. By training semantic features, the conceptual representation is strengthened, allowing for faster and more accurate retrieval of the lexical form (Newton & Barry, Reference Newton and Barry1997). Because abstract concepts are weaker and more diverse to begin with, this training method is particularly effective for boosting productive knowledge of abstract words. An added benefit of this type of training is that because of the broad associative network for abstract concepts, the connections between the trained abstract concepts and other concepts in the semantic network are similarly strengthened through spreading activation, leading to improvement on retrieval of the untrained, but related words in the same context-category (see Kiran et al., Reference Kiran, Sandberg and Abbott2009; Sandberg & Kiran, Reference Sandberg and Kiran2014).

In Experiment 3, we included a receptive translation-recognition vocabulary task, the results of which showed that participants had significant receptive knowledge of the trained items prior to beginning the training intervention, even though their performance in the timed generative naming task was low at baseline, particularly for abstract words. This is not unexpected, as receptive vocabulary size is typically larger than productive vocabulary size (Nation, Reference Nation2013; Webb, Reference Webb2008). This also suggests that the participants may have already had some knowledge of the forms and meanings of the L2 target words, but did not yet have the ability to actively recall and produce them in a controlled context, namely, associations within a context-category (Laufer & Nation, Reference Laufer and Nation1999; Nation, Reference Nation2013). We also found that receptive knowledge of the target words improved as a result of the abstract word training, suggesting that among L2 learners, the benefits of this training extend to receptive vocabulary knowledge. However, it must be noted that the task we used (translation recognition) represents the most basic form of receptive knowledge (passive recognition, according to the taxonomy outlined in Laufer & Goldstein, Reference Laufer and Goldstein2004). It is uncertain whether performance at pretest—and subsequent improvement at posttest—would be similarly high had we used a different measure of receptive knowledge that required a higher level of skill.

The fact that the participants in Experiment 3 exhibited a high level of receptive vocabulary knowledge at baseline also raises the question of whether this training would be effective for learners at lower L2 proficiency levels, who have an overall smaller receptive vocabulary size and who are less likely to have any preexisting knowledge of L2 words in a particular context-category. Preliminary results from a follow-up study in an L2 classroom setting suggest that this training paradigm can also lead to increased productive knowledge of abstract and concrete vocabulary among less-proficient L2 learners (Kerschen, Sandberg, Carpenter, & Jackson, Reference Kerschen, Sandberg, Carpenter and Jackson2018).

Unexpectedly, participants in all three experiments also showed improvement outside of the trained context-category, mainly in the exposed control context-category soccer. This is most likely due to the inclusion of this context-category during the category sorting step of training. In this task the participants were exposed to the target abstract and concrete words in soccer and had to process them for meaning in order to place them into the appropriate category. They also received feedback on their responses until all words had been sorted correctly. Therefore, this task provided some training of the form and meaning of the target words in soccer, which could have led to the acquisition of and/or activation of existing knowledge for these words. Evidence to support this interpretation comes from Experiments 2 and 3, in which little improvement was found on an unexposed control context-category (university), where participants were not exposed to any target words at any point during training. Of importance here, both university and soccer were tested equally often during the baseline and testing probes and both categories likely had a similar level of baseline familiarity the learners, suggesting that the improved performance in the context-category soccer was not simply the result of repeated testing. This improvement was also surprisingly robust, with multiple participants showing large ES, suggesting that the category sorting task played a key role in the observed training effects. Nonetheless, this small amount of training for soccer differed in quantity and quality from the full training for the context-category restaurant, and results from all three experiments showed that improvement on the generative naming task was greater in the directly trained context-category. However, these findings suggest that future research should test individual training components separately, and more accurately measure receptive knowledge of target words in the control categories prior to training, to help identify how each task contributes to the development of productive knowledge of target vocabulary.

An alternative explanation for the training effects observed in this study is that prior to training, participants had the productive knowledge, but failed to produce target words because they were not considered category exemplars and that training increased the scope of the category. While plausible, one argument against this explanation is that target words were chosen based on consensus and typical vocabulary for L2 learners, and the participants were specifically prompted to name both concrete and abstract words in the instructions for the vocabulary probes both prior to and after training. In addition, in Experiment 2, participants who were trained on concrete words saw the abstract words during category sorting and were therefore cued that these words belong in the category restaurant. However, they did not end up producing any of the target abstract words by the end of training, which one would have predicted to occur if the training effects were simply the result of participants learning to increase the scope of possible category exemplars over time.