3.1 Simulating the effect of context pictures in forward and backward translation

The first simulation concerned forward (L1 to L2) and backward (L2 to L1) translation in the context of semantically related and unrelated pictures. For example, the model had to translate the Dutch written word KONIJN into English rabbit (forward translation) or the English word RABBIT into Dutch konijn (backward translation). Figure 3 shows the simulation results together with the empirical data of La Heij et al. (Reference La Heij, Hooglander, Kerling and Van der Velden1996).

Figure 3. Semantic effect of context pictures in forward (L1 to L2) and backward (L2 to L1) translation of words: empirical data (La Heij et al., Reference La Heij, Hooglander, Kerling and Van der Velden1996, Experiment 4) and WEAVER++ simulation results. Shown is the latency in the semantically related condition minus the latency in the unrelated condition. A negative difference indicates semantic facilitation.

In the model, a semantically related context picture (e.g., a picture of a dog) will activate the target lexical node via the network connections (e.g., rabbit in the forward translation of KONIJN and konijn in the backward translation of RABBIT), whereas unrelated context pictures (e.g., a picture of a chair) will not activate the target lexical node. Lexical competition is restricted to the target language (i.e., English in forward translation and Dutch in backward translation). Because pictures activate lexical nodes regardless of language, there will be some competition induced by the picture names (e.g., English dog in forward translation and Dutch hond in backward translation). However, the simulation results shown in Figure 3 demonstrate that the facilitatory effect from the context picture may outweigh the lexical competition that it induces.

3.2 Simulating the differential effect of context pictures and words

The second simulation concerned backward word translation (L2 to L1) in the context of semantically related and unrelated pictures and words. For example, the model had to translate the English word RABBIT into Dutch konijn in the context of a picture of a dog or an unrelated picture, or the model had to perform the translation in the context of the Dutch word HOND (meaning “dog”) or an unrelated word. Figure 4 shows the simulation results together with the empirical data of Bloem and La Heij (Reference Bloem and La Heij2003).

Figure 4. Semantic effect of context words and pictures in backward (L2 to L1) translation of words: empirical data (Bloem & La Heij, Reference Bloem and La Heij2003, Experiment 1) and WEAVER++ simulation results. Shown is the latency in the semantically related condition minus the latency in the unrelated condition. A negative difference indicates semantic facilitation and a positive difference indicates semantic interference.

In the model, a semantically related context picture (e.g., a picture of a dog) will activate the target lexical node (e.g., konijn in backward translating RABBIT), whereas unrelated context pictures (e.g., a picture of a chair) will not. Although words within a language compete, the facilitatory effect from the context picture may outweigh the interference from the lexical competition, as we just saw. This surfaces as semantic facilitation in the RTs. In contrast, a semantically related Dutch context word (e.g., HOND) will activate the target word node (e.g., konijn in backward translation of RABBIT), whereas an unrelated context word (e.g., STOEL, meaning “chair”) will not. Again, only words within a language compete. However, activation of a competitor lexical node (e.g., hond) by a context word (i.e., HOND) will be much higher than the activation of a competitor lexical node (e.g., hond) by a context picture (e.g., of a dog). The simulation results shown in Figure 4 demonstrate that the facilitatory effect of a semantically related context word may be insufficiently large to outweigh the interference from the lexical competition induced by the context word. Consequently, context words yield semantic interference in the RTs, as empirically observed.

3.3 Language selectivity

Although activation of the English lexical node (e.g., rabbit) of the to-be-translated English word (RABBIT) was always much higher than that of the Dutch target lexical node (i.e., konijn) in the simulations, the Dutch lexical node was nevertheless correctly selected. This was achieved by the condition-action rules, which restricted competition to the lexical nodes of the target language. Thus, the rules allow for the selection of words in the appropriate language even though words in the irrelevant language are more highly activated. Alternatively, it is possible to reduce or eliminate the competition of words in the irrelevant language by inhibition (e.g., Green, Reference Green1998; Kroll, Bobb, Misra & Guo, Reference Kroll, Bobb, Misra and Guo2008) rather than by language-specific selection. Costa and Santesteban (Reference Costa and Santesteban2004) argued that balanced bilinguals have developed a mechanism that allows for language-specific selection, whereas non-balanced bilingual speakers use inhibition. However, the empirical evidence for inhibition as the mechanism to achieve language selectivity is mixed (e.g., Finkbeiner, Almeida, Janssen & Caramazza, Reference Finkbeiner, Almeida, Janssen and Caramazza2006; Verhoef, Roelofs & Chwilla, Reference Verhoef, Roelofs and Chwilla2009; but see Kroll et al., Reference Kroll, Bobb, Misra and Guo2008).

La Heij and colleagues did not include an English lexicon in their simulations, thereby not addressing the issue of how a Dutch lexical node is selected in the face of the activation of the English lexical node corresponding to the to-be-translated word. In the WEAVER++ simulations, both a Dutch and an English lexicon (i.e., an integrated Dutch–English lexicon) was included (illustrated in Figure 2). Although activation of English lexical nodes (in addition to Dutch nodes) was part of the translation process (i.e., the to-be-translated English word provided the source of activation for the translation process), English lexical nodes did not compete for selection in the case of Dutch targets. The issue of how a Dutch word is selected in the face of the activation of English words was addressed by assuming that language-specific selection is achieved through production rule application. These rules only considered words of the target language for selection (based on the nodes indicating language membership).

The assumption of language-specific selection in word translation readily explains empirical findings from Miller and Kroll (Reference Miller and Kroll2002) on the effect of matching or mismatching target and context languages, although Miller and Kroll did not interpret their findings in terms of language-specific selection (i.e., Kroll and colleagues assume inhibition as the mechanism, e.g., Kroll et al., Reference Kroll, Bobb, Misra and Guo2008). Miller and Kroll asked English–Spanish bilingual speakers to translate written words from L2 to L1 (backward translation) or from L1 to L2 (forward translation) while trying to ignore written context words in the language of input (i.e., the to-be-translated word) or the language of output (i.e., the target language, as in Bloem & La Heij, Reference Bloem and La Heij2003; Bloem et al., Reference Bloem, Van den Boogaard and La Heij2004). When the language of output and the language of the context word were the same (i.e., either English or Spanish), semantic interference was obtained (i.e., RT semantic > RT unrelated), corresponding to what La Heij and colleagues obtained (i.e., Bloem & La Heij, Reference Bloem and La Heij2003; Bloem et al., Reference Bloem, Van den Boogaard and La Heij2004). However, when the language of the context word corresponded to the input language (i.e., the to-be-translated and context words were either English or Spanish), no semantic effect was obtained (i.e., RT semantic = RT unrelated). These findings are readily explained by assuming language-specific selection. The lexical node corresponding to the context word will compete for selection when the language of output and the context word are the same, yielding semantic interference. However, the lexical node corresponding to the context word will not compete for selection when the language of output and the context word are different, yielding no semantic effect.

To summarize, in the simulations, semantic facilitation was obtained from context pictures in both forward and backward translation, as empirically observed (La Heij et al., Reference La Heij, Hooglander, Kerling and Van der Velden1996). Moreover, context words yielded semantic interference, as empirically observed (Bloem & La Heij, Reference Bloem and La Heij2003; Miller & Kroll, Reference Miller and Kroll2002). The outcomes of the simulations demonstrate that an important empirical observation taken to be in favor of discrete-flow models is, in fact, only consistent with those models and equally compatible with continuous-flow models.

3.4 Cattell's legacy

As mentioned previously, Cattell (Reference Cattell1887) made a number of seminal observations on language performance. In particular, he observed that it takes longer to name pictures than to read words aloud, longer to name pictures in a foreign than the native language (depending on the familiarity with the languages), and longer to orally translate than to read words aloud. These findings have been replicated and extended in later studies (e.g., La Heij et al., Reference La Heij, Hooglander, Kerling and Van der Velden1996; Roelofs, Reference Roelofs2006). The findings are readily explained by modern models of picture naming, word reading, and word translation.

In WEAVER++, picture naming takes more time than reading, because picture naming has to proceed via the conceptual level, whereas word reading may proceed via the lexical level without engaging the conceptual level (see Roelofs, Reference Roelofs2003, Reference Roelofs2006). As a consequence, the processing pathway through the lexical network is longer for picture naming than for word reading, and naming takes more time than reading, as empirically observed. Moreover, it takes longer to name pictures in a foreign than the native language (depending on the familiarity with the languages), because of differences in the strength of connections between nodes in the lexical network. For example, if the connection strength between concept nodes and lexical nodes is weaker for a foreign than the native language in the model (as assumed for L1 and L2 in the simulations reported above), then it takes longer to name pictures in the foreign than the native language, as empirically observed. Finally, word translation takes more time than word reading, because translation proceeds via the conceptual level, whereas word reading may proceed via the lexical level without engaging the conceptual level (see Roelofs, Reference Roelofs2003, Reference Roelofs2006). Moreover, whereas word translation engages two lexical items (e.g., English rabbit and Dutch konijn, see Figure 2), word reading only involves a single lexical item. The involvement of two lexical items diminishes the activation flow somewhat. In addition, word reading may often be mediated by the application of rules mapping graphemes onto phonemes (see Coltheart et al., Reference Coltheart, Rastle, Perry, Langdon and Ziegler2001), whereas word translation may not (i.e., there are no rules that map the graphemes of English RABBIT onto the phonemes of Dutch konijn). As a consequence, word translation will take more time than word reading.

4.1 Robustness of the context effects

In a recent study, Schwieter and Sunderman (Reference Schwieter and Sunderman2009) failed to replicate the semantic interference of context words obtained by Bloem and La Heij (Reference Bloem and La Heij2003). In the experiment of Schwieter and Sunderman, English–Spanish speakers translated Spanish (L2) words into English (L1) in the context of pictures or English words. They obtained semantic facilitation of context pictures, but no effect of context words, regardless of whether the participants were low- or high-proficiency English–Spanish speakers. This suggests that the semantic facilitation effect of context pictures is robust, but the semantic interference effect of context words is not. Perhaps one may argue that the failure to replicate the semantic interference of context words is due to a difference in materials or participants (i.e., Dutch–English for Bloem & La Heij vs. English–Spanish for Schwieter & Sunderman). However, this does not seem to be the case. Miller and Kroll (Reference Miller and Kroll2002) obtained semantic interference using English and Spanish context words and English–Spanish bilingual participants. Moreover, using the materials of Bloem and La Heij (Reference Bloem and La Heij2003) and testing Dutch–English bilingual participants, we obtained semantic facilitation from context pictures, but also no effect of context words (Gerakaki, Reference Gerakaki2011; Roelofs, Dijkstra, Gerakaki & Piai, Reference Roelofs, Dijkstra, Gerakaki and Piai2012). That is, like Schwieter and Sunderman (Reference Schwieter and Sunderman2009), we could not replicate the semantic interference from context words in word translation.

One special feature of the experimental design of Bloem and La Heij (Reference Bloem and La Heij2003) is that the context words were presented in red letters, whereas the context pictures were presented in gray color. The target words were presented in black letters. All materials were presented on a white background. Schwieter and Sunderman (Reference Schwieter and Sunderman2009) also adopted this way of displaying the target and context stimuli in their experiment, and we did the same. However, by presenting the context words in red and the context pictures in gray, the context words seem to be made more visually salient than the context pictures. Perhaps the participants in the experiments of Bloem and La Leij (Reference Bloem and La Heij2003), Schwieter and Sunderman (Reference Schwieter and Sunderman2009), and our own study were differentially sensitive to the visual salience of the context words. For example, the red color of the context words may have made it easier for some participants to ignore the context words, thereby reducing their impact so that they yielded no effect, as observed by Schwieter and Sunderman (Reference Schwieter and Sunderman2009) and ourselves. Alternatively, the red color of the context words may have captured attention for some participants and made it more difficult to ignore the context words, thereby increasing the words’ impact so that they yielded semantic interference, as observed by Bloem and La Heij (Reference Bloem and La Heij2003). The ease of ignoring the context words (indexed by the value of the distractor duration parameter) also influences their impact in WEAVER++. In the simulations, semantic interference was obtained when the distractor duration was set at 125 ms, but semantic interference was absent when the duration was set at 100 ms.

We further empirically examined the possibility that the visual salience of the context words is responsible, at least in part, for the varying results. In a subsequent experiment, we made the to-be-translated words more salient by presenting them in green color, whereas the context words and pictures were presented in white color on a black background. Now, we obtained semantic facilitation from both the context pictures and the words, although the facilitation from the pictures was twice as large as that from the words. These findings suggest that the differential salience of the context words and pictures in the original experiments of Bloem and La Heij (Reference Bloem and La Heij2003) may have been a determinant of the differential direction of the semantic effects of context pictures (semantic facilitation) and context words (semantic interference). In terms of the continuous-flow model, manipulating the relative salience of the target and context words may cause the priming of the target word by the context word via the conceptual level to offset the interference from lexical competition induced by the context word.

In the experiments of Miller and Kroll (Reference Miller and Kroll2002), the to-be-translated word and the context word were indicated by relative timing. The to-be-translated word was always presented first and the onset of the context word was 200 ms or 500 ms later. It is possible that when the context word is presented after the to-be-translated word, it captures attention (just as presenting the context word in red may do, as suggested above). In terms of WEAVER++, attentional capture may increase the distractor duration parameter, which may yield semantic interference, as shown above.

To conclude, whereas the semantic facilitation from context pictures is reliably found, the semantic interference from context words is not so consistently obtained. Note that the semantic facilitation from context pictures was the finding that challenges continuous-flow models according to La Heij and colleagues.

4.2 Other effects of context pictures on word production in L1

Above, we demonstrated that a continuous-flow model can account for the semantic facilitation from context pictures and the semantic interference from context words in word translation. Next, we discuss other findings on effects of picture contexts on L1 word production in the literature, which specifically support the continuous-flow model and challenge the discrete-flow model.

If pictures activate their names only if a speaker wants to name them, then concept selection in word translation is critical for obtaining the semantic facilitation effect from context pictures. The effect should not be obtained when only lexical level information, such as a word's grammatical gender, needs to be selected (in WEAVER++, grammatical gender is specified at the lemma level, which corresponds to the lexical level in Figure 2). This prediction has been tested in experiments that exploited the linguistic fact that nouns take gender-marked articles in Dutch definite noun phrases, namely het with neuter gender and de with common gender (Roelofs, Reference Roelofs2003, Reference Roelofs2006). When a written noun is presented and participants have to read aloud the noun while preceding the noun by its gender-marked article (not visually presented), the grammatical gender of the noun needs to be retrieved to determine the right article, de or het. For example, if Dutch participants have to respond to the word HOND (“dog”) by saying “de hond”, the gender of the noun hond needs to be accessed to determine the correct determiner, de. If context pictures do not activate their names, as the discrete name-flow model holds, semantic facilitation of noun phrase production should not be obtained.

However, contrary to this prediction by the discrete-flow model (Bloem & La Heij, Reference Bloem and La Heij2003), semantic facilitation from context pictures was obtained in the experiments (Roelofs, Reference Roelofs2003, Reference Roelofs2006). For example, the RTs were smaller in producing “de hond” in response to the written word HOND in the context of a semantically related picture (e.g., of a rabbit) relative to an unrelated picture (e.g., of a chair). This finding suggests that activation cascades from concepts to lexical items, as assumed by the continuous-flow model. Moreover, in other experiments, context pictures yielded a gender congruency effect. Saying “de hond” to the written word HOND went faster when a semantically unrelated context picture had a name with the same grammatical gender as the word than when the gender of context picture and word differed (Roelofs, Reference Roelofs2006). The context pictures had no effect at all when the words were simply read aloud without article. These findings challenge the discrete-flow model.

Moreover, the discrete-flow model is challenged by evidence that suggests that context pictures yield phonological facilitation effects on naming target pictures (Morsella & Miozzo, Reference Morsella and Miozzo2002; Navarette & Costa, Reference Navarrete and Costa2005; Roelofs, Reference Roelofs2008a). Participants were given pictures in green superimposed onto pictures in red. The task was to name the pictures in green while ignoring the pictures in red. The picture names were phonologically related or unrelated. It was observed that target pictures were named faster when the context picture was phonologically related than when it was unrelated. This suggests that activation spreads continuously from the context picture to the phonological form of its name, unlike what the discrete-flow model assumes.

In response to these empirical challenges, Bloem et al. (Reference Bloem, Van den Boogaard and La Heij2004) suggested that lapses of attention, leading to an erroneous selection of the concept corresponding to the context picture and to activation of its name on some of the trials, explain the context effects of the pictures. To test this account, Roelofs (Reference Roelofs2007) examined the latency distributions of the responses in the earlier experiments. If the semantic and grammatical gender effects of context pictures on producing gender-marked noun phrases are due to an inadvertent activation of the context picture name on some of the trials, followed by a covert repair, the effects should be present for only a part of the latency distribution, namely for the slow responses only. Instead, if the context effects are due to a continuous flow of activation from concepts to words, the effects are expected to be present across the whole latency distributions. The distributional analyses revealed that the magnitude of the semantic effect increased with increasing RT, but importantly, the effect was present throughout the entire latency range. Similarly, the gender congruency effect was present throughout the entire latency range, except for the longest RTs. Again, the effect was robust and not due to an occasional odd trial. Moreover, the phonological effect of context pictures in picture naming was present across the RT distribution (Roelofs, Reference Roelofs2008a). The presence of the semantic, grammatical gender, and phonological effects across the whole latency distribution challenges the suggestion by Bloem et al. (Reference Bloem, Van den Boogaard and La Heij2004) that the effects are due to occasional lapses of attention. If this were the case, the context effects should have been present for the slowest responses only. To conclude, the semantic, gender congruency, and phonological effects of context pictures support a continuous-flow model and challenge the discrete-flow model.

To recapitulate, modern theoretical and computational models of word production account for Cattell's (Reference Cattell1887) seminal findings and later replications and extensions by making explicit assumptions about the underlying mental operations. Whereas most models assume a continuous flow of activation from concepts to lexical items, one prominent model assumes that the mapping of concepts onto words happens in a discrete fashion. Differential context effects of pictures and words on the time to translate words from L2 to L1 have been claimed to support a discrete-flow model. In the present article, we argued that, contrary to this claim, it remains unclear whether the discrete-flow model can account for word translation performance. We reported the results of computer simulations showing that a continuous-flow model (i.e., WEAVER++) accounts for key findings on context effects of pictures and words in translating from L2 to L1 and vice versa. Moreover, we discussed other findings concerning effects of context pictures on word production in L1, which support the continuous-flow model and challenge the discrete-flow model.

To conclude, we demonstrated that an important empirical observation on word translation (i.e., semantic facilitation from context pictures) previously taken to be in favor of discrete models is, in fact, only consistent with those models and equally compatible with more continuous models of word production. Continuous models are specifically and independently supported by other empirical evidence on the effect of context pictures on word production in L1.