Speech processing appears to be easy and automatic in a first language (L1), but it becomes more challenging when it comes to a second language (L2). Many studies have shown that (late) bilinguals encounter difficulties at different levels when processing L2 speech. For example, lexical access can be altered by phoneme perception (Pallier, Colomé & Sebastián-Gallés, Reference Pallier, Colomé and Sebastián-Gallés2001), speech segmentation is affected by L1 routines (Cutler, A., Mehler, J., Norris, D. & Segui, Reference Cutler, Mehler, Norris, Segui and Altmann2002) and speech-in-noise comprehension is reduced in L2 compared to L1 (Hervais-Adelman, Pefkou & Golestani, Reference Hervais-Adelman, Pefkou and Golestani2014). These phonological and acoustic impairments, in addition to the speed at which speech unfolds, render listening comprehension less fluent and effortful in L2. To compensate for their difficulties, late bilinguals seem to develop strategies such as relying more on visual cues than native speakers, for instance (Navarra & Soto-Faraco, Reference Navarra and Soto-Faraco2007; Wang, Xiang, Vannest, Holroyd, Narmoneva, Horn, Liu, Rose, deGrauw & Holland, Reference Wang, Xiang, Vannest, Holroyd, Narmoneva, Horn, Liu, Rose, deGrauw and Holland2011). Similarly, given that disfluent comprehension can affect anticipation processes (Corley, MacGregor & Donaldson, Reference Corley, MacGregor and Donaldson2007; MacGregor, Corley & Donaldson, Reference MacGregor, Corley and Donaldson2010), the effort of comprehending L2 speech, making L2 comprehension less fluent than in L1, may reduce (or even suppress) the use of processes that would take place during L1 sentence listening, such as anticipation processes. In the present paper we investigate whether late bilinguals can make online use of the sentence context to anticipate words during sentence listening.
Previous studies have revealed sentence context effects in late bilinguals, reflected by facilitation in word recognition (e.g., Chambers & Cooke, Reference Chambers and Cooke2009; Fitzpatrick & Indefrey, Reference Fitzpatrick and Indefrey2009; Lagrou, Hartsuiker & Duyck, Reference Lagrou, Hartsuiker and Duyck2012). For example, Chambers and Cooke (Reference Chambers and Cooke2009) presented English–French bilinguals with a display containing inter-lingual competitors (e.g., ‘chicken’ [‘poule’ /pul/ in French] and ‘pool’ /pul/) and unrelated distracters. The recording of the eye-movements revealed that bilinguals fixated more the inter-lingual competitor when it was possible within the preceding sentence context (e.g., ‘Marie va décrire la poule’, Marie will describe the chicken) than when it was not (e.g., ‘Marie va nourrir la poule’, Marie will feed the chicken). Similarly, FitzPatrick and Indefrey (Reference Fitzpatrick and Indefrey2010) concluded from an ERP study that cross-lingual effects are reduced in semantically constraining contexts. Their results showed a similar N400 component (an ERP component reflecting semantic integration difficulty) for semantically incongruent critical words, independently of whether the L1 translation of the L2 words had initial phonological overlap or not (e.g., ‘My Christmas present came in a bright-orange doughnut’; initial overlap with ‘doos’ meaning ‘box’ in Dutch). These conclusions are consistent with the more recent findings from Lagrou et al. (Reference Lagrou, Hartsuiker and Duyck2012) showing that inter-lingual homophone effects are reduced in semantically constrained sentences.
These studies converge in that they all show that word recognition is facilitated by the use of sentence context in L2 sentence listening. However, they do not allow differentiating whether sentence context facilitates word integration or word anticipation. As revealed by the ERP literature in L1, words that are expected from the sentence context provoke a reduced N400 effect (Kutas & Hillyard, Reference Kutas and Hillyard1980). This reduction can reflect facilitated integration of a word upon its presentation because it matches the semantic network activated by the sentence context (Kuperberg, Paczynski & Ditman, Reference Kuperberg, Paczynski and Ditman2011; Myers & O’Brien, Reference Myers and O’Brien1998; Paczynski & Kuperberg, Reference Paczynski and Kuperberg2012); alternatively, this reduction can reflect active processes allowing the anticipation of upcoming words prior to their presentation (DeLong, Urbach & Kutas, Reference DeLong, Urbach and Kutas2005; Neely, Reference Neely1977; see Lau, Holcomb & Kuperberg, Reference Lau, Holcomb and Kuperberg2013, for an extensive discussion on the debate). Importantly, the integration and anticipation processes are not incompatible, and actually most probably equally take place during sentence comprehension. Nevertheless, it is likely that comprehenders may rely more on one or the other depending on the complexity of the comprehension situation. One might say that the simple fact of processing an L2 already puts late bilinguals in a complex linguistic situation and may already affect anticipation processes. This assumption was discarded in a recent visual study showing that late bilinguals are able to anticipate upcoming words, at least in sentence reading (Foucart, Martin, Moreno & Costa, Reference Foucart, Martin, Moreno and Costa2014; but see Martin, Thierry, Kuipers, Boutonnet, Foucart & Costa, Reference Martin, Thierry, Kuipers, Boutonnet, Foucart and Costa2013). As mentioned earlier, however, listening to sentences in L2 does create a complex linguistic situation affecting comprehension fluency. Hence, the use of anticipation processes might be reduced (or absent) during speech processing. This does not mean that anticipation mechanisms are not available in L2 but rather that late bilinguals might not be able to apply them in complex situations. To verify this hypothesis we conducted the same ERP experiment used by Foucart et al. (Foucart, Ruiz-Tada & Costa, Reference Foucart, Ruiz-Tada and Costa2015).
In their study, Foucart et al. investigated word anticipation in L1 speech comprehension. They presented auditory highly constrained Spanish sentences to native speakers and time-locked the ERPs on the article preceding the critical noun. Importantly, the critical noun was muted to avoid overlapping effects. The results revealed an early (200—280 ms) and a late negativity (450–900 ms) for articles that mis-matched the gender of the expected nouns. The authors took the modulation of the magnitude of the N400 as the evidence that anticipation processes are at play during L1 speech processing. In addition, following the sentence listening task, they conducted a lexical recognition task that revealed that, although both ‘expected’ and ‘unexpected’ words were muted during the listening phase, ‘expected’ words were (falsely) recognized significantly more often than ‘unexpected’ words, and as often as ‘old’ words that were actually presented. The authors suggested that anticipation processes allow creating a memory trace of a word prior to presentation.
The present study
In the present study we conducted the same listening and lexical recognition tasks as in Foucart et al. (Reference Foucart, Ruiz-Tada and Costa2015) to obtain two indices of anticipation processes in L2. The main index of interest is the presence of anticipation processes during online speech processing examined using ERPs (the listening task). The secondary index is the consequence of anticipation processes on lexical recognition. We hypothesized that, if late bilinguals anticipate words during listening comprehension, a larger N400-like component should be observed for articles whose gender mis-matches that of the expected noun. On the other hand, if anticipation processes are reduced (or absent) during speech processing, no differences should be observed. On a secondary level, the results of the lexical recognition task should indicate whether anticipation processes in L2 allow the creation of a memory trace of the (muted) expected word, like in L1. If it is indeed the case, late bilinguals should (falsely) assess expected words as previously heard more often than unexpected words. On the other hand, if they rely more on integration processes during listening comprehension, expected words should be treated as ‘new’ words.
Method
Participants
Twenty-two French–Spanish late bilinguals (16 females) took part in the experiment (participants’ details are reported in Table 1). They had learned Spanish at school and lived in Spain. To participate, they were required to pass a language test (B2 level of the Common European Framework). Participants were also asked to self-assess their proficiency in Spanish for written/oral comprehension/production. They received oral and written information about the procedure. Written consent form was obtained from each participant.
Table 1. Participants’ details (N = 22)
The essential information regarding the method is reported here, the details are available in the Supplementary Material.
Materials and design
Fifty-two high-constrained sentences were designed in Spanish; they had two possible noun outcomes: an expected noun and an unexpected noun (a total of 104 experimental sentences). They were designed so that the expected and unexpected nouns differed in gender (e.g.: el tesoro [masculine] (the treasure) vs. la gruta [feminine] (the cave); see Table 2 for examples of sentences). Another 52 sentences were added as filler sentences and contained nouns that were matched in frequency and length to the experimental nouns and that were presented as ‘old’ words in the recognition task. In each experimental sentence, the noun was completely muted for 500 ms after article offset and then the sentence resumed normally until the end.
Table 2. Example of sentences
Listening phase
Procedure
Participants’ brain activity was recorded as they were listening to sentences silently and answered yes-no comprehension questions (after a third of the sentences) designed to ensure full attention (94% of correct answers; the answers were not further analysed). They were told they would be asked questions about the sentences after the listening phase (it was not specified that it was a lexical recognition task).
EEG recording and data analysis
Visual inspection of the grand mean (Figure 1) revealed a long lasting effect starting around 280 ms and lasting until 680 ms. This negativity was confirmed by a 2 ms-by-2 ms paired t-tests (run from 0 to 1000 ms) for the difference between the Expected and Unexpected conditions. Unstable differences (remaining below p = .05 for less than 30 ms) were discarded (Rugg, Doyle & Wells, Reference Rugg, Doyle and Wells1995). Analyses of variance (ANOVAs) were therefore conducted on the 280–680 ms time-window with the factors Expectation (expected vs. unexpected critical article) and Region defined as Frontal (F7, F3, FC5, FC1, Fz, F8, F4, FC6, FC2), Central (T3, C3, CP5, CP1, Cz, T4, C4, CP6, CP2) and Parietal (T5, P3 P1, O1, Pz, O2, P2, T6, P4)Footnote 1 . The Greenhouse-Geisser correction (Greenhouse & Geisser, Reference Greenhouse and Geisser1959) was applied to all repeated measures with greater than one degree of freedom; the corrected p value is reported.
Figure 1. ERP grand average. Left panel: Event-related potential results for the critical article of the sentence. Time zero indicates the presentation of the article. Black lines depict ERPs measured for expected articles; dotted lines depict ERPs measured for unexpected articles. ERPs measured over single channels at midline sites (Fz, Cz, Pz) and averaged channels at Frontal left (F7, F3, FC5, FC1), Frontal right (F8, F4, FC6, FC2), Central left (T3, C3, CP5, CP1), Central right (T4, C4, CP6, CP2), Parietal left (T5, P3, P1, O1) and Parietal right (O2, P2, T6, P4) regions. The grey rectangle represents the time-windows analysed. Negativity is plotted up. Right panel: Topographic distribution of the difference between the expected and unexpected conditions across the 280–680 ms time-window.
Lexical recognition task
Following the listening phase, participants were presented with the 52 ‘expected’ words, 52 ‘unexpected’ words, 52 ‘old’ words (from the filler sentences) and 52 ‘new’ words (equally matched in length and frequency with the words from the other conditions). Participants were presented with all the words, independently of the list they received in the listening phase (see Table 3). This was done to examine whether the presentation of an unexpected article would reduce the memory trace of a word expected from the context; for example, looking at Table 3, whether the word “tesoro” would be expected to the same extent from sentence Context 1 (expected article) than from Context 2 (unexpected article). Participants were instructed to indicate whether they had heard the word during the listening phase using the yes-no keys. To balance the number of yes-no answers, an extra 52 words were selected from the sentences heard in the listening phase (not analysed).
Table 3. Design of the lexical recognition task in relation to the sentences heard in the listening phase.
*The pirate had the secret map, but he never found the treasure/the cave he was looking for.
*I need to buy my ticket to go to London.
Results
EEG
The ANOVA revealed significant main effects of Expectation (F(1, 21) = 6.66, p = .02) and Region (F(2, 42) = 25.73, p <.001). These two factors did not interact with each other (F(2, 42) = 0.63, p = .53). These results indicate that unexpected articles generated a significantly more negative deflection than expected articles.
Lexical recognition task
Table 4 reports the results in terms of hits and false alarms for each condition (only 26 out of the 52 stimuli of the conditions ‘Old’ and ‘New’ were randomly selected for the analyses to balance the number of items across conditions). Analyses (t-test) were conducted on the percentage of words that were assessed as not heard during the listening phase (i.e., ‘misses’ for the ‘old’ condition and ‘correct rejections’ for the other conditions). Given we had clear predictions, the analyses (reported in Table 5) were conducted only on the conditions of interest. We first took the ‘old’ vs. ‘new’ conditions as the baseline comparison (since they were the only conditions contrasting words that were actually heard during the listening phase against new words). This comparison allowed us to make sure participants were not performing the task at chance (d’ = 1.01), and that therefore, they correctly assessed ‘new’ words as not heard more often than ‘old’ words. Further analyses confirmed our predictions; participants falsely assessed (muted) ‘Expected’ words as heard significantly more often than ‘Unexpected’ words. Furthermore, ‘Expected’ words were falsely considered as ‘old’ words more often than ‘Unexpected’ words. Finally, ‘Expected’ words that were preceded by sentence Context 2 (unexpected article) were assessed as not heard significantly more often than when preceded by Context 1 (expected article), suggesting that the presence of an unexpected article (Context 2) may reduce the context effects on upcoming words. More concretely, the expectation of the word ‘tesoro’ (treasure) is reduced when the sentence context contains the article ‘la’ (instead of ‘el’) since in this case ‘tesoro’ becomes an impossible candidate for the continuation of the sentence.
Table 4. Results of the lexical recognition task presented in terms of percentages of hits and false alarms for each condition. Standard deviations are reported in parentheses.
Table 5. Results of the comparisons of the relevant conditions of the lexical recognition task; t- and p-values are reported. Degree of freedom (21).
Discussion
We report an ERP study examining whether anticipation processes take place during L2 speech comprehension. We hypothesized that late bilinguals may not be able to anticipate upcoming words when listening to sentences due to the difficulty of processing speech in L2. We presented French–Spanish late bilinguals with highly-constrained audio sentences in Spanish triggering an expected noun. We time-locked the ERPs on the article preceding the critical noun, which was muted to avoid overlapping effects. Results revealed a long lasting negativity (280–680 ms), larger for unexpected than for expected articles. A subsequent lexical recognition task was conducted to examine whether anticipation processes allow creating a memory trace of a word that is not actually heard. The results indicated that, although both ‘Expected’ and ‘Unexpected’ nouns were muted during the listening phase, ‘Expected’ nouns were (falsely) recognised as having been heard significantly more often than ‘Unexpected’ words. The implications of these results are discussed below.
The main contribution of the present study is the ERP data associated to expected and unexpected articles. This observation is in line with the L1 literature in visual modality (DeLong et al., Reference DeLong, Urbach and Kutas2005; Foucart et al., Reference Foucart, Martin, Moreno and Costa2014; Martin et al., Reference Martin, Thierry, Kuipers, Boutonnet, Foucart and Costa2013; Otten, Nieuwland & Van Berkum, Reference Otten, Nieuwland and Van Berkum2007; Van Berkum, Brown, Zwitserlood, Kooijman & Hagoort, Reference Van Berkum, Brown, Zwitserlood, Kooijman and Hagoort2005; Wicha, Moreno & Kutas, Reference Wicha, Moreno and Kutas2004, Reference Wicha, Moreno and Kutas2003) and auditory modality (Otten et al., Reference Otten, Nieuwland and Van Berkum2007; Van Berkum et al., Reference Van Berkum, Brown, Zwitserlood, Kooijman and Hagoort2005; Wicha, Bates, Moreno & Kutas, Reference Wicha, Bates, Moreno and Kutas2003) as they similarly reveal a modulation of the ERP component in response to unexpected itemsFootnote 2 . Importantly, the present N400-like converges with the effect observed in L2 sentence reading (Foucart et al., Reference Foucart, Martin, Moreno and Costa2014; but see Martin et al., Reference Martin, Thierry, Kuipers, Boutonnet, Foucart and Costa2013), with the only difference that the present effect had a slightly earlier onset (as usually found in speech comprehension; Van Petten, Coulson, Rubin, Plante & Parks, Reference Van Petten, Coulson, Rubin, Plante and Parks1999), and lasted longer (up to 680 ms instead of 500 ms). A long lasting negativity was also reported in native speakers with the same material and procedure (Foucart et al., Reference Foucart, Ruiz-Tada and Costa2015), so it could be attributed to the experimental design. Note that the long-lasting negativity observed here could also be the combination of two components (an early and a later negativity), with the first negativity reflecting a conflict at phonological level related to the expected word, followed by a delayed N400-like effect (see Foucart et al., Reference Foucart, Ruiz-Tada and Costa2015 for similar proposal). The study does not allow us to tell apart these two alternatives. Importantly, the ERP results suggest that, although speech processing is more difficult than written processing, late bilinguals are able to use the sentence context incrementally to predict upcoming words and their features. Consequently, it implies that similar processes take place during L1 and L2 online speech comprehension.
The other contribution of the study is the effect of anticipation processes on lexical recognition. As mentioned above, the lexical recognition task following the main experiment was designed to explore the recollection of words that were predicted but were never heard. The results show that when a word was expected from the context it was (falsely) recognised significantly more often than an unexpected word, even though both were muted. Moreover, ‘Expected’ words were (again falsely) considered as ‘old’ words (that were actually heard) more often than ‘Unexpected’ words. Finally, the results also suggest that the presence of an unexpected article in the sentence context reduces context effects on upcoming words (‘Expected’ words were not recognised to the same extent depending on whether they were preceded by Context 1 or Context 2). These results are very similar to those observed in native speakers (Foucart et al., Reference Foucart, Ruiz-Tada and Costa2015) and imply that, in L2 speech comprehension like in L1, anticipation processes allow a memory trace of a word to be created prior to presentation. In other words, upcoming words are somehow ‘hallucinated’ and the L2 word recognition system benefits from lexical pre-activation. These results are also evidence that, although speech comprehension is effortful, late bilinguals do not rely only on integration processes during speech comprehension.
This proposal is in line with theories of perception and sensory predictability which claim that top-down information is used to generate a prediction and bottom-up information detect whether the prediction is correct (Arnal & Giraud, Reference Arnal and Giraud2012; Friston, Reference Friston2005; Wacongne, Changeux & Dehaene, Reference Wacongne, Changeux and Dehaene2012). Applying this reasoning to speech perception, we can assume that late bilinguals, like native speakers, use anticipation processes to create a template of an upcoming word (top-down). If the actual input (bottom-up) is not presented, as it was the case here since the word was muted, only the template remains, creating a ‘hallucination’ of the word (see for example, SanMiguel, Widmann, Bendixen, Trujillo-Barreto & Schröger, Reference SanMiguel, Widmann, Bendixen, Trujillo-Barreto and Schröger2013).
Overall, our findings complement those that previously reported context effects in L2 (Chambers & Cooke, Reference Chambers and Cooke2009; Fitzpatrick & Indefrey, Reference Fitzpatrick and Indefrey2009; Lagrou et al., Reference Lagrou, Hartsuiker and Duyck2012) as they show that anticipation processes are at play during L2 speech comprehension. Hence, it suggests that the reported facilitation of word recognition is not only due to integration but also to anticipation. Furthermore, our results also indicate that anticipation processes allow a word to be ‘hallucinated’, which further supports the claim that late bilinguals do not rely only on integration processes in complex linguistic situations (here, listening). The results support a top-down view of L2 speech processing.
Supplementary Material
For supplementary material accompanying this paper, visit http://dx.doi.org/10.1017/S1366728915000486
