1. Introduction
On the basis of clinical studies on aphasic patients, neurocognitive research first suggested that the bilinguals’ first (L1) and second language (L2) might rely on different neural substrates and thus be differently represented in the brain (see Paradis, Reference Paradis1995; Zatorre, Reference Zatorre1989). This view was supported by electro-cortical stimulation studies (Lucas, McKhann & Ojemann, Reference Lucas, McKhann and Ojemann2004), which indicated that certain cortical sites might selectively be involved in one of the bilingual's languages but not in the other. In addition, this first theoretical account found some support in the early functional studies (Dehaene, Dupoux, Mehler, Cohen, Paulesu, Perani, van de Moortele, Lehericy & Le Bihan, Reference Dehaene, Dupoux, Mehler, Cohen, Paulesu, Perani, van de Moortele, Lehéricy and Le Bihan1997; Kim, Relkin, Lee & Hirsch, Reference Kim, Relkin, Lee and Hirsch1997) that suggested that distinct cortical areas were associated with the processing of the second language. However, subsequent neurofunctional investigations taking into account the issue of language proficiency in bilinguals did not support the hypothesis of a differential cortical representation for L1 and L2 (Abutalebi, Reference Abutalebi2008; Chee, Hon, Lee & Soon, Reference Chee, Hon, Lee and Soon2001; Illes, Francis, Desmond, Gabrieli, Glover, Poldrack, Lee & Wagner, Reference Illes, Francis, Desmond, Gabrieli, Glover, Poldrack, Lee and Wagner1999; Klein, Milner, Zatorre, Zhao & Nikelski, Reference Klein, Milner, Zatorre, Zhao and Nikelski1999; Perani, Paulesu, Galles, Dupoux, Dehaene, Bettinardi, Cappa, Fazio & Mehler, Reference Perani, Paulesu, Galles, Dupoux, Dehaene, Bettinardi, Cappa, Fazio and Mehler1998). Indeed, functional studies suggests that L1 and L2 rely on a common neural network whose activation might be modulated by several factors including the age of acquisition, the extent of exposure to L2 and the level of proficiency in L2 (for reviews, see Abutalebi, Reference Abutalebi2008; Abutalebi & Green, Reference Abutalebi and Green2007; Perani & Abutalebi, Reference Perani and Abutalebi2005). Still, it appears that no real consensus exists between the theoretical view that defends the notion of a different cerebral organization for L1 and L2 and the other that supports the notion of convergence towards the same brain network. Actually, some authors consider that differences observed between L1 and L2 processing in functional studies are mostly quantitative (Abutalebi, Reference Abutalebi2008) while others suggest that the differences might also be of a qualitative nature (Hull & Vaid, Reference Hull and Vaid2007).
At the behavioural-cognitive level, one of the questions that has occupied researchers in the field of bilingualism was that of how the mental lexicon is organized in bilinguals and how they access their lexicon/s. Very often, researchers who aimed at investigating these issues relied on the processing and recognition of L1 and L2 words during semantic priming paradigms. Semantic priming effects refer to response facilitation to target words that are preceded by semantically related words (as compared to unrelated ones) due to cross-links between concepts in the semantic network (Collins & Loftus, Reference Collins and Loftus1975). The fact that priming effects were observed both in L1 and L2, in intra-language (when the prime and target words are in the same language) and in inter-language conditions (prime and target are in different languages) led some authors to the suggestion that bilinguals access a common semantic system but that L1 and L2 words are represented in separate lexicons (de Groot & Nas, Reference de Groot and Nas1991; Dufour & Kroll, Reference Dufour and Kroll1995; Francis, Reference Francis1999; Gollan & Kroll, Reference Gollan, Kroll and Rapp2001; Grainger & Beauvillain, Reference Grainger and Beauvillain1988; Keatley & de Gelder, Reference Keatley and de Gelder1992). Nevertheless, the bilinguals’ reaction times (RTs) remain generally slower during recognition and naming of L2 than L1 words, even in highly proficient subjects (Grainger & Beauvillain, Reference Grainger and Beauvillain1988; Keatley & de Gelder, Reference Keatley and de Gelder1992; Kroll & Stewart, Reference Kroll and Stewart1994; Tzelgov & Eben-Ezra, Reference Tzelgov and Eben-Ezra1992; von Studnitz & Green, Reference von Studnitz and Green2002). In this regard, various cognitive models differing in their basic theoretical focus (i.e., language representation, learning, production etc) have been proposed (see Kroll, Van Hell, Tokowicz & Green, Reference Kroll, van Hell, Tokowicz and Green2010; van Heuven & Dijkstra, Reference van Heuven and Dijkstra2010) to account for the organisation and the access to the lexicon in bilinguals. A major difference between these models is related to the question of whether the bilingual lexicon is integrated and accessed in a language-nonselective or in a language-selective way. For instance, the “Word Association model” assumes that words in L2 access concepts via lexical links with L1 words (Potter, So, Von Eckardt & Feldman, Reference Potter, So, Von Eckardt and Feldman1984), whereas the “Concept Mediation model” proposes a direct access to concepts for words in each language (Potter et al., Reference Potter, So, Von Eckardt and Feldman1984). The “Revised Hierarchical model” (RHM, Kroll & Stewart, Reference Kroll and Stewart1994, Kroll et al., Reference Kroll, van Hell, Tokowicz and Green2010) suggested that beginning bilinguals access meaning for L2 words via lexical links with L1 words and that only after increasing their proficiency in L2 (i.e., supposing the existence of a developmental shift) can they directly access concepts from L2 words (see Alvarez, Holcomb & Grainger, Reference Alvarez, Holcomb and Grainger2003; Menenti, Reference Menenti2006). The “Bilingual Interactive Activation model” (BIA+, Dijkstra & Van Heuven, Reference Dijkstra and van Heuven2002) postulates the existence of two distinct systems: a word identification and a task/decision system. For word identification, the BIA+ model assumes that the bilingual lexicon is integrated and accessed in a language-nonselective manner. Hence, the BIA+ model is seen as an alternative to the RHM in that it allows accounting for interactions between word candidates active in both languages (Dijkstra & Van Heuven, Reference Dijkstra and van Heuven2002). At present, the behavioural literature appears to support, in certain cases, one model and in other cases another model, a finding that can be attributed to the use of different language tasks (see Alvarez et al., Reference Alvarez, Holcomb and Grainger2003) or to the fact that neither model can account for the different data sets due to changes in the dynamics of L2 development (Kroll et al., Reference Kroll, van Hell, Tokowicz and Green2010).
Presently, various functional studies support the notion that the processing of L1 and L2 converges towards the same neural modules in the brain (Abutalebi, Reference Abutalebi2008). Also, studies investigating word processing in the bilingual brain during lexico-semantic tasks led to different theoretical models (Kroll et al., Reference Kroll, van Hell, Tokowicz and Green2010, van Heuven & Dijkstra, Reference van Heuven and Dijkstra2010), some suggesting that differences in the processing of L1 and L2 words exist both at the lexical and post-lexical semantic levels. In addition, accumulating evidence supports the view of language non-selectivity and interaction between the two languages’ lexicon (Kroll et al., Reference Kroll, van Hell, Tokowicz and Green2010). Nevertheless, the locus of the time cost difference in processing words from the dominant language and the less dominant language is still not clear. To further investigate this issue and determine when in time the processing of L1 and L2 words differs, using event-related potentials (ERPs), which allow tracking changes in the brain electric activity with a millisecond time resolution, is the suitable technique. The main assumption of this study was that differences in the temporal dynamics of the recruitment of the hypothesized same neuronal network might explain time cost differences in the processing of words in the two languages. This hypothesis has, to the best of our knowledge, not been fully explored using spatio-temporal dynamics of the brain global field potential analysis. Contrary to classical ERP studies that concentrate on comparing latencies and amplitudes of components at few or more recording sites, the high density, strength-independent topographic spatio-temporal analysis used in this study allows the comparison between conditions of the electric potential distribution over the entire scalp (ERP mapping). Only if the configuration of the electric potential distribution differs between conditions, can one assume that different generators have been activated in the brain during a given processing step (Lehmann, Reference Lehmann, Gevins and Remond1987). Here, behavioural and ERP measures were collected from bilinguals while processing printed word pairs. Words in each pair were either from the same language (unmixed in L1 or in L2) or from two different languages (mixed L1-L2 or L2-L1). The words in each pair were sequentially presented and subjects had to decide after the second word (i.e., the “target”), whether or not the two words were semantically related (SR) or unrelated (SU). In this task, similar to lexical decision tasks (e.g., Bentin, McCarthy & Wood, Reference Bentin, McCarthy and Wood1985; Rugg, Reference Rugg1985), SR words generally induce faster reaction times than SU ones (Khateb, Michel, Pegna, O’Dochartaigh, Landis & Annoni, Reference Khateb, Michel, Pegna, O’Dochartaigh, Landis and Annoni2003; Khateb, Pegna, Michel, Custodi, Landis & Annoni, Reference Khateb, Pegna, Michel, Custodi, Landis and Annoni2000b) thus mimicking the classical priming effect. Also, as in other semantic categorisation (Barrett & Rugg, Reference Barrett and Rugg1989, Reference Barrett and Rugg1990; Khateb, Michel, Pegna, Thut, Landis & Annoni, Reference Khateb, Michel, Pegna, Thut, Landis and Annoni2001; Khateb et al., Reference Khateb, Michel, Pegna, O’Dochartaigh, Landis and Annoni2003; Khateb, Pegna, Landis, Mouthon & Annoni, Reference Khateb, Pegna, Landis, Mouthon and Annoni2010; Perez-Abalo, Rodriguez, Bobes, Guttierrez & Valdes-Sosa, Reference Perez-Abalo, Rodriguez, Bobes, Guttierrez and Valdes-Sosa1994; Polich, Reference Polich1985) and lexical decision tasks (Bentin et al., Reference Bentin, McCarthy and Wood1985; Rugg, Reference Rugg1985), the ERPs for SU words show an enhancement of the N400 component. This component, which is linked to semantic integration/matching processes (Khateb et al., Reference Khateb, Pegna, Landis, Mouthon and Annoni2010; Kutas & Federmeier, Reference Kutas and Federmeier2009), will be utilized here as evidence of the validity of the paradigm to illustrate the classical electrophysiological correlate of semantic processing. Accordingly, based on the assumption that the two languages rely on the same brain network but that L2 uses certain neural modules of this network differently (as suggested by functional imaging studies), we hypothesized that some of the processing steps will be modulated by either language, a modulation that should to some extent reflect time cost differences (see examples in other tasks in Khateb, Michel, Pegna, Landis & Annoni, Reference Khateb, Michel, Pegna, Landis and Annoni2000a; Khateb, Pegna, Michel, Landis & Annoni, Reference Khateb, Pegna, Michel, Landis and Annoni2002; Pegna, Khateb, Spinelli, Seeck, Landis & Michel, Reference Pegna, Khateb, Spinelli, Seeck, Landis and Michel1997). Thus, in terms of spatio-temporal analysis of the ERP map series (referred to also as “functional microstate analysis”, see Michel, Thut, Morand, Khateb, Pegna, Grave de Peralta, Gonzalez, Seeck & Landis, Reference Michel, Thut, Morand, Khateb, Pegna, Grave de Peralta, Gonzalez, Seeck and Landis2001; Murray, Brunet & Michel, Reference Murray, Brunet and Michel2008), we predicted that the same processing steps will be found in L1 and L2, but some steps might either be of prolonged duration in L2 (see Khateb et al., Reference Khateb, Michel, Pegna, Landis and Annoni2000a; Reference Khateb, Pegna, Michel, Landis and Annoni2002; Pegna et al., Reference Pegna, Khateb, Spinelli, Seeck, Landis and Michel1997) or appear later in this same language. Such difference should thus account for the expected L2 vs. L1 time processing differences. Alternatively, if the two languages rely on different neural resources, then this analysis will allow the identification of additional and new language-specific microstates (i.e., processing steps with specific field configuration and probably different cerebral sources) that should be found during L2 but not during L1 word processing.
2. Materials and Methods
2.1. Subjects
Twelve healthy, young, German–French bilingual students (10 women and 2 men, mean age = 25 ± 4y) were recruited from the School of Translation of the University of Geneva to take part in the experiment. All were right-handed according to the Edinburgh Inventory (Oldfield, Reference Oldfield1971, mean laterality quotient = 78 ± 13). They all had normal or corrected-to-normal vision and none presented any history of neurological or psychiatric diseases. They all gave formal written consent and were paid for their participation in this study.
Assessment of proficiency in L2
All participants had German as a first language (L1) and French as a second language (L2). All subjects received their schooling in German since early childhood and started to learn French as L2 at the age of 11.5 ± 1.2 years. Before their admission to the University, they all passed the examination allowing them to be admitted to the School of Translation with French as the first active language. At the time of this experiment, all but two had successfully completed their second year of studies and were already enrolled in their third year. Prior to their participation in the study, they responded to a questionnaire regarding their exposure to German and French in areas that included audio-visual media, family, university, friends, girlfriends/boyfriends, reading and other various activities and hobbies (Wartenburger, Heekeren, Abutalebi, Cappa, Villringer & Perani, Reference Wartenburger, Heekeren, Abutalebi, Cappa, Villringer and Perani2003). On a daily basis, this questionnaire indicated that they were, on the average, exposed to L1 for 4.6 ± 1.6 hours and to L2 for 5.7 ± 3.9 hours. This difference is due to the fact that all students lived in an L2 speaking environment (i.e., the French-speaking Geneva area).
Language proficiency assessment also included a translation test that evaluated the quality and times of translation from L2 to L1 as indices of language proficiency. Subjects were asked to translate a text of 150 words long from French into German without any time constraint. Timing measures were collected through the translation process using TRANSLOG2000 (http://www.translog.dk), a computer software that tracks keyboard activity (Jakobsen, Reference Jakobsen and Hansen1999). Two independent professional translators were asked off-line to assess the quality of the translation. This measure indicated that subjects’ performance was relatively homogenous among the group (translation quality mean score = 54 ± 10 out of 80) and attested a good proficiency in L2. Timing measures also showed that the total translation times were homogenous across subjects (30 ± 7 minutes). Finally, we considered as an index of the proficiency level in L2 the subjects’ performance in a mixed L1-L2 computerized image-naming task conducted in another context (Khateb, Abutalebi, Michel, Pegna, Lee-Jahnke & Annoni, Reference Khateb, Abutalebi, Michel, Pegna, Lee-Jahnke and Annoni2007a). In this bilingual time-constrained task, the subjects’ rate of correct responses in L2 naming was 85 ± 10% (vs. 96 ± 3% in L1 for the same pictures, p < .001) confirming their high proficiency in the second language. Moreover, their naming time was, as expected, slightly longer in L2 than in L1 (see Abutalebi, Annoni, Zimine, Pegna, Seghier, Lee-Jahnke, Lazeyras, Cappa & Khateb, Reference Abutalebi, Annoni, Zimine, Pegna, Seghier, Lee-Jahnke, Lazeyras, Cappa and Khateb2008; Khateb et al., Reference Khateb, Abutalebi, Michel, Pegna, Lee-Jahnke and Annoni2007a).
2.2. Stimuli and procedure
The stimuli consisted of word pairs composed of 200 German (L1) and 200 French (L2) nouns that were concrete and imaginable of 4 to 8 letters in length (mean length = 6.1 ± 1.3 letters). In both languages, about one half of the words were selected from six natural categories (fruits, vegetables, trees, flowers, birds and animals) and the other half selected from six other categories representing manufactured objects (transport, tools, furniture, buildings, clothes and utensils). These categories were inspired from Dubois and Poitou's database (Dubois & Poitou, Reference Dubois and Poitou2002). The stimulus consisted of 400 word pairs. These word pairs were composed using the 200 German words and the 200 French words, with each word repeated once in the whole stimulus set. In most instances, French L2 words (selected from Brulex database, see Mousty & Radeau, Reference Mousty and Radeau1990, average lexical frequency = 29.2 per million), were translation equivalents of middle to high frequency German L1 words (selected from CELEX, see Baayen, Piepenbrock & Van Rijn, H., Reference Baayen, Piepenbrock and Van Rijn1993, average lexical frequency = 17.0 per million). In one subset of 100 pairs, the first word (hereafter the prime word) and the second word (hereafter the target word) in each pair were in L1. In a second subset of 100 pairs, the prime and target in each pair were in L2. In the third subset of 100 mixed pairs, the primes were in L2 and the targets in L1. Finally, in the last subset of 100 mixed pairs, the primes were in L1 and the targets in L2. In 50% of each of these four subsets, the two words in each pair were exemplars of the same semantic category (hereafter semantically related: SR) and in the other 50%, the two words in each pair belonged to different semantic categories (hereafter semantically unrelated: SU, see examples in Table S1 in Supplementary Materials Online, Supplementary Materials). Accordingly, the stimulus list was composed of 8 experimental conditions, each condition having 50 pairs. During the experiment, the whole set of 400 pseudo-randomized pairs was subdivided into four equivalent blocks of 100 pairs, representing equivalently all the 8 experimental conditions. The order of presentation of the four blocks was balanced over subjects. The 8 conditions provided a 2×2×2 repeated measures analysis of variance (ANOVA) design with the language mix (intra- vs. cross-language), the target language (L1 vs. L2 target words) and the semantic relatedness (SR vs. SU target words) as within subjects’ factors
Experiments were carried out in an isolated, electrically shielded room. In order to allow for the same visual presentation of German and French (given the need to capitalize the initial letter of German nouns) and to avoid identification of language-specific diacritical marks, the words from both languages were presented in uppercase format. The stimuli were presented on the computer screen using E-Prime software (Psychology Software Tools, Inc., www.pstnet.com/ PA, USA). Each stimulation trial (of about 3.5 s duration) started with the presentation of a central fixation cross that lasted for 500 ms. Afterwards, the prime was presented for 150 ms centrally (with its borders extending up to 2.5 degrees laterally), followed by an inter-stimulus interval of 700 ms (thus yielding a stimulus onset asynchrony of 850 ms), and by the second word (target) which also appeared for 150 ms. A blank screen of 2 s followed the presentation of the second word to allow for the subjects’ motor responses (see Figure S1 in Supplementary Materials Online, Supplementary Materials). The subsequent appearance of a central cross informed the participants of the imminence of the following trial and allowed for gaze fixation and the return to the EEG baseline. A break of about 3 minutes was given after the presentation of each stimulation block of 100 pairs.
The subjects were seated 120 cm from the screen and asked to fixate on the central cross, to silently read the words pairs, and to indicate after the presentation of the target whether or not the two words in each pair were semantically related. They responded as quickly and accurately as possible by pressing one of two buttons using their right hand middle and index fingers (half of the subjects responded with the middle finger for SR and with the index for SU targets and the other half did the inverse). All subjects underwent a training session consisting of 15 mixed SR and SU pairs in order to ensure full comprehension of the task demands.
2.3. EEG recordings and ERP analysis
The EEG was continuously recorded (at 500 Hz, band-pass filtered between 0.1–200 Hz) from 111 electrodes (128-channels system from Electrical Geodesic, Inc., Oregon, U.S.A). The data were analysed off-line using the Cartool software© developed by Denis Brunet (Brunet, Murray & Michel, Reference Brunet, Murray and Michel2011, http://brainmapping.unige.ch/cartool). ERP epochs were filtered between 1 and 30Hz and averaged separately for each of the experimental conditions from –100 ms before the presentation of target words to 800 ms post-stimulus (i.e., time of the participants’ mean response). Trials with erroneous behavioural responses, eye-movements (as determined by the visual inspection of the data) or sweeps exceeding ± 100μV were rejected before epoch averaging. On the average, the number of artifact-free epochs over subjects and experimental conditions was 41 for L1 targets and 38 for L2 targets (respectively in intra-language conditions: L1 SR = 41 ± 5 and SU = 42 ± 5; L2 SR = 36 ± 5 and SU = 38 ± 8, and in cross-language conditions: L1 SR = 38 ± 5 and SU = 42 ± 5; L2 SR = 37 ± 4 and SU = 41 ± 8). Before computing the grand mean ERP of each condition, the individual ERPs of each condition, recorded against Cz as reference, were recomputed against the average reference (Lehmann & Skrandies, Reference Lehmann and Skrandies1980).
ERP waveshape analysis
This analysis aimed at assessing amplitude differences between ERPs for SU and SR conditions during the 400 ms time period. This allowed to verify the presence of the N400 component that is generally expected in this type of paradigm, and to test its modulation by the language of the target word. Thus, based on visual inspection of ERP traces on centro-parietal electrodes, the mean signal from individual ERPs for each condition were computed in the time period between 360 and 500 ms post-stimulus (i.e., determined around the peak amplitude in grand-mean data, see Figure 1). These values were compared statistically on a subset of 9 central electrodes around Cz (in the left: FC1, C1, CP1; midline: FCz, Cz, CPz; and right: FC2, C2, CP2) using language mix (intra- vs. cross-language pairs), target language (L1 vs. L2) and semantic relatedness (SR vs. SU), scalp locations (left, midline and right) and recording sites (anterior, central and posterior sites) as within-subjects’ factors. These additional factors (scalp locations and sites) were used to assess the lateralization of the N400 as frequently used in the literature.
Figure 1. The N400 component shown by the superposition of the grand-mean ERPs from SR (black traces) and SU (red) conditions. A1–4: Illustration of the average signal of a subset of 9 recording sites around Cz (see Methods for the electrodes) where major differences between SU (red lines) and SR (black lines) were determined by visual inspection after L1 (A1 and A3) and L2 target words (A2 and A4). B1–4: Graphs illustrating the mean amplitude (between 360–500 ms) of the N400 response for SR and SU conditions on the 9 central. Statistical analysis included the scalp location (left, midline and right electrodes) and antero-posterior axis (1, 2 and 3 electrodes) as within subject factors. The analysis confirmed the presence of a significant semantic relatedness but not language effect (see text for details).
Temporal segmentation of ERP map series
Since the aim of the study was to identify the evoked electric field patterns that characterized the different conditions, this analysis was conducted on all ERP map series from stimulus onset to 800 ms post-stimulus. For that, we first subjected the grand-mean ERP map series of the 8 conditions to a temporal segmentation procedure implemented in Cartool software (Brunet et al., Reference Brunet, Murray and Michel2011) and based on a k-means clustering (Pascual-Marqui, Michel & Lehmann, Reference Pascual-Marqui, Michel and Lehmann1995). In a first step, this analysis allows the definition of the optimal number of electric topographic maps that explain the most dominant field configurations in the grand-mean ERP map series and the time of appearance of these maps in the grand means. The time period during which each map is found is referred to as a “map segment” or a “functional microstate”. The succession of these microstates is hypothesized to reflect the successive steps of information processing in the brain (see Brandeis, Naylor, Halliday, Callaway & Yano, Reference Brandeis, Naylor, Halliday, Callaway and Yano1992; Lehmann, Reference Lehmann, Gevins and Remond1987; Murray et al., Reference Murray, Brunet and Michel2008). In a second step, this analysis allows locating these topographic (template) maps in the individual ERP map series of each condition (see details in Brunet et al., Reference Brunet, Murray and Michel2011; Khateb, Pegna, Landis, Michel, Brunet, Seghier & Annoni, Reference Khateb, Pegna, Landis, Michel, Brunet and Seghier2007b; Murray et al., Reference Murray, Brunet and Michel2008, and in Supplementary Materials Online, Supplementary Materials). Here, we compared statistically, based on the analysis of the maps in individual data, the duration of each of the segments observed in the grand-means in order to determine the conditions-specific map segments. This procedure allows defining those segments appearing preferentially in some but not in other conditions and those that are present in the different conditions but differ by duration. Statistical analyses were also performed on the time of occurrence of the best-fit maps (i.e., the time in individual data when individual maps showed the highest spatial correlation with template maps found in the grand-means, Murray et al., Reference Murray, Brunet and Michel2008). This procedure allows determining and verifying the time course of certain segments of interest in the individual data. Note that, in these and all other statistical analysis, only significant main effects and interactions (at p < .05) were considered and followed up by post-hoc analyses. In addition, all repeated measures with more than one degree of freedom were first subjected to the Geisser–Greenhouse correction.
Source localisation analysis
This analysis used the LAURA source localisation (Grave de Peralta Menedez, Gonzalez Andino, Lantz, Michel & Landis, Reference Grave de Peralta Menedez, Gonzalez Andino, Lantz, Michel and Landis2001) to estimate brain regions that gave rise to electric fields that differentiated conditions. LAURA is a distributed linear inverse solution that has been used in a large variety of cognitive tasks (see examples in Andrade, Butler, Mercier, Molholm & Foxe, Reference Andrade, Butler, Mercier, Molholm and Foxe2015; Cappe, Thut, Romei & Murray, Reference Cappe, Thut, Romei and Murray2010; Khateb et al., Reference Khateb, Pegna, Landis, Mouthon and Annoni2010; Taha & Khateb, Reference Taha and Khateb2013 and in Supplementary Materials Online, Supplementary Materials). Here, for each subject, we applied the LAURA solution to the mean signal of the time periods whose maps significantly differentiated conditions either as a function of language or semantic relatedness (see Supplementary Materials Online, Supplementary Materials). The mean inverse solution over subjects was then computed to illustrate the location of the maxima of the activated brain areas for each time segment. T-tests were then conducted on all solution points (i.e., voxels) to compare individual inverse solutions between conditions or segments of interest. The anatomical location and Brodmann areas were then reported based on the Talairach's coordinates for the centre of gravity of the significant differences.
Behavioural analysis
The median reaction time (RT) and percent of correct responses were computed separately for each participant in each experimental condition. Statistical analyses were then performed using 2×2×2 ANOVAs with the three within subjects’ factors.
3. Results
3.1. Behavioural results
The mean reaction time and performance (±SD) over subjects are detailed in Table 1 for the different experimental conditions. The 2×2×2 ANOVA performed on the individual RTs showed highly significant main effects for language (F(1, 11) = 17.3, p < .002) and semantic relatedness (F(1, 11) = 34.1, p < .0001). The language effect was due to the fact that on the average RTs for L1 words were faster (740 ms) than for L2 words (782 ms). The relatedness effect was due to faster RTs for SR (718 ms) than for SU words (803 ms). No significant interaction was observed between the analysis factors.
Table 1. Mean of the subjects’ reaction times (in ms, ±SD) and performance (in % of correct responses, ±SD) in the different experimental conditions.
The 2×2×2 ANOVA performed on the individual percentage of correct responses (Table 1) revealed only a highly significant main effect of language (F(1, 11) = 21.0, p < .001). This effect was attributable to the fact that L1 target words elicited a higher percentage of correct responses (91%) than L2 targets (84%). A highly significant interaction was observed between language mix and target language (F(1, 11) = 22.9, p < .001). This interaction was due to the fact that language mixing (as compared to the intra-language condition) decreased performance for L1 targets but increased performance for L2 targets thus, yielding a smaller difference between L1 and L2 targets in the cross-language conditions. Finally, a slightly significant interaction was also found between language mix and relatedness (F(1, 11) = 6.0, p < .05) due to the fact that the difference in performance between SR and SU targets was higher in the cross-language than in intra-language conditions (see details in Table 1).
3.2. ERP waveform analysis
Figure 1A illustrates the superposition of grand-mean ERP traces induced by SR (black traces) and SU (red traces) for the average signal from a subset of 9 central recordings sites around Cz. Panels A1 and A2 present traces for L1 and L2 in intra-language conditions and panels A3 and A4 present traces for L1 and L2 in cross-language conditions. These grand mean traces show that the N400 effect was present in all comparisons although its amplitude and peak latency seemed to vary slightly across conditions (see blue triangles pointing at 400 ms below the traces).
In order to statistically verify whether or not the N400 response was sensitive to the target language and/or to language mix, the individual mean signals were computed between 360–500 ms from the 9 central recording sites and compared statistically using analysis of variance (see Methods above). This analysis showed only a highly significant main effect of semantic relatedness (F(1, 11) = 25.4, p < .0004). As illustrated in Figure 1B, this effect was due to the fact that ERPs for SU words were more negative (on the average ~0.7 μV) than for SR words. No significant interaction was observed between this and the other experimental factors (interaction: with language mix p = .29, with target language p = .86, with both together p = .09). Also, no significant interaction was found between semantic relatedness and scalp location and recording site factors.
In order to assess possible differences in the peak latency of the N400 component, the most negative time point between 360–500 ms was determined from the averaged signal of the 9 central electrodes (as illustrated in Figure 1A) as the peak of the component for each participant in SU conditions. These values were then subjected to a 2×2 ANOVA with language mix (intra- vs. cross-language) and target language (L1 vs. L2) as within-subjects’ factors. This analysis revealed a significant main effect for target language (F(1, 11) = 6.0, p < .03) due to the fact that the N400 peaked slightly earlier in L1 than in L2 (average latency = 394 and 406 ms respectively for L1 and L2). No effect of language mix or interaction between the two factors was observed.
3.3. Topographical analysis using the temporal segmentation of ERP map series
The temporal segmentation of the grand-mean ERP map series of the 8 experimental conditions showed that 10 topographic maps explained the most dominant field configurations found in the whole data set. Figure 2A illustrates successively the different maps labelled 1 to 10 (referred to hereafter as “map segments” or S1 to S10), as a function of the order of their appearance in the grand-mean ERPs (see colour code below each map and in B). Figure 2B displays the global field power traces (Lehmann & Skrandies, Reference Lehmann and Skrandies1980) for each grand-mean ERP, and indicates, using colour codes representing the maps, the time segments where each of these maps was found in the group average of each condition. This illustration demonstrates that while some of the map segments exhibited varying durations and onset times (i.e., shifts in terms of time of occurrence) in the grand-means (see for instance the map segments S9–S10), others were only found in some but not in all conditions (see for instance the maps S6 and S8 which appeared only in SU conditions).
Figure 2. Temporal segmentation of the grand-mean ERP map series. A: The temporal segmentation of the 8 experimental conditions revealed that 10 topographic potential maps explained the dominant field configurations found in all conditions. These maps are labelled 1 to 10 (S1 to 10 I text), are colour-coded (see colour bar under each map) and presented as a function of their time of appearance in the grand-mean data. B: Global Field Power traces of the 8 experimental conditions illustrate the time of occurrence of the successive map segments which are indicated both by the colour code the map number appearing below each segment. The Global Field Power is calculated as the spatial standard deviation of the average reference maps and thus represents the strength of the field (Lehmann & Skrandies, Reference Lehmann and Skrandies1980). These traces show that: Segment 1 coincides with the P1 component (average peak = 89 ± 2 ms); Segment 2 with N1 and Segment 3 with P2. Note that the 2 dimension maps are shown from top with left ear left, with blue values representing negative potentials and red values positive potentials. The illustration shows that some map segments appeared only in certain but not in other conditions (e.g., Maps 6 and 8 appearing only in SU conditions). Also, it shows that the time of appearance of some map segments is delayed in some conditions relative to others (e.g., map segments 9 and 10 shifted in L2 and SU conditions).
Analysis of segments’ duration in individual ERP map series
In order to statistically assess the maps’ specificity to the one or to other condition, we analysed their presence in the individual data of each condition using the map fitting procedure (see Methods; and Murray et al., Reference Murray, Brunet and Michel2008). This analysis, referred to as “segments’ duration”, allows determining the total time during which a given map was found in the individual map series of a given condition. Thus, based on the time of occurrence of the different map segments in the grand-means, we fitted the maps S1 to S4 (corresponding to the period of the P1-N1-P2-N2 complex) in the time window between 60 and 320 ms and fitted the maps S5 to S10 in the time window between 310–800 ms.
A 2×2×2×4 ANOVA was performed on the individual duration of the map segments’ appearing in the first time window, using the three experimental factors (language mix: intra- vs. crossed language; target language: L1 vs. L2 and semantic relatedness: SR vs. SU) and map segments (S1 to S4) as within-subject factors. The mean duration (±SE) for each of these map segments across conditions was of 77 ± 9, 54 ± 11, 78 ± 10 and 54 ± 9 respectively for S1 to S4. The ANOVA showed a highly significant interaction between target language and map segments (F(3, 33) = 5.1, p < .008, Geisser–Greenhouse corrected here and below). Post-hoc Fisher's LSD tests showed that segment S3 was significantly longer in L2 than in L1 (p < .003, mean = 84 ms vs. 71 ms respectively in L2 and L1). Moreover, a significant interaction was observed between relatedness and map segments (F(3, 33) = 4.4, p < .02). Post-hoc tests showed a longer duration for S4 in the SR conditions than in the SU conditions (p < .006, mean = 63 ms vs. 45 ms respectively). No interaction was found between language mix and the other factors or between language and relatedness.
In the second time window, the 2×2×2×6 ANOVA performed on the duration of the map segments S5 to S10 showed a highly significant main effect for segments’ maps (F(5, 55) = 57.4, p < 0.00001). This effect was due to the fact that these segments’ were of varying durations across conditions (mean duration for these segments across conditions was of 59 ± 11, 49 ± 9, 38 ± 7, 59 ± 9, 43 ± 8 and 244 ± 12 respectively for S6 to S10). Also, a highly significant interaction was observed between semantic relatedness and map segments (F(5, 55) = 7.2, p < .0006). As expected from the segmentation of the grand-mean ERP map series (Figure 2B), post-hoc tests showed that S8 was of longer duration in the SU than in the SR conditions (p < .0001, on the average 80 ms vs. 38 ms respectively SU and SR). In contrast and as already appeared in the grand mean segmentation (See Figure 2B), the last segment, S10, was of longer duration (p < .0003) in the SR than in the SU conditions (263 vs. 225 respectively). Here again, no interaction was observed between language mix and the other analysis factors.
Analysis of the segments’ time of occurrence in the individual ERP map series
This analysis aimed at verifying the latency shift observed in the late segments (namely S7, S9 and S10) between the SR and SU conditions, but also between languages (see Figure 2B). For this purpose, the time of best-fit of maps S7, S9 and S10 (see Methods) was computed in the time window of their appearance (~310–800 ms) in the grand-mean data. The 2×2×2×3 ANOVA (for maps S7, S9, and S10) performed on these time values showed first a language effect (F(1, 11) = 5.6, p < .04). As illustrated in Figure 3A, this effect was due to the fact that these segments occurred earlier in L1 than in L2 conditions. A highly significant main effect was also found for semantic relatedness (F(1, 11) = 15.6, p < .003). This effect was attributable to an earlier time of occurrence for the segments in the SR (Figure 3A, solid lines) than in the SU (dashed lines) conditions in both languages. In addition, a highly significant main effect was found for map segment (F(2, 22) = 24.2, p < .00001) due to the fact that the time of occurrence increased as a function of the map segment number (on the average: 475 ms, 547 ms and 631 ms respectively for S7, S9 and S10). This latter finding shows that the order of appearance of these segments in the grand-mean data is confirmed in the individual map series analysis. Finally, a significant interaction was found between language and segment maps (F(2, 22) = 4, p < .04). This interaction is explainable by the fact that the language effect (i.e., earlier time of occurrence in L1 than in L2) was significant for S9 and S10 only but not for S7.
Figure 3. A–B: Map segments times of occurrence. Statistical analysis performed on the time of appearance of the late segments, which appeared shifted in the grand-mean data (see Figure 2B), showed that these segments occurred significantly earlier in L1 than in L2 conditions (left vs. right graph) but also earlier in SR than in SU conditions (black lines vs. dashed lines). C–D: Significant correlations between the averaged individual duration of segment S3 and the averaged individual RTs in L1 in L2 respectively.
Correlation between segment duration and behavioral data
Given that segment S3 differentiated languages in terms of duration, this analysis sought to verify whether or not this duration contributed to the subjects’ RTs during the processing of L1 and L2 words and could thus explain processing cost differences between the two languages. For this purpose, for each subject, we computed the average duration of segment S3 separately in L1 and L2 conditions and the average RT for L1 and L2 words. As shown in Figure 3B–3C, a positive correlation was found between the individual averaged S3 duration in L1 and RT in L1 (r = 0.71; p < .01) as well as between the individual averaged S3 duration in L2 and RT in L2 (r = 0.63; p < .03). No correlation was found between the subjects’ RTs in L1 and L2 and the duration of any other segment both in the first and the second time window.
3.4. Distributed source localisation analysis
This analysis aimed at estimating brain regions that gave rise to the map segments (i.e., functional microstates) which differentiated conditions. For this purpose, we applied the LAURA source localisation to the time segments of the maps S3 and S4. Since S3 was the earliest segment to differentiate languages in terms of duration but was present in all condition, we sought to contrast it directly with the subsequent segment S4 which (also found in all conditions) showed longer duration in SR than is SU. Beyond illustrating the active sources shown by the inverse solution of each segment (Figure 4A, first two rows), the analysis of the two successive segments allowed determining whether some specific modules were active in S3 but not in S4. Another analysis was performed on the time segments of the maps S7 and S8 both appearing in the 400 ms time range. The choice of these segments was motivated by the fact that while S7 appeared mainly in the SR conditions, S8 was found in the SU conditions during the 400 ms time range. Contrasting S8 and S7 could allow determining the brain areas specifically involved in SU beyond those involved in both SR and SU (Figure 4B, first two rows). As shown by Figure 4A, segment S3 (first row) showed that the most dominant sources appeared bilaterally in the middle and inferior occipital areas (BA 17/19), lingual gyrus (BA 17/18), but also in the inferior temporal and fusiform gyri bilaterally (BA 37; x,y,z Talairach coordinates at -47, -52, -10, see arrow on the lower slices). In the subsequent S4 segment (Figure 4A, second row), the pattern of activation also involved the bilateral posterior areas but was clearly less extended. Statistical analysis using paired t-tests and contrasting the individual inverse solutions during S3 vs. S4 showed significant differences in various areas which were more active during S3 (see Figure 4A, third row for significant p values at p < .01 uncorrected). These included mainly the bilateral middle temporal gyrus (BA 37; at -41, -63, 5), the left fusiform gyrus (BA 37; at = -47, -47, -11), the right inferior temporal/fusiform gyrus (BA 20; at 53, -52, -10), the left inferior parietal lobule (BA 40, at -47, -38, 47), the right middle occipital gyrus and Cuneus (BA 18/20; at 5, -86, 11). Significant difference between the segment S4 vs. S3 (not illustrated here) were found mainly in the left superior and medial frontal gyrus (BA 10; at -23, 53, 15), bilateral Cingulate (BA 32; at -5, -46, 5) and left insula (BA 13; at -35, 6, 12).
Figure 4. Source localisation estimated by LAURA inverse solution for the periods of interest. A: LAURA solution for the time period of map segments S3 and S4 (averaged individually over all conditions) shows that the most dominant activity (displayed here on successive axial MRI slices) is localized in posterior bilateral areas (see text for more details and Supplementary Materials Online, Supplementary Materials). Note that the colour scale in the right of each panel shows the current density values. These illustrations indicate that the major activity observed during S3 extended more to the bilateral temporo-occipital and fusiform areas (see arrow on the first row). Contrasting statistically the individual inverse solutions for S3 vs. S4 showed significant differences (see third row, significant voxels at p < .01) in several areas including the bilateral fusiform gyrus (see the lowest two slices in the right side). B: LAURA solution for the time period of map segment S7 (averaged individually over SR conditions) and S8 (averaged individually over SU conditions) shows that the most dominant activity exhibited a clear resemblance in several areas but with a more extensive recruitment of the temporal cortex during S8 (see arrows in the second row). Contrasting the individual inverse solutions for S8 vs. S7 showed significant differences in the left middle frontal gyrus and inferior parietal lobule and more interestingly in the right superior/middle temporal gyrus.
During the time segment of map S7, which was observed mainly in SR grand-means in the 400 ms time window, a bilateral pattern of activation dominantly involved both occipital and temporal areas (Figure 4B, first row). More importantly, the activation during S8 which corresponds to the N400 period in SU conditions revealed a more extensive recruitment of the temporal cortex (superior and middle/inferior temporal gyri; BA 21/22, see arrows in lower slices), with other concomitant activation in the frontal regions (Figure 4B, second row). Paired t-tests comparing the individual inverse solutions of S8 vs. S7 maps (respectively in the SU and the SR conditions, Figure 4B, third row) revealed higher activity during S8 mainly in the right middle and superior temporal gyri (BA 21/22; at 65, -12, -7), but also in the left middle frontal gyrus (BA 6; at -29, -3, 51) and postcentral gyrus/inferior parietal lobule (BA 40; at -41, -32, 53). The comparison of S7 vs. S8 showed higher activity in S7 (not illustrated here) in the left fusiform gyrus (BA 37/20; at -35, -47, -11) and left middle occipital gyrus (BA 18; at -29, -82, -9).
4. Discussion
In this study, we analyzed behavioural and ERP map series, collected during a semantic categorization task, to investigate the locus of the time cost difference in the processing of words from the dominant and less dominant language in bilinguals. The choice of the semantic task was motivated by the fact that it permits the elicitation of the semantic N400 ERP component. In so doing, this experiment allowed first testing the validity of the paradigm using classical waveshape analysis on the N400 amplitude and latency and then conducting strength-independent spatio-temporal analysis of whole field configuration measured by high density ERPs. Actually, we assumed that differences in the temporal dynamics of the recruitment of the same neuronal modules might explain time cost differences in the processing of words from the bilingual's two languages.
Behavioural data showed a classical L1 vs. L2 effect in terms of RTs with faster RTs for L1 than for L2 words, and a semantic relatedness effect with faster RTs for SR than for SU words. ERP waveshape analysis conducted during the 400 ms time period confirmed the presence of the N400 component in this type of paradigm and confirmed the modulation of its peak latency by the target language. Spatio-temporal analysis of ERP map series using microstate segmentation procedures allowed the characterization of the most dominant field configurations found in the ERP map series. Of the early map segments (referred to also as functional microstates, Michel & Murray, Reference Michel and Murray2012), we found that S3, whose duration in the individual data correlated with RTs, was of longer duration in L2 than L1 conditions, a finding that explains (at least partially) processing costs differences between L2 and L1. During the late segments, we found that S8, coinciding with the N400 component (Kutas & Federmeier, Reference Kutas and Federmeier2009), was specifically found in SU conditions. Also, we found that some of the late map segments (S9 and S10) exhibited a shift in their time of occurrence in SU (compared to SR) and in L2 (compared to L1). The inverse solution analysis conducted on the segments of interest allowed estimating their neural sources. In the first part of the discussion, we will discuss the behavioural observations and relate them to previous literature. We will then discuss the electrophysiological effects around the N400 time window in relation to previous findings including in bilingual research. Finally, we will focus on the language effect observed during the early S3 segment.
4.1. Behavioural findings
Our analysis showed that, independent of language mixing, L1 words elicited faster RTs than L2 ones, thus confirming the presence of the well-known dominant language effect (von Studnitz & Green, Reference von Studnitz and Green1997). In addition, independent of the target language and language mixing, SR words induced faster RTs than SU ones, also confirming the presence of the semantic relatedness or the so called “priming effect” generally observed in lexical decision tasks (Keatley & de Gelder, Reference Keatley and de Gelder1992).
As for the semantic relatedness effect, to which, for convenience, we refer to here as the “priming effect” (Boddy & Weinberg, Reference Boddy and Weinberg1981; Meyer & Schvaneveldt, Reference Meyer and Schvaneveldt1971), it is in line with many previous studies using lexical decision (see Neely, Reference Neely, Besner and Humphreys1991) and semantic categorization tasks (Boddy & Weinberg, Reference Boddy and Weinberg1981; Khateb et al., Reference Khateb, Michel, Pegna, O’Dochartaigh, Landis and Annoni2003; Reference Khateb, Pegna, Michel, Custodi, Landis and Annoni2000b; Nieto, Hernandez, Gonzalez-Feria & Barroso, Reference Nieto, Hernandez, Gonzalez-Feria and Barroso1990; Walker & Ceci, Reference Walker and Ceci1985). Of note is the fact that the priming effect for L1 and L2 words was quite symmetrical in both intra-language (i.e., unmixed) and cross-language (i.e., mixed word pairs) conditions. Previous studies have suggested that, as a consequence of the initial reliance on L1 words, lexical links are stronger from L2 to L1 (than from L1 to L2) and consequently greater priming effects can be expected in L1-L2 word pairs than in L2-L1 ones (Kroll & Stewart, Reference Kroll and Stewart1994). A comprehensive understanding and a formal discussion on this issue using this type of task would necessitate further research and the comparison of different levels of L2 proficiency. The fact that some studies could show such predicted asymmetrical effects (Gollan, Forster & Frost, Reference Gollan, Forster and Frost1997; Jiang, Reference Jiang1999; Keatley, Spinks & de Gelder, Reference Keatley, Spinks and de Gelder1994) while others failed (Jin, Reference Jin1990) has been explained in terms of the processes involved: automatic vs. controlled (Nakamura, Kouider, Makuuchi, Kuroki, Hanajima, Ugawa & Ogawa, Reference Nakamura, Kouider, Makuuchi, Kuroki, Hanajima, Ugawa and Ogawa2010). According to such explanations, symmetrical semantic priming would occur when controlled processes are involved, whereas asymmetrical priming would occur in relation to automatic processes (see Kotz & Elston-Güttler, Reference Kotz and Elston-Güttler2004). Experimental designs involving a high proportion of related words and long stimulus onset asynchronies (SOA) are thought to increase the predictability of the target from the prime, and to induce controlled priming. Inversely, experiments with a low proportion of related words and short SOAs (by decreasing the conscious recognition of the prime and consequently decreasing the likelihood of predicting the target) would favour automatic priming processes (De Groot, Reference De Groot1984). The symmetrical effects observed here appear compatible with the highly controlled semantic access process induced by the task used. Indeed, our paradigm not only comprised a high proportion of related pairs (50%) and a long SOA (850 ms), but the task itself explicitly required a relatedness judgement in each pair independent of the language. In this respect, the semantic categorisation differs considerably from other language-specific and language-general lexical decision tasks (i.e., respectively deciding whether a string is a real word in one language rather than in another or to decide if a letter string is a word in either language, von Studnitz & Green, Reference von Studnitz and Green2002). The absence of mixing effect or the lack of interaction between this factor and the others indicate that the change of language within mixed pairs (either from L1 to L2 or L2 to L1) did not necessitate, as previously suggested (see Alvarez et al., Reference Alvarez, Holcomb and Grainger2003), additional processing time whether for L1 or for L2 words. Our results suggest that, in these mixed pairs, the semantic content of the prime but not the primes’ language modulated the RTs to the target (i.e., increasing it for SR and decreasing it for SU). In this unpredictable and highly mixed bilingual context, the lack of response differences between the unmixed and mixed pairs suggests that participants always responded to targets according to the same schema, i.e., they were always functioning in a “bilingual mode”. Indeed, we observed that, in this highly mixed bilingual context, the average RTs measured for the monolingual SR and SU conditions (L1-L1 pairs: respectively 702 and 779 ms) were considerably longer than RTs previously observed in a monolingual experiment using exactly the same paradigm (e.g., 621 and 650 ms respectively in SR and SU words, Khateb et al., Reference Khateb, Michel, Pegna, O’Dochartaigh, Landis and Annoni2003). This difference in RTs in the unmixed conditions indicates the presence of an important decrease in response speed in both languages despite the fact that the subjects were not explicitly required to identify the language in use. Also, longer RTs in bilinguals (compared to monolinguals) during the same tasks have already been reported in the literature and were attributed to word frequency effects (Lehtonen, Hultén, Rodríguez-Fornells, Cunillera, Tuomainen & Laine, Reference Lehtonen, Hultén, Rodríguez-Fornells, Cunillera, Tuomainen and Laine2012). The findings presented here seem to provide some support to the view that access to lexicon occurs in a language-nonselective manner as proposed by the BIA+ model (Dijkstra & Van Heuven, Reference Dijkstra and van Heuven2002).
4.2. Electrophysiological findings
The N400 period and the late time segments
The analysis of the amplitude of N400 component showed only a semantic relatedness effect. The interaction between this factor with language and language mix failed to reach significance. However, as could be observed in Figure 1, a slight modulation of the N400 response appeared between languages and as a function of language mix. The small size of the sample used in this study might have compromised the investigation of the language effects, explaining thus the null result found here. In terms of latency, the N400 component measured in SU conditions peaked slightly earlier in L1 than in L2 conditions. This finding is in line with previous findings showing a latency shift during L2 word processing (see below). The N400 component, repeatedly reported in relation to semantically incongruous words during sentence processing (Kutas & Federmeier, Reference Kutas and Federmeier2009; van den Brink, Brown & Hagoort, Reference van den Brink, Brown and Hagoort2001), was also found in various priming/matching contexts (see Khateb et al., Reference Khateb, Pegna, Landis, Mouthon and Annoni2010). Interestingly, the spatio-temporal analysis reported here showed that the segment S4 which occurred at ~230 ms to 310 ms was longer in SR than in SU. Such a finding confirms previous reports using spatio-temporal analysis and suggests that semantic processing starts much earlier than the N400 time range (see Brandeis, Lehmann, Michel & Mingrone, Reference Brandeis, Lehmann, Michel and Mingrone1995; Khateb, Annoni, Landis, Pegna, Custodi, Fonteneau, Morand & Michel, Reference Khateb, Annoni, Landis, Pegna, Custodi, Fonteneau, Morand and Michel1999).
As for bilinguals, an impressive amount of ERP studies have also investigated the modulation of the N400 component in the sentence context (see for instance Ardal, Donald, Meuter, Muldrew & Luce, Reference Ardal, Donald, Meuter, Muldrew and Luce1990; Moreno & Kutas, Reference Moreno and Kutas2005; Proverbio, Cok & Zani, Reference Proverbio, Cok and Zani2002; Proverbio, Leoni & Zani, Reference Proverbio, Leoni and Zani2004; Weber-Fox & Neville, Reference Weber-Fox and Neville1996) and in priming tasks (see for instance Alvarez et al., Reference Alvarez, Holcomb and Grainger2003; Chauncey, Grainger & Holcomb, Reference Chauncey, Grainger and Holcomb2008; Elston-Guttler, Paulmann & Kotz, Reference Elston-Güttler, Paulmann and Kotz2005; Kerkhofs, Dijkstra, Chwilla & de Bruijn, Reference Kerkhofs, Dijkstra, Chwilla and de Bruijn2006; Palmer, van Hooff & Havelka, Reference Palmer, van Hooff and Havelka2010; Paulmann, Elston-Guttler, Gunter & Kotz, Reference Paulmann, Elston-Güttler, Gunter and Kotz2006; Thierry & Wu, Reference Thierry and Wu2007). Regarding the time course of the N400 effect in bilinguals, it has often been reported that the N400 peaked later (or was more prolonged) when the bilinguals processed the less dominant language or the language to which they were exposed later in life (Alvarez et al., Reference Alvarez, Holcomb and Grainger2003: Ardal et al., Reference Ardal, Donald, Meuter, Muldrew and Luce1990; Moreno & Kutas, Reference Moreno and Kutas2005; Weber-Fox & Neville, Reference Weber-Fox and Neville1996). When compared to monolinguals, the bilinguals also showed a longer N400 latency even when they processed their L1 (Ardal et al., Reference Ardal, Donald, Meuter, Muldrew and Luce1990). Concerning the N400 amplitude, the picture is a little more complicated. In fact, some studies have reported larger N400 amplitude in monolinguals than in bilinguals (Ardal et al., Reference Ardal, Donald, Meuter, Muldrew and Luce1990; Thierry & Wu, Reference Thierry and Wu2007), while others suggested that N400 amplitude was larger in bilinguals relative to monolinguals (Proverbio et al., Reference Proverbio, Cok and Zani2002) and larger during L2 than during L1 word processing (Lehtonen et al., Reference Lehtonen, Hultén, Rodríguez-Fornells, Cunillera, Tuomainen and Laine2012; Proverbio et al., Reference Proverbio, Cok and Zani2002). Such discrepancies, which might be explained by several factors such as proficiency in L2 or the tasks used, necessitate further research including in particular larger sample sizes and different levels of L2 proficiency to better assess the modulation of this component. All together, these findings (in terms of latency and the slight but non-significant amplitude modulation) and the previously reported ones suggest that differences between the two languages might also exist during the post-lexical meaning-integration processes (de Groot & Nas, Reference de Groot and Nas1991; Keatley & de Gelder, Reference Keatley and de Gelder1992).
The N400 effect found here in terms of ERP amplitude and reflected in spatio-temporal analysis by the map segment S8 has been discussed extensively in terms of semantic integration processes. Our spatio-temporal microstate analysis did not show a modulation of the duration of this information processing step either by language or by language mix. Here again, the small number of participants used in this study might have affected the study of the language effects and further research (including a larger sample) will be necessary to definitively answer this question and to characterize factors that might modulate this processing step. Given this limitation, a discussion regarding the cognitive/functional significance of the N400 would thus go at present beyond the scope of this paper. However, the present findings using inverse solution analysis provide additional insights into the cerebral origin of this component. Actually, this analysis showed the bilateral involvement of posterior and temporal areas. This result is in line with many other studies using fMRI (Kuperberg, McGuire, Bullmore, Brammer, Rabe-Hesketh, Wright, Lythgoe, Williams & David, Reference Kuperberg, McGuire, Bullmore, Brammer, Rabe-Hesketh, Wright, Lythgoe, Williams and David2000; Newman, Pancheva, Ozawa, Neville & Ullman, Reference Newman, Pancheva, Ozawa, Neville and Ullman2001; Rossell, Price & Nobre, Reference Rossell, Price and Nobre2003), intracranial ERPs (McCarthy, Nobre, Bentin & Spencer, Reference McCarthy, Nobre, Bentin and Spencer1995; Nobre & McCarthy, Reference Nobre and McCarthy1995) and MEG recordings (Gallagher, Béland, Vannasing, Bringas, Sosa, Trujillo-Barreto & Lassonde, Reference Gallagher, Béland, Vannasing, Bringas, Sosa, Trujillo-Barreto, Connolly and Lassonde2014; Geukes, Huster, Wollbrink, Junghofer, Zwitserlood & Dobel, Reference Geukes, Huster, Wollbrink, Junghofer, Zwitserlood and Dobel2013; Halgren, Dhond, Christensen, Van Petten, Marinkovic, Lewine & Dale, Reference Halgren, Dhond, Christensen, Van Petten, Marinkovic, Lewine and Dale2002; Helenius, Salmelin, Service & Connolly, Reference Helenius, Salmelin, Service and Connolly1998; Matsumoto, Iidaka, Haneda, Okada & Sadato, Reference Matsumoto, Iidaka, Haneda, Okada and Sadato2005), suggesting a temporal origin for the N400 response. Also, it fits with our previous analysis that compared the N400 sources in three matching tasks (semantic, phonological and pictorial) where the effect involved temporal regions (Khateb et al., Reference Khateb, Pegna, Landis, Mouthon and Annoni2010). A review of neuroimaging studies suggested that temporal areas were those most likely involved in the N400 effect (Lau, Phillips & Poeppel, Reference Lau, Phillips and Poeppel2008). Here, the fact that the inverse solution difference map (i.e., t-tests map) showed the involvement of only right temporal areas is explainable by the observation that both S8 and S7 recruited left temporal areas and thus the difference could only be found in the right areas when contrasting S8 to S7.
Our analysis showed that in unmixed and mixed conditions as well as in SR and SU conditions, RTs for L2 words were longer than for L1 words. Hence, the question remains as to how can one explain this robust language effect despite the participants’ relatively high proficiency in L2? From the preceding discussion, it appears that the language effect observed behaviourally is not primarily associated with the N400-related semantic processes. In fact, our analysis indicated that the functional microstate S3 (starting at ~170), observed in all conditions could explain, at least partially, the language effect since it lasted longer after L2 than L1 words (and its duration correlated with the individual RTs, see discussion below on S3). As a consequence of this initial cost in the beginning of the stream of information processing, the late segments (particularly S9 and S10) occurred later in L2 as compared to L1 conditions. In this regard, we have previously assumed that the modulation of the duration of the successive microstates does not follow a one-to-one relationship (Khateb et al., Reference Khateb, Pegna, Michel, Landis and Annoni2002) and an interplay seems to exist between the duration of some crucial microstates and the duration of the subsequent ones (see other examples in Khateb et al., Reference Khateb, Annoni, Landis, Pegna, Custodi, Fonteneau, Morand and Michel1999; Reference Khateb, Michel, Pegna, Landis and Annoni2000a; Reference Khateb, Pegna, Michel, Landis and Annoni2002). Indeed, we observed here that the late map segments were delayed not only in L2 compared to L1, but also in SU compared to SR. The delay in the time of occurrence of these processing steps in SU conditions is certainly influenced by the emergence of the new and condition-specific S8 map segment in these conditions. The fact that segments S5 to S7 did not differ between conditions strengthens our proposition that, at the individual level, the interplay between the duration of the successive microstates and their time of occurrence allows for a sort of an on-line monitoring of the response time during the long stream of information processing. As for the late microstates, which differentiated both languages and SR vs. SU conditions in terms of their time of occurrence, we have already proposed (Khateb et al., Reference Khateb, Pegna, Michel, Landis and Annoni2002) that they are related to decisional and motor processes (i.e., the passage of information into executive steps, Pollen, Reference Pollen1999) and thus are of little interest to our current purpose.
The early S3 time segment
Our spatio-temporal analysis showed that the longer the subjects remained in S3, the slower were their RTs. This microstate started at ~170 ms post-stimulus in all conditions and “peaked” in individual data (in terms of time of best-fit) at ~215 ms. In terms of its correspondence with the early ERP components, the segment started during the returning/ascending phase of the N170 component (itself reflected by the segment S2) and covered the period of the P2. Many electrophysiological studies have already shown that the time window between 150–200 ms is involved in the processing of visual and orthographic stimuli and have for the most part focused on the modulation of the N170 component (see references in Taha, Ibrahim & Khateb, Reference Taha, Ibrahim and Khateb2013; Taha & Khateb, Reference Taha and Khateb2013). The N170 component is thought to be elicited by the activation of the visual word form area (VWFA, Cohen, Dehaene, Naccache, Lehéricy, Dehaene-Lambertz, Henaff & Michel, Reference Cohen, Dehaene, Naccache, Lehéricy, Dehaene-Lambertz, Henaff and Michel2000) in the left occipito-temporal region, damage to which causes pure alexia (Déjerine, Reference Déjerine1892) or word-form dyslexia (Warrington & Shallice, Reference Warrington and Shallice1980, see review in McCandliss, Cohen & Dehaene, Reference McCandliss, Cohen and Dehaene2003). Also experimental evidence from MEG (Helenius, Tarkiainen, Cornelissen, Hansen & Salmelin, Reference Helenius, Tarkiainen, Cornelissen, Hansen and Salmelin1999) and EEG data (Maurer, Brem, Bucher, Kranz, Benz, Steinhausen & Brandeis, Reference Maurer, Brem, Bucher, Kranz, Benz, Steinhausen and Brandeis2007) suggest that the N170 is altered in dyslexic readers. In parallel, brain imaging studies suggested that the VWFA region shows both functional (van der Mark, Bucher, Maurer, Schulz, Brem, Buckelmuller, Kronbichler, Loenneker, Klaver, Martin & Brandeis, Reference van der Mark, Bucher, Maurer, Schulz, Brem, Buckelmuller, Kronbichler, Loenneker, Klaver, Martin and Brandeis2009) and structural (Kronbichler, Wimmer, Staffen, Hutzler, Mair & Ladurner, Reference Kronbichler, Wimmer, Staffen, Hutzler, Mair and Ladurner2008) abnormalities in dyslexics. Here, the inverse solution indicated the involvement in S3 of the bilateral temporo-occipital areas. Statistical analysis showed that S3 differed from S4 by the activation of the left fusiform gyrus involving a localization (BA 37; at -47, -47, -11) that corresponds to the locus of the VWFA (at -43, -54, -12, see McCandliss et al., Reference McCandliss, Cohen and Dehaene2003). This observation suggests that a certain overlap exists between neural generators of the N170 and the following component (i.e., P2). However, given the timing of S3, we will focus this part of discussion on the time window of the P2 (i.e., ~200 ms onwards) and relate it to some previous findings in the literature.
Using intra-cranial depth data, Nobre, Allison & McCarthy (Reference Nobre, Allison and McCarthy1994) observed that an electrical response recorded in the occipito-temporal-posterior fusiform cortex at ~200 ms differentiated words from images and faces and was insensitive to the semantic content of the words. Using scalp recordings, a modulation of the ERPs at the level of the P2 component has also been found in relation to syllable frequency differences (Barber, Vergara & Carreiras, Reference Barber, Vergara and Carreiras2004). Bar-Kochva and Breznitz, (Reference Bar-Kochva and Breznitz2012) found that while the N1 component was modulated by the presence/absence of diacritics in Hebrew, the P2 component differed between words and pseudowords. Korinth and Breznitz (Reference Korinth and Breznitz2014) reported that the P2 component recorded during word recognition differed between fast and slow readers. In a recent study, Taha and Khateb (Reference Taha and Khateb2013) reported that ERP differences between real words and pseudohomophones were observed during the N1 and P2 components. More interestingly, we have in a previous study using microstate analysis (Khateb et al., Reference Khateb, Pegna, Michel, Landis and Annoni2002) shown that a map segment, appearing during the same time period (between ~170–250 ms, see Figure 2 in Khateb et al., Reference Khateb, Pegna, Michel, Landis and Annoni2002) and having a field topography similar to the one found in S3, was of longer duration during the processing of pseudowords than in words. In bilinguals, Midgley, Holcomb & Grainger (Reference Midgley, Holcomb and Grainger2009) have shown that differences in ERPs between word processing in L1 and L2 started from 150 ms onwards and were interpreted as due to language dominance. Also, differences between languages in the time window 150–300 ms were also found in another study (Alvarez et al., Reference Alvarez, Holcomb and Grainger2003) using a repetition priming paradigm, with the P2 component being larger in L2 than in L1. Considering the fact that several linguistic factors (frequency, lexicality and word length effects) appear to affect ERPs in the time between 120 to 350 ms (see Hauk, Patterson, Woollams, Watling, Pulvermuller & Rogers, Reference Hauk, Patterson, Woollams, Watling, Pulvermuller and Rogers2006; Hauk & Pulvermuller, Reference Hauk and Pulvermuller2004; Proverbio, Proverbio, Zani & Adorni, Reference Proverbio, Zani and Adorni2008), we are tempted to explain the S3 language effect in relation to word frequency differences induced by the language dominance. The effect observed here on S3 and its resemblance with the lexicality effect observed in our previous study (Khateb et al., Reference Khateb, Pegna, Michel, Landis and Annoni2002), together with the fact that its time of occurrence fits well with many other observations in this literature (see above) make this dominance explanation a plausible one. In fact, L2 words are subjectively of a lower frequency than L1 words due to the fact that almost all bilinguals start learning L2 later in life and are thus less exposed to L2 words (Duyck, Vanderelst, Desmet & Hartsuiker, Reference Duyck, Vanderelst, Desmet and Hartsuiker2008; Gollan, Montoya, Cera & Sandoval, Reference Gollan, Montoya, Cera and Sandoval2008). Accordingly, L2 words are thought to be activated more slowly than L1 words and consequently, the time required for their recognition is longer, implying a certain temporal delay in ERP responses and RT measures (see van Heuven & Dijkstra, Reference van Heuven and Dijkstra2010).
5. Conclusion
Our objective in this study was to investigate the electrophysiological correlates of the time cost difference in the processing of L2 and L1 words. For this purpose, RTs and ERPs were collected during a semantic categorization task. The combination of ERP waveform and spatio-temporal analysis (together with source localization) allowed dissociating brain responses modulated by the semantic processing and those modulated by the language of the words. The language of the stimuli modulated the peak latency of the N400 component, but not its amplitude despite some slight differences. Further research, involving a larger sample size, is needed to better assess the language effect on the N400. Microstate analysis showed that the map segment S8 reflected the N400 and was found mainly in SU conditions and bilaterally involved the temporal areas. We found also that an early brain electric microstate was of longer duration in L2 than in L1, independent of semantic relatedness. The timing of this processing step, whose duration correlated with RTs, and the areas supposedly generating it indicated that early differences in the processing of L2 and L1 words occur during early recognition processes. This language difference was interpreted in terms of frequency effects that result from language dominance. Further research is needed to better investigate this language effect and to assess its possible modulation by other factors such as word frequency and the participants’ level of proficiency in L2. Based on this interpretation, we speculate that the duration of this map segment in L2 should increase in low compared to high proficiency bilinguals. Also, this same processing step would be of longer duration after low than after high frequency L1 words.
Supplementary Material
For supplementary material accompanying this paper, visit http://dx.doi.org/10.1017/S1366728915000140
