Introduction
In their article “Heritage languages: In the ‘Wild’ and in the Classroom”, Polinsky and Kagan (Reference Polinsky and Kagan2007) provide an eye-opening statistic: approximately a third of current college students in the US, i.e., 6 million, are heritage language (HL)–English bilingual speakers. They represent a special category of bilinguals from both sociolinguistic and linguistic perspectives. Their sociolinguistic situation is unique in that the overwhelming majority of them were born in another country and started acquiring that country's language as the first language and then some time later relocated with their families to the US. Crucially, the immigration event took place before puberty: these young people were children when they switched to English in public and educational settings while they continued to speak the family language at home (Polinsky, Reference Polinsky2007; Valdés, Peyton, Ranard & McGinnis, Reference Valdés, Peyton, Ranard and McGinnis2001). From the linguistic perspective, HL bilingual speakers seem to be indistinguishable from native English speakers in English but vary enormously in proficiency in their heritage language, from strong comprehension to numerous deficiencies in the lexical, morphological, grammatical and pragmatic domains of their linguistic system (Brinton, Kagan, & Bauckus, Reference Brinton, Kagan and Bauckus2008; Montrul, Reference Montrul2008; Polinsky, Reference Polinsky2007). They constitute an untapped resource for a variety of language-dependent occupations spanning from homeland and international security to business, education and healthcare (Carreira & Armengol, Reference Carreira, Armengol, Peyton, Ranard and McGinnis2001). Meanwhile, the field of heritage languages, in both basic research and pedagogy, is rapidly emerging in an attempt to catch up with pressing practical needs to adequately assess and educate HL speakers.
Studying heritage languages is no less important for the theory of bilingualism, in particular, for the debate on the critical period hypothesis (CPH), i.e., a maturationally delimited period for language acquisition that suggests that complete language learning occurs only when exposure to a language begins early in life, and that outside of this limited developmental period, the ability to acquire language at a native proficiency level declines (Lenneberg, Reference Lenneberg1967). In order to address the main predictions of CPH – what causes younger learners’ advantage over older learners, and which aspects of language learning are implicated – psycholinguists have been looking at the combinations of two factors, age of acquisition (AoA) and language proficiency, by considering only late AoA, various proficiency (L2 learners) or early AoA, high proficiency (balanced bilinguals). Incomplete HL acquisition revealed by both observable errors in heritage language production and less obvious deficits in heritage language comprehension provide evidence that the onset of bilingualism and reduced exposure to the L1 are as critical as the age of L1 acquisition in attaining (or not) native levels of proficiency in bilingual language acquisition (Montrul, Reference Montrul2008).
Describing the language processing system of HL speakers is complicated by the fact that its different components are not independent of each other but necessarily work in tandem to make interactions between sound, structure and meaning possible during language processing; language is, after all, a classical example of a complex system that extends well beyond the sum of its parts. These interactions between phonology and syntax or syntax and discourse are known as interfaces (Platzack, Reference Platzack, van der Zee and Nikanne2001) and are of particular importance when studying such multifaceted phenomena as contrastiveness that result from the interaction of linguistic and non-linguistic factors. Our purpose in this article is to experimentally investigate online processing of contrastive constituents in Russian by monolingual and HL Russian−English bilingual adults. We will compare how and when these two groups of participants compute contrastiveness online using such cues as word order, prosody and context in spoken Russian language comprehension.
Contrastive constituents
Contrastiveness is usually evoked when a contrast set is explicitly or implicitly present in discourse, i.e., there is a set of possible alternatives with respect to which some semantically highlighted material is interpreted (for the theoretical accounts see Lee, Gordon, & Büring, Reference Lee, Gordon and Büring2008; Molnár, Reference Molnár, Kruijff-Korbayová and Steedman2001; Rooth, Reference Rooth1992). The most commonly attested way of expressing contrastiveness linguistically is with the help of prosodically marked narrow focus that highlights salient information. In English, it is realized by the contrastive pitch accent on the stressed syllable of the word in focus (represented by capital letters). For example, in the sentence Pick up the blue CYLINDER (Krahmer & Swerts, Reference Krahmer and Swerts2001), the pitch accent is located on the noun, the default focus position in the sentence. Without context, this sentence is ambiguous with respect to focus: it can signal either a narrow contrastive focus (the blue cylinder vs. the blue cube) or a broad focus (the blue cylinder that is different from cubes, pyramids, spheres, etc.). In contrast, in the example Push the RED button, the pitch accent on RED unambiguously signals narrow contrastive focus on the adjective, a non-default focus position: it is clear to the listener that there is a set of alternative buttons, with the red one being contrasted to the button of some other color.
While highlighting salient information by pitch accents appears to be universal (Bolinger, Reference Bolinger1961), other language-specific linguistic and non-linguistic means can be used to mark contrastiveness. These include focus particles (e.g., Korean and Japanese), morphological affixes (e.g., Wolof and Chickasaw), and word order (e.g., Italian, Hungarian, Finnish) (Hasselgård, Johansson, Behrens, & Fabricius-Hensen, Reference Hasselgård, Johansson, Behrens and Fabricius-Hansen2002; Molnár, Reference Molnár, Kruijff-Korbayová and Steedman2001; Steube, Reference Steube2004). Russian is a particularly intriguing case because, in this language, contrastiveness can be encoded not only by contrastive pitch accents and non-canonical word order but also in a fairly unusual way by using the so-called ‘split constituents’ in which one portion of a noun phrase, i.e., a modifying adjective, is separated from the head noun. Word order and prosody then combine to create numerous possibilities for expressing contrastiveness in spoken Russian.
To illustrate the usage of split constituents in Russian, imagine a game in which listeners are presented with a 3 × 3 vertical board (see Figure 1), with five colored cardboard shapes representing common objects (e.g., stars, birds, frogs, etc.) placed upon it. The participants’ task is to follow spoken instructions to find a particular object and move it into one of the empty cells on the board, as illustrated in (1). Listeners presumably construct a mental model of this circumscribed visual world as they scan the objects and listen to the instructions. When the visual context contains a contrast set for color (e.g., a red star and a yellow star) and a contrast set for shape (e.g., a red star and a red bird) coupled with the pitch accent, contrastiveness is implicitly encoded in the resulting mental model.
(1)
The rich case morphology of Russian ensures unambiguous thematic role assignment of the NP krasnuju zvezdočku ‘redACC-FEM starACC-FEM’ as the direct object regardless of its position in the sentence because it is marked with the Accusative case, default for direct objects in Russian. Example (1a) represents the canonical word order of Russian, with the direct object red star following the verb, while (1b) expresses the same proposition by moving it to the sentence-initial position via an operation referred to as scrambling in generative grammar. Example (1c) is a grammatically marked option of split scrambling that encodes contrastiveness with split constituents by separating the adjective from the head noun and moving it to the sentence-initial position. Despite the surface word order variation, the three examples mean the same in terms of their truth value. The pitch accent on the adjective is realized as the ‘hat’ contour, i.e., a combination of High and Low accents in the ToBI notation (Beckman, Hirchberg, & Shattuck-Hufnagel, Reference Beckman, Hirschberg, Shattuck-Hufnagel and Jun2005) which takes the form H+L* in Russian (Meyer & Mleinek, Reference Meyer and Mleinek2006; Odé, Reference Odé, Houtzagers, Kalsbeek and Schaeken2008).
Figure 1. The two types of scenes compared in Experiment 1 (Monolingual) and 2 (Bilingual HL Russian). (A) The 1-Contrast scene: Target red star (position 6), Color Competitor red bird (position 7), Target's contrast object yellow star (position 1), two distracters, blue frog and green frog (positions 2 and 8). (B) The 2-Contrast scene is the same except that the blue frog (position 2) has been replaced with a blue bird that serves as the contrast object for the Color Competitor (position 7).
The split-constituent construction in Russian is formally characterized by (a) obligatory placement of the split-constituent adjective and the split-constituent noun to the leftmost and rightmost periphery of the VP, respectively, and (b) special prosodic features, i.e., a contrastive pitch accent, lengthening of the stressed syllable, and a boundary low tone (L%) at the end of the word (Sekerina, Reference Sekerina1997; see Pereltsvaig, Reference Pereltsvaig2008, for syntactic analysis). These features must be assigned to only one of the two components, either the split-constituent adjective (1c), as H+L*, or the split-constituent noun (e.g., Krasnuju položite ZVEZdočku . . . “Red put STAR . . .”), as L+H*. While the split-constituent construction is undoubtedly a marked option compared to canonical and regular non-split scrambled word orders, it is quite frequent in colloquial Russian, poetry and folk literature. However, in the absence of a formal spoken corpus count, observations about its frequency remain impressionistic and need to be verified in the future.
The processing of contrastiveness
Monolinguals
There is solid empirical evidence from psycho-acoustic studies in different languages (see Cutler, Dahan, & van Donselaar, Reference Cutler, Dahan and van Donselaar1997, for an overview) that monolingual listeners easily perceive contrastive pitch accents and use them to rapidly infer the meaning of an utterance in the appropriate linguistic context (e.g., for English: Welby, Reference Welby2003; for German: Baumann, Grice, & Steindamm, Reference Baumann, Grice, Steindamm, Hoffmann and Mixdorff2006, and Féry & Kügler, Reference Féry and Kügler2008; for Dutch and Italian: Krahmer & Swerts, Reference Krahmer and Swerts2001; and for Russian: Makarova, Reference Makarova2007, Mehlhorn, Reference Mehlhorn and Steube2004, and Meyer & Mleinek, Reference Meyer and Mleinek2006). Eye-tracking methodology allows us to go beyond the simple psycho-acoustic analysis of contrastiveness by bringing in other factors, both linguistic and non-linguistic, that contribute to the computation of contrast sets. For example, visual context plays a critical role in contrastiveness when utterances with contrastive pitch accent are presented against the backdrop of a circumscribed visual world (Trueswell & Tanenhaus, Reference Trueswell and Tanenhaus2004), as in the game described above in (1).
Sedivy and colleagues (Sedivy, Reference Sedivy2003; Sedivy, Tanenhaus, Chambers & Carlson, Reference Sedivy, Tanenhaus, Chambers and Carlson1999) have applied the Visual World paradigm to study contrastiveness in English. Listeners were presented with a pair of utterances like Touch the pink comb. Now touch the yellow comb, pronounced with neutral prosody (no contrastive pitch accent) while viewing pictures of four objects – a yellow comb, a pink comb, a yellow bowl and a metal knife. The first instruction served as a preamble for the second, experimental, instruction that contained the target yellow comb, and together with the visual setup imposed an explicit contrastive interpretation for the comb in terms of color (i.e., pink vs. yellow comb). As soon as listeners heard the adjective yellow, even in the absence of contrastive pitch accent, they were expected to prefer the yellow comb, and not the yellow bowl, because their attention was drawn to the color contrast set established directly by mention of the pink comb in the preamble. On the other hand, when the experimental instruction was changed to Now touch the yellow bowl, the context did not support a contrastive interpretation any more, and both the yellow comb and the yellow bowl were equally possible targets. These expectations were borne out: there was a robust bias to interpret the target yellow comb but not the target yellow bowl contrastively.
Surprisingly, in the follow-up experiment, when the pitch accent L+H* was added to the adjective YELLOW, it did not facilitate eye-movements to the target referent compared to the control one. Sedivy (Reference Sedivy2003) proposed a Gricean account for the absence of the prosody effect when contrast sets for color are involved. She noted, and demonstrated experimentally, that speakers overuse color adjectives, sometimes uttering them even when such a term is not needed to pick out a unique referent, e.g., saying the yellow comb when there is only one comb in view. This deviation from a default description, she argued, is what triggers contrastive interpretations of scalar but not color adjectives (e.g., saying the large comb deviates from a default description more than saying the yellow comb); therefore, contrastiveness is not inherently a linguistic phenomenon but is achieved instead via pragmatic (discourse) inference.
Ito and Speer (Reference Ito and Speer2008) (also see Weber, Braun & Crocker, Reference Weber, Braun and Crocker2006, for similar results in German) challenged Sedivy's null effect of contrastive prosody using almost identical materials but a different setup. Monolingual English participants decorated a miniature Christmas tree using forty ornaments arranged on a 10-cell grid with each cell containing from three to five ornaments of the same type (e.g., drums, angels, bells, etc.) but of different colors. Just as in Sedivy's studies, the visual scene stayed the same while the target object was changed. The trial again consisted of a pair of instructions, with the preamble (2a) directly establishing a contrast set either for color Adjective (the green drum vs. the blue drum) or (3a) for category Noun (the onion vs. the drum).
- (2)
a. First, hang the green drum.
b. Now, hang the BLUE drum.
- (3)
a. First, hang the blue onion.
b. Now, hang the blue DRUM.
In addition to the type of contrast (Adj vs. N), Ito and Speer also manipulated the location of contrastive prosody (pitch accent L+H* on Adj or on N) in the experimental instruction (2b and 3b). There were two complementary conditions, i.e., the blue DRUM for (2) and the BLUE drum for (3), that created infelicitous contrasts but they are omitted here for brevity.
Ito and Speer refuted Sedivy's null effect of contrastive pitch: when the adjective carried the felicitous contrastive pitch accent, as in (2b), the visual display, the explicit setup of the contrast set in the preamble and pitch accent all converged to create a very strong bias to interpret the color adjective BLUE as contrastive. As a result, the listeners started to look at the target referent blue drum even before they could process the acoustic information for the noun, i.e., they anticipated the noun. The pitch accent on the noun (3b), however, did not facilitate early identification of the target referent; it started to affect eye-movements only after the noun was encountered in speech. Ito and Speer speculated that the discrepancy between their results and those of Sedivy had to be explained by the methodological confounds of the latter: excessive informativeness of color adjectives enhanced by very long preview times, lack of control over the prosodic characteristics of the instructions delivered live, and a shorter region of interest for eye-movement analysis, all of which could have led to the ceiling effect in Sedivy's experiments.
L2 bilinguals
How do bilingual listeners process contrastiveness? Little is known about how individual factors (i.e., context, prosody and word order) affect the bilingual processing of contrastive spoken utterances in real time, let alone how these factors interact. We are aware of only one experimental study of the processing of contrastive pitch accents in bilingual speakers, by Akker and Cutler (Reference Akker and Cutler2003). They tested L2 learners, namely, Dutch-native speakers of English, and employed a metacognitive task of phoneme detection. In Experiment 4, 48 Dutch learners of English listened to pairs of utterances – a question and an answer to this question – in English, as illustrated in (4)–(5) (Table 1, p. 84).
- (4)
a. Which bones were found by the archeologist?
b. The bones of the DINOSAUR were found by the Cuban archeologist.
- (5)
a. Which archeologist found the bones?
b. The bones of the dinosaur were found by the CUBAN archeologist.
The pairing of the question in (4a) with the answer in (4b) (also for (5a–b)) was the target-in-focus condition while pairing (4a–5b) (also for (5a–4b)) was the target-not-in-focus condition. The second manipulation was the location of contrastive pitch accent in the answer, on the target word (4b) or not (5b). The participants had to perform a metalinguistic task of phoneme detection (e.g., listening and identifying /d/ in (4) or /k/ in (5)).
Table 1. Mean durations of the three regions in spoken instructions, ms (frame) (1 fr. = 1/30 s).
Both English (Exp. 1) and Dutch monolingual controls (Exp. 2) detected the focused and accented target DINOSAUR in the pair (4a–b) faster than when it was unfocused and unaccented (4a–5b), and the two factors interacted: the pitch accent facilitated the phoneme detection in the unfocused answers but made no difference in the focused ones. Bilingual Dutch–English listeners (Exp. 4) were 90 ms slower in this phoneme detection task but showed the same facilitation for focus and pitch accent. What was different for the bilingual group was that the two factors did not interact: the processing advantage of pitch accent was significant for both the focused and unfocused answers. The authors concluded that non-native listeners, just like monolinguals, used each of the two sources of information (e.g., prosody and semantic implications of focus) independently. However, it took them longer to perform the task because of the necessity to combine and process the two factors that slowed down their overall sentence processing.
Heritage language bilinguals
Akker and Cutler's (Reference Akker and Cutler2003) results complement the well-established fact that although some categories of bilinguals, such as early and balanced ones, are often indistinguishable from monolinguals in accuracy, they are slower in production, such as speeded verbal tasks (Portin & Laine, Reference Portin and Laine2001; Ransdell & Fischler, Reference Ransdell and Fischler1987), picture naming (Ivanova & Costa, Reference Ivanova and Costa2008), and the reading of complex sentences (Frenck-Mestre, Reference Frenck-Mestre, Heredia and Altarriba2002). In a large number of experiments with bilingual adults and children, Bialystok and colleagues (see Bialystok, Reference Bialystok2009, for an overview) have convincingly shown that bilinguals have a smaller vocabulary (children) and are slower in lexical retrieval (adults) in both languages. While it is not clear why this is the case, possible explanations include less automaticity in the usage of both languages (Paradis, Reference Paradis2004), persistent competition from the other language even when only one is being used (Bialystok, Reference Bialystok2009), the effects of the age of acquisition (Montrul, Reference Montrul2008), and costly reanalysis (Frenck-Mestre, Reference Frenck-Mestre, Heredia and Altarriba2002). Thus, even though monolingual and bilingual language processing is qualitatively the same, at least in production, it takes bilinguals longer to process their weaker language and perhaps even their dominant one as well (Ivanova & Costa, Reference Ivanova and Costa2008).
HL bilingual language processing
The verbal fluency and processing speed of HL bilingual speakers are yet to be formally investigated but, depending on the nature of their bilingualism (whether the home language was fully developed before acquiring the dominant language or not) and the command of the two languages, we would expect processing speed in their heritage language to vary. If we find that even high-proficiency HL bilingual speakers are indeed slower than monolinguals, it is important to establish what this slowness can be attributed to because it has a direct impact on their linguistic competence. One possibility, as Montrul (Reference Montrul2008) suggests, is that slowed processing effects “[m]ay simply reflect a superficial loss of automaticity and fast retrievability rather than actual loss of deep linguistic knowledge” (p. 69). But there is another possibility as well: slowdown may originate at the perceptual level, such as word recognition and lexical retrieval, and then propagate through other language domains, from morphology to the integration of multiple sources of information at the interfaces. This hypothesized effect could culminate at the lexical–semantic level and ultimately affect the grammatical competence of HL speakers by speeding up the process of heritage language attrition.
In L2 research, a subfield of bilingual studies that is concerned with how adults progress in learning a second language from beginners to advanced competence levels, performance deficits are easy to detect because not only are bilingual L2 learners slower than monolinguals but they also make overt errors. In contrast, in balanced bilinguals these deficiencies often go unnoticed when only offline measures of accuracy are used. HL bilingual speakers constitute a diverse category of bilinguals who can be quite different from both L2 learners and balanced bilinguals. They fall on a continuum from ‘overhearers’ to native-like fluent speakers (Polinsky & Kagan, Reference Polinsky and Kagan2007), and many of them are highly proficient comprehenders of their heritage language. Nevertheless, there are preliminary experimental results from production that demonstrate that HL speakers, just like L2 learners, take longer to perform verbal fluency tasks.
Hulsen (Reference Hulsen2000) showed that the second-generation heritage Dutch speakers who were born in New Zealand (86.7% of the sample) were 695 ms slower than the first generation (adult immigrants) in naming pictures. For those participants, Dutch was undoubtedly their weaker language (self-rating proficiency for Dutch was 2.6 out of 5 compared to 4.96 in English). Polinsky and Kagan (Reference Polinsky and Kagan2007) measured the speech rate of low-proficiency heritage Russian speakers, i.e., their word-per-minute output in speech production, and found that they produced 30% fewer words than monolinguals in a picture description task. O'Grady (Reference O'Grady2009) found that heritage Korean speakers born in the US were on average 425 ms slower in rapid naming latencies for the basic vocabulary words in Korean than for the exact same words in English. However, it remains an open question whether there is a comparable slowdown in comprehension, the strongest performance component of high-proficiency HL speakers whose spoken heritage language comprehension and pronunciation are close to those of native speakers.
The processing of contrastiveness by HL Russian bilingual speakers: predictions
Recent L2 research has found that while L2 bilinguals successfully acquire the syntactic constraints that govern a particular interface-related construction (for example, the distribution of null subjects, clitic-doubling and à + animate direct objects in Spanish), they often fail to learn the pragmatic conditions associated with it (Montrul, Reference Montrul2002; Rothman, Reference Rothman2009). Sorace (Reference Sorace, Cornips and Corrigan2005) proposed an interface vulnerability hypothesis according to which the correct processing of interface-related phenomena is particularly difficult for L2 learners because of the extra processing load caused by the necessity to combine linguistic and non-linguistic factors (see also Akker & Cutler, Reference Akker and Cutler2003). We set out to investigate whether highly proficient HL Russian–English bilinguals (Experiment 2) have difficulty with the processing of contrastive constituents in Russian compared to monolinguals (Experiment 1). We predicted that rapid coordination of multiple sources of information from different linguistic domains, e.g., the discourse–syntax–prosody interface required for the computation of contrastiveness in Russian, would be taxing for HL speakers and result in quantitatively slower processing even in short and easy sentences with simple lexical items (color adjectives and common nouns).
In order to uncover such quantitative differences in online language processing between monolingual and HL speakers, we needed moment-by-moment measures that did not rely on offline end-of-sentence accuracy. Therefore, we used the Visual World paradigm (VWP) that is sensitive to the dynamics of real-time activation of referents and offers a natural task that relies on oral comprehension, one of the strongest skills in HL bilinguals. Eye-movements reliably tap into the earliest phases of online processing, and have revealed robust correlations with linguistic behavior (eye-movements are launched to the referent as soon as it is mentioned in speech). The use of the VWP also allowed us to eliminate metalinguistic tasks, such as phoneme detection (cf. Akker & Cutler, Reference Akker and Cutler2003), that could contribute additional processing load to the HL processing system. In addition, because the split-constituent construction (Example 1c above) is a predominantly spoken-language phenomenon in Russian, only eye-movements in the VWP could reveal when and how factors contributing to contrastiveness, e.g., word order, prosody and context, were computed. Finally, we also explored a relationship between sociolinguistic factors, such as years of exposure to Russian before switching to English and self-rated proficiency, that could possibly affect variability in processing efficiency in HL bilinguals.
When the visual context contains a target object (e.g., a red star) that contrasts with a different object in color (e.g., a red bird) and the same object in color (e.g., a yellow star), the simple mentioning of color creates referential ambiguity: Which of the two red objects is the target? If the split-constituent word order, i.e., the sentence-initial split color adjective red followed by a verb instead of a noun, is preferentially interpreted as contrastive, the visual context and word order should direct monolingual listeners to interpret it as modifying the target object (the star), and not the competitor object (the bird), and the contrastive pitch accent should enhance this effect. The average 450-ms time period (the duration of the verb) between the split adjective and its head noun in the spoken input allows us to see if such an interpretation occurs early and leads participants to anticipate the target object even before the head noun appears in speech. In previous eye-tracking experiments on contrastiveness (Ito & Speer, Reference Ito and Speer2008; Sediviy, Reference Sedivy2003; Sedivy et al., Reference Sedivy, Tanenhaus, Chambers and Carlson1999; Weber et al., Reference Weber, Braun and Crocker2006), the effects of contrastiveness (when found) occurred approximately 200–400 ms after the onset of the noun. If we could find evidence for anticipatory eye movements, i.e., the ones that happen before the onset of the noun, in the split-constituent construction, this would suggest that monolingual listeners rapidly compute the referential implications of interface-based non-canonical word orders by effectively integrating multiple sources of information (e.g., prosody, word order and context) ‘on the fly’. When the visual context contains two pairs of objects (e.g., a red and a yellow star vs. a red and a blue bird), referential ambiguity persists until the head noun is encountered, at which point its lexical identity becomes the decisive factor in its resolution.
Experiment 1: monolingual Russian listeners
Method
Participants
Twenty-four monolingual undergraduate students (20 women, mean age 20.7) from the Moscow State University, Russia, volunteered to participate in the experiment in exchange for $3 (equivalent in rubles). All participants were naive with respect to the nature of the experiment, which took approximately 45 minutes to complete.
Design and materials
Participants were presented with a small vertical board that contained a central fixation point and five colored objects made out of cardboard. On critical trials, participants heard spoken instructions containing a split constituent like that given above in (2c), which was accompanied by one of two possible types of visual context (Figure 1). In both context types, two objects were the same color as the color named by the adjective in the instruction; one was the ‘Target’ (a red star) and the other the ‘Color Competitor’ (a red bird).
We carefully selected the Target–Competitor pairs of objects so that their names were of the same grammatical gender in Russian, preventing participants from using gender agreement on the adjective as a disambiguating cue. Hence, all things being equal, the Russian adjective krasnuju “redACC-FEM” by itself was referentially ambiguous: it could be used to refer to either the Target zvezdočku “starACC-FEM” or the Color Competitor ptičku “birdACC-FEM” (the split NP was the direct object following the verb “put” and thus was in the Accusative case). The first experimental factor was the Visual Context. In the 1-Contrast context (Fig. 1A), only the Target belonged to a contrast set for color (i.e., there was a yellow star but no additional bird). The two remaining objects for this context were of the same category (frogs) but different in color (e.g., blue and green), and were never the same color as the Target–Competitor pair. In the 2-Contrast context, both the Target and the Competitor were part of contrast sets for color (Figure 1B), i.e., a red and yellow star and a red and blue bird. This was achieved by replacing one of the distractors, the blue frog, with the contrast object for the Color Competitor, i.e., the blue bird. This design allowed us to compare two visual contexts, both of which have two sets of objects, but only the 1-Contrast context was expected to elicit early contrastive interpretation of the adjective “red”.
The second experimental manipulation, Prosody, varied the pitch accent placement. For each of the two types of context, one of two placements of contrastive pitch accent was used in the instruction. Early Accent sentences had the contrastive pitch accent on the adjective (KRASnuju položite zvezdočku . . . “RED put star . . .”) whereas Late Accent sentences had the contrastive pitch accent on the head noun (Krasnuju položite ZVEZdočku . . . “Red put STAR . . .”). The acoustic analysis revealed the highly marked prosodic differences between the sentences with different pitch accent placements. In the Early Accent condition (Figure 2A), the stressed syllable KRAS- “RED” of the adjective had a longer duration, greater amplitude and a sharply falling pitch accent H+L* which are typical characteristics of contrastive accents in Russian (Bryzgunova, Reference Bryzgunova1977; Odé, Reference Odé, Houtzagers, Kalsbeek and Schaeken2008). In the Late Accent condition (Figure 2B), the stressed syllable on the Noun ZVEZ- “STAR” had a greater amplitude and sharply rising pitch accent L+H*.
Figure 2. Waveform and F-zero analysis of representative example sentence, for (A) Early Accent and (B) Late Accent conditions.
Spoken instructions consisted of adjective−noun pairs, the verb “put”, and the locative prepositional phrase. Twelve different color adjectives were used repeatedly (“white”, “yellow”, “red”, “orange”, “pink”, “tan”, “brown”, “green”, “blue”, “navy”, “lilac” and “black”) combined with singular nouns in the Accusative case. All adjectives and nouns were common and highly frequent Russian words, and no special efforts were taken to match them in frequency of occurrence in the language. Instructions were recorded by a female native speaker of Russian in a soundproof booth, sampling at 22,050 Hz. Durations of the adjective, the verb and the noun were measured (Table 1). The adjective length varied from two to five syllables, with a mean duration of 668 ms. The nouns were shorter, varying from two syllables to four syllables, with a mean duration of 550 ms. The mean duration of the verb položite “put” was 478 ms. The words with contrastive pitch accent were on average 120 ms longer than the same words without such accent.
Four stimuli lists were constructed consisting of twenty experimental trials randomly intermixed with twenty-two fillers. Fillers also contained at least one contrast set for color but it was never referred to in the first instruction. In total, there were forty experimental and filler items, and two additional practice trials in the experiment. Each experimental item appeared in one of the four possible conditions of the 2 × 2 (Visual Context x Prosody) factorial design, generating five items per condition. Three additional lists were constructed by rotating experimental items through the four conditions in a Latin square design. Six participants were randomly assigned to each of these four lists.Footnote 1
Procedure
Participants' eye-movements were recorded by the ISCAN ETL-500 head-mounted eye-tracking system (for technical details, see Trueswell, Sekerina, Hill & Logrip, Reference Trueswell, Sekerina, Hill and Logrip1999, pp. 98−99). Each participant underwent a short calibration procedure prior to the experiment. Eye-movements were sampled at a rate of 30 times per second and were recorded on a digital SONY DSR-30 videotape-recorder. Auditory stimuli were played to the participant through speakers connected to a separate speech-controlling computer and were recorded simultaneously with eye movements (see Figure 3).
Figure 3. The ISCAN portable head-mounted eye-tracking system (ETL-500).
Participants were seated in front of the vertical board located at arm's length. Prior to each trial, the experimenter positioned five objects on the board and named them. Each trial consisted of three pre-recorded instructions, as illustrated in (6) for the Early Accent condition.
(6)
The initial locations of the Target and the Competitor, as well as their destination locations, were randomized across the trials. Participants were told to listen to the instructions and perform the required action as quickly as possible.
Data treatment and analysis
Eye-movement data were analyzed from videotape using a SONY DSR-30 VCR with frame-by-frame control and synchronized video and audio. Eight fixations were coded according to the following categories: the cross, the Target (“red star”), the Color Competitor (“red bird”), the contrast object for the Target (“yellow star”), the contrast object for the Competitor (“blue bird”) (only in the 2-Contrast condition), distractors (“frogs”), looks between objects, and track loss. The participants completed the action (i.e., moved the Target object) by 2500 ms after the onset of the adjective in 94% of the trials. The trials with a track loss during the adjective or verb constituted 4.6% of the data and were dropped from analyses.
In order to investigate the time course of each experimental factor's contribution in the processing of contrastiveness, we established three temporal regions of interest that corresponded to the first three words in the experimental instruction (5b). Region 1 corresponded to the adjective, Region 2 to the verb “put”, and Region 3 to the head noun. Regions 1 and 2 combined span the referentially ambiguous portion of the utterance; this ambiguity was resolved as soon as the Noun's acoustic information became available in the input. The duration of each region was factored into the analysis for each experimental trial to balance the durational differences between the adjective and the noun because the words with contrastive pitch accent were longer than the same words without the pitch accent.
We analyzed the participants’ eye-movements using coarse- and fine-grain measures. The coarse-grain analysis revealed general patterns of eye-movements and was based on the calculation of total proportions of time during the utterance spent looking at the Target (or the Color Competitor) and probabilities of looking first at the Target (or the Color Competitor). Fine-grained analysis provided a moment-by-moment record of fixations to the Target as the dependent variable throughout the duration of the utterance with 33 ms resolution. Eye-movement data were subjected to 2 × 2 × 4 factorial ANOVAs with Visual Context (1-Contrast vs. 2-Contrast) and Prosody (Early vs. Late Accent) as within- and List as between-subjects independent variables.
Results
For our experiment, we operationalized participants’ resolution of referential ambiguity as the proportion of time they spent fixating the Target and the Color Competitor in each of the four conditions, beginning at approximately 200 ms after the onset of the adjective. This adjustment was necessary to accommodate the time necessary for the programming of an eye-movement (conservatively estimated to be 180 ms in Altmann & Kamide, Reference Altmann, Kamide, Henderson and Ferreira2004).
Figure 4A plots the mean proportion of time monolingual listeners spent looking at the Target (e.g., the red star) vs. the Color Competitor (e.g., the red bird) during the referentially ambiguous portion of the utterance, Regions 1 and 2 Krasnuju položite . . . “Red put . . .”. Here a preference to look at the Target would suggest that listeners preferred to interpret the adjective “red” as describing the Target rather than the Competitor. Indeed, the ANOVA revealed that listeners showed a reliable preference to look at the Target over the Competitor (27% vs. 21%) in the 1-Contrast context (F1[1,20] = 11.09, p < .01; F2[1,19] = 4.32, p < .05) but not in the 2-Contrast context (24.5% vs. 22%) (F1 = 1.75; F2 = 1.04). Surprisingly, the prediction that this preference would be facilitated by the contrastive pitch accent on the adjective (Early Accent) was not borne out (all Fs < 1.6) although means were in the expected direction, with Early Accent numerically increasing the Target preference in the 1-Contrast context.
Figure 4. Monolingual participants. (A) Mean proportion of time spent looking at the Target or the Color Competitor while hearing the ambiguous regions Adjective–Verb Krasnuju položite . . . “Red put . . .” but prior to hearing the noun zvezdočku “star”. (B) The probability of first look at the Target or the Color Competitor during the ambiguous regions Adjective–Verb. (Error bars indicate one standard error from mean.)
We also examined the probability that listeners would look first to the Target compared to the Color Competitor upon hearing the adjective krasnuju “red” prior to hearing the noun (Figure 4B). A similar pattern was found: listeners preferred to look first to the Target in the 1-Contrast context (53% vs. 39.5%) (significant by subjects, F1[1,20] = 6.21, p < .05; marginal by items, F2[1,19] = 3.75; p = .07) but not in the 2-Contrast context (46.5% vs. 42.5%) (F1 = 1.40; F2 = 0.74). Again, Early Accent did not reliably affect the Target preference in either the 1-Contrast (F1 = 1.31; F2 = 2.97) or 2-Contrast contexts (Fs < 1). The data suggest that the monolingual listeners were sensitive to the contrast in color induced by the visual context: at the color adjective they were more already likely to look to the Target object because it was the only one that could be contrasted in color with another object of the same category.
Finally, the moment-by-moment time course of how multiple sources of information were integrated by monolingual listeners is illustrated in Figure 5 for each of the four conditions (e.g., 1-Contrast and 2-Contrast for Early and Late Accents). It plots looks to the Target (filled circles) vs. the Competitor (hollow circles) over the approximately 1100 ms (33 frames) duration of the ambiguous Regions 1 and 2, adjective and verb. Statistical analyses of looking times were conducted for each of the four windows marked with the vertical lines in the graph, i.e., the first and second half of the adjective (A1 and A2) and the first and second half of the verb (V1 and V2). Collapsing across the Prosody manipulation, we found that for the 1-Contrast context (Figure 5A–B), the Target preference emerged during the second half of the adjective (A2) (F1[1,20] = 9.86, p < .01; F2[1,19] = 5.22, p < .05), persisted through the beginning of the verb (V1), (F1[1,20] = 5.84, p < .05; F2[1,19] = 6.85, p < .05) but dissipated during V2 (Fs < 1.5). No such preference is observed in the 2-Contrast context (Fs < 1.5) (Figure 5C–D) where looks alternated between the two equally plausible red referents because each formed its own contrast set.
Figure 5. Monolingual participants. The probability of looking at the Target vs. the Color Competitor over time. (A) 1-Contrast context, Early Accent. (B) 1-Contrast context, Late Accent. (C) 2-Contrast context, Early Accent. (D) 2-Contrast context, Late Accent. (Time from onset of Adjective in 1/30 sec frames.)
In summary, pragmatic effects of visual context were observed quite early in the monolingual processing of contrastive constituents in Russian. Almost a second before hearing the head noun, just upon hearing the color adjective, monolingual listeners were able to compute the inherent color contrast relationship in the 1-Contrast context which was only present for the Target (two stars), but crucially, not for the Competitor (one bird). However, prosody in the form of the contrastive pitch accent placed early on the adjective had no facilitatory effect on the participants’ ability to identify the Target referent.
Experiment 2: heritage Russian bilingual listeners
Method
Participants
The participants were thirty-two undergraduate HL Russian–English bilingual students at the College of Staten Island who were paid $10 for participation. They all had normal or corrected-to-normal vision and hearing, and had no history of neurological problems. All reported as being highly proficient in both Russian and English. The bilingual participants provided detailed information regarding their language histories. At the beginning of the experiment, each participant filled out a bilingual background questionnaire in English adapted from Fernández (Reference Fernández2003; see Appendix E “Bilinguals’ background information”). Bilinguals’ demographics revealed the following general characteristics of the sample. There were eleven men and twenty-one women. The mean age was 20.8 years and all the participants with one exception were right-handed. The overwhelming majority (21 participants) reported being full-time students at the CSI; the remaining people reporting that they worked part-time.
All of the participants were born to the Russian-speaking parents so their exposure to Russian was from birth and has continued throughout their lives. Russian was reported to be actively used at the time the study was conducted, which allowed us to classify them as heritage language Russian speakers. All of the participants indicated that they came in contact with English when their family relocated from the former Soviet Union to the United States. The geographic areas from which the participants’ families relocated included Ukraine (13 participants), Russia (10), Uzbekistan (5), Belarus (3) and the Caucasus (1). The first contact with English ranged from two to seventeen years (M = 9.2 years, SD = 3.7), which was the age of the participant when she arrived in the US. Half of the participants had parents who were monolingual Russian speakers and half indicated that their parents spoke both English and Russian. In the case of the bilingual parents, English was acquired as a second language when the family arrived in the US.
The bilinguals’ education history indicated that Russian was the primary language of elementary education for the participants because the mean age of arrival in the US was 9.2 years. However, during high school and current college education, Russian was replaced with English as the primary language. The bilinguals were also asked about the language that they speak and are spoken to in more frequently in different sociolinguistic contexts throughout their lives. These data are represented in Table 2. The frequency of language use was indicated on a five-point scale, with 1 representing “Russian all of the time”, 3 “Russian as much as English” and 5 “English all of the time”.
Table 2. Mean self-rated frequency of language used, in different time periods and contexts; 1 = Russian all of the time, 3 = Russian as much as English, 5 = English all of the time.
family context was defined as grandparents, parents, siblings, spouses and children, private contexts as other relatives, acquaintances and friends, and public contexts as teachers, peers at school and at work, and other people in public settings. The data are divided into input, e.g., the language the participants were addressed in vs. output, the language they used to address other people (Fernández, Reference Fernández2003). In general, inspection of Table 2 reveals a typical pattern of language switch from the heritage Russian to the dominant English language found in the educational setting: Russian was the language more frequently used during childhood and adolescence, as well as the more frequently used language in family and private contexts, whereas English was the language more frequently used in adulthood and in public contexts. The summary of the use of the two languages demonstrated the same switch from Russian into English, with the mean rating of 2.21 throughout life (almost balanced use of English and Russian), with the strong advantage of English, 3.55, at the time of the study.
Table 3 shows the data from the final group of questions in the questionnaire that addressed the issue of self-reported language dominance. When asked to rate their proficiency in both languages on a five-point scale ranging from 1 = “very good” to 5 = “very poor”, the participants’ self-reported ratings as a group were consistently higher for English (minus indicates higher ratings for English) in reading (M = −1.13) and writing (M = −1.22) but not in speaking (M = −0.38) and oral comprehension (M = −.34). Thus, the participants were advanced bilinguals as they viewed their spoken language comprehension in Russian to be as good as in English, which was critical for the purposes of our experiment.
Table 3. Difference scores in self-rated proficiency in four areas, spoken comprehension, speaking, reading and writing; 1 = very good, 5 = very poor (minus indicates higher proficiency in English, zero indicates balanced, and plus indicates higher proficiency in Russian).
Thus, with the average age of language switch of 9.2 years, the participants were typical heritage Russian speakers, with English as their dominant language. According to the informal classification proposed by Polinsky and colleagues (Kagan & Dillon, Reference Kagan and Dillon2006; Polinsky, Reference Polinsky, Antonenko, Bailyn and Bethin2008), they were advanced HL Russian speakers with restricted vocabulary and limited deficiencies in grammar but with native-like proficiency in oral comprehension. We divided them into two groups based on their average differences in scores for self-rated proficiency in English minus Russian. We based the distinction on the averages in the first two columns in Table 3 by drawing the line at the point when differences in both speaking and comprehension between the two languages were below zero (the English-dominant group) or zero or above (the balanced group). This criterion loosely separated fourteen participants whose literacy was better in English than in Russian (M = −1.91) from eighteen participants who were biliterate (M = .13).
Design, materials, and procedure
Design, materials, equipment and procedure were the same as in Experiment 1.
Data treatment and analysis
The bilingual participants took substantially longer to perform the task of moving the Target object in comparison to monolinguals with the action completed by 2500 ms in only 79% of the trials compared to 94% of the trials in the monolingual group. Reaction time data were not collected due to the fact that the experiment was dependent on gross and fine motor skills of moving a hand and picking up an object. The dropped trials with track loss constituted 6.25% of the bilingual eye-movement data.
Results
Eye-movements of all the participants
Figure 6 plots the proportion of time HL Russian listeners spent looking at the Target (e.g., the red star) vs. the Color Competitor (e.g., the red bird) during the referentially ambiguous adjective–verb portion of the utterance, i.e., Krasnuju položite . . . “Red put . . .”. (To facilitate the comparison of the monolingual (Figure 4A) and bilingual data (Figure 6), the same scale for proportion of time, 0.8, is used in the y-axis.) In stark contrast to the monolingual data (cf. Figure 4A), bilingual listeners showed no preference to interpret the adjective “red” as describing the Target, with neither of the two factors, Visual Context or Prosody, affecting the proportion of time spent looking (all Fs < 1). In fact, they almost did not look at either the Target (7.4%) or the Color Competitor (7.8%) at all during this referentially ambiguous portion of the utterance because they preferred to hold gaze on the central fixation point, the cross (87%), throughout the first 1000 ms (33 frames) of the utterance.
Figure 6. Bilingual HL participants. (A) Mean proportion of time spent looking at the Target or the Color Competitor while hearing the ambiguous regions Adjective–Verb Krasnuju položite . . . “Red put . . .” but prior to hearing the noun zvezdočku “star”. (Error bars indicate one standard error from mean.)
This ‘wait-and-see’ behavior continued not only through the referentially ambiguous regions, adjective and verb, but persisted well into the post-noun region (PN1 and PN2) when the acoustic information about the Target was completely revealed. For this reason, it made no sense to examine the ambiguous Regions 1 and 2, adjective and verb, any further, the way it was done for the monolingual data. The floor effect found for the mean proportion of time spent looking at the Target or the Color Competitor in the beginning of the utterance (Figure 6) also rendered the further analysis of the probability of the first look to the Target averaged across all the performed for the monolingual listeners (Figure 4B) meaningless for the bilingual group.
However, despite the initial insensitivity to contrastive cues, HL Russian listeners processed the sentences correctly but their processing was remarkably slow. The time course of computing contrastiveness is shown in Figure 7 (parallel to Figure 5 for the monolingual data) for each of the four conditions (e.g., 1- and 2-Contrast for Early and Late Accent) separately.
Figure 7. Bilingual HL participants. The probability of looking at the Target vs. the Color Competitor over time. (A) 1-Contrast context, Early Accent. (B) 1-Contrast context, Late Accent. (C) 2-Contrast context, Early Accent. (D) 2-Contrast context, Late Accent. (Time from onset of Adjective in 1/30 sec frames.)
Sigmoidal curves for the proportion of fixations in Figures 5 and 7 illustrate readily observable characteristics of a four-parameter logistic function, such as the slope, baseline, crossover and peak. For our purposes, the most important one was the slope: the looks to the Target and Competitor were significantly delayed and rose much slower for the HL than for the monolingual listeners. This was the most noticeable difference between the two groups: monolinguals computed contrastiveness and resolved the referential ambiguity using multiple sources of information substantially faster. They launched anticipatory eye-movements to the Target early on in the utterance, during Regions 2 and 3, the verb and the noun: at this point in processing, the looks to the Target started to reliably diverge from the looks to the Color Competitor. Even more impressive is the finding that the ambiguity was not only rapidly detected but was also successfully resolved before the noun was processed. In contrast, the HL bilinguals showed no signs of noticing the referential ambiguity: they had to hear the noun in its entirety before they could identify the Target. Statistical analyses for Regions 1, 2 and 3 showed no differences between the looks to the Target and the Color Competitor (Fs < 1) but revealed a gradual increase in looks to both red objects and a drop in looks to the cross.
The Target preference in the HL listeners finally emerged in the post-noun (locative PP) region that spans frames 50–70 (F1[1,31] = 107.28, p < .001) but there were no significant effects of either the Visual Context (34.9% in the 1-Contrast vs. 34.6% in the 2-Contrast condition) or Prosody manipulation for this region taken as a whole. The Target preference in this relatively long region was driven by the overwhelming preference to fixate the Target object just as the participants were preparing to move it. Thus, we conducted a separate analysis on the first half of this post-noun region by dividing it into two 5-frame (165 ms) windows, Post-N1 (PN1) and Post-N2 (PN2), indicated by the dashed line in Figure 7. ANOVAs confirmed the importance of analyzing these windows separately: there were no significant effects or interactions during the PN1, with only a trend in preference for the Target over the Color Competitor (F1[1,31] = 3.98, p = .0548). All the processing took place in the second half of the post-Noun region (PN2). Here, 1-Contrast context (Figure 7A–B) showed a preference for the Target (47.5%) while 2-Contrast contexts (Figure 7C–D) did not (38.3%) (significant in the subject-based-analysis only, F1[1,31] = 4.34, p < .05; F2[1,19] = 2.68, p = .098). Finally, as shown by a significant Visual Context × Prosody interaction (F1[1,31] = 6.32, p < .05; F2[1,19] = 6.03, p < .05), the preference to look at the Target was enhanced by Prosody but, unexpectedly, in the 1-Contrast, late accent condition: the HL bilingual listeners ignored the contrastive pitch accent on the adjective but seemed to benefit from it on the noun. This is perhaps because the late pitch accent on the noun could be utilized immediately to confirm the identity of the Target object already singled out by the 1-Context visual context.
In summary, we found that HL Russian bilinguals, in contrast to monolingual native listeners, did not compute contrast sets as soon as the relevant linguistic information became available, and even the visual context was not enough to push them to do so. Instead they had to hear the entire head noun before picking out the Target, effectively resorting to a ‘wait-and-see’ strategy. They were also not sensitive to prosody manipulation: pitch accent had only a small facilitatory effect in the identification of the Target and only when it was on the noun, and not on the adjective in the split-constituent construction.
Eye-movements of English-dominant vs. balanced HL Russian listeners
In the analyses described above we treated the HL listeners as a homogeneous group, and it is possible that the lack of early effects of visual context and especially prosody was driven by individual variation in the Russian proficiency of the participants. Our HL bilingual listeners' language abilities were clearly continuous (see Table 3), and could be treated as a dimension along which individual differences exist. Thus, with respect to heritage Russian, this prompted us to divide the sample into two qualitatively distinct groups – fourteen English-dominant and eighteen balanced participants – based on the difference in scores for self-rated proficiency in speaking, comprehending, reading and writing in English and Russian. Could it be the case that the weaker of the two groups, i.e., English-dominant participants, washed out possible effects?
To confirm that there was individual variation in Russian proficiency, we first conducted a correlational analysis to look for a relationship between the participants’ age at the time of arrival in the US (equivalent to the years of learning Russian) and the difference in self-reported scores in proficiency in two languages averaged across the four domains (see Table 3). As the scatter plot in Figure 8 demonstrates, there was a moderate but significant positive correlation between years of learning Russian and balanced bilingual status (r(30) = .378, p < .05), indicating that the later the participants switched from Russian to English, the more proficient they were in their heritage Russian. No such relationship was found between the difference in scores and years spent learning English (r(30) = .004).
Figure 8. Correlation between mean difference in scores (Russian minus English) and years spent learning Russian.
We performed statistical analyses parallel to the ones reported for the HL speakers as a group to compare the coarse- and fine-grained measures in computing contrastiveness for balanced and English-dominant groups separately selecting the same post-Noun region (frames 50–70). We excluded the four Russian-dominant participants from the balanced group, arriving at the even number of fourteen participants per group, which made it possible to conduct a balanced factorial ANOVA of eye-movements using the language status as a new between-subjects factor.
First, we examined the latency of the first fixation being to the Target, comparable to the analysis of the monolingual data (cf. Figure 4B) but without restricting our attention to the referentially ambiguous adjective–verb portion of the utterance (recall that the lack of fixations to the red objects in general during this region rendered it uninformative for the HL speakers as a group.) If the balanced HL participants were better in Russian spoken-word recognition and lexical retrieval than the English-dominant ones, their latencies of the first fixation to the Target should be faster (although still slower than those of monolinguals). The analysis was based on 82% of the trials which contained at least one fixation to the Target within the average trial duration set at 70 frames (2333 ms); beyond this point, the fixations to the Target reached the asymptote and the fixations to the Color Competitor floored (see Figure 7). There was no significant difference in the latency of first fixating the Target between the two groups – 1700 ms for the balanced and 1759 ms for the English-dominant HL speakers. Neither were there any other effects (of the visual context or prosody) or interactions. Thus, both groups were equally slow in identifying the red star as the Target and launching their first eye-movement to it, and they were not differentially affected by the two factors of interest. By the same token, no difference was found between the two groups in the latency of the first fixation to the Color Competitor (1470 ms for the balanced and 1491 ms for the English-dominant participants). Although it appears that the HL speakers as a group were on average 235 ms faster in locating and fixating the Color Competitor compared to the Target, these numbers are not informative because only half of the trials even contained a fixation to the Target within the set trial duration.
Finally, we investigated the fine-grained time course of computing contrastiveness separately for each of the two groups (in parallel to the monolingual in Figure 5 and the bilingual data taken as a whole in Figure 7) for each of the four conditions (e.g., 1- and 2-Contrast for Early and Late Accent) in the post-noun region. In order to keep the visual representation manageable, we chose to present the difference in fixations (a single line for the Target minus the Color Competitor) instead of the two separate lines (the solid circles for the Target and the hollow circles for the Color Competitor in Figures 5 and 7). This allowed us to combine the time courses of fixations for the balanced and English-dominant groups on one panel per condition (note that the x-axis is aligned with the beginning of the verb region at Frame 30 instead of the adjective, to visually maximize the differences).
In Figure 9, the x-axis represents the line where the fixations to the Target and the Color competitor diverge. When either of the two lines (the solid squares for the balanced group and the hollow triangles for the English-dominant group) is above the x-axis, it means that there were more fixations to the Target; when either is below, there were more fixations to the Color Competitor. Divergence of the two lines is indicative of different eye-movement patterns between the two groups in identifying the Target and the Color Competitor.
Figure 9. Balanced vs. English-dominant participants. The probability of looking at the Target vs. the Color Competitor over time. (A) 1-Contrast context, Early Accent. (B) 1-Contrast context, Late Accent. (C) 2-Contrast context, Early Accent. (D) 2-Contrast context, Late Accent. (Time from 30 ms into the trial in 1/30 sec frames.)
Visual inspection of the data reveals no differences in eye-movement patterns between the balanced and the English-dominant group in the Late Accent conditions (Panels B and D). While it appears that the Early Accent is helpful for the balanced group in both visual contexts (Panels A and C), the statistical analysis showed that the main effect of the Language Dominance found for the differences in looks for frames 50–70 (Target minus Competitor) (F1[1,26] = 4.28, p = .0487) (Figure 9C) is driven by a strong preference of the English-dominant group for the Color Competitor in the 2-Contrast, Early Accent condition, a preference that dissipated toward the end of the 50–70 frame region. Thus, the English-dominant group seemed to get confused by the early contrastive pitch accent on the adjective while the balanced group somewhat benefited from it in both visual contexts.
Finally, the two main effects, of Visual Context and Prosody, were also significant (no interactions) showing both groups’ late sensitivity to the contrastive cues. Both groups were faster in identifying the Target in the 1-Contrast condition than in the 2-Contrast one (F1[1,26] = 9.10, p < .01) when the visual context was supportive of the single contrast set for color, but it took them longer to do the same in the ambiguous visual context. The role of Prosody remains complicated, because although it was significant (F1[1,26] = 5.51, p < .05) in the separate group analysis, again, as in the HL data analysis as whole, it facilitated identification of the Target only when the contrastive pitch accent was on the noun.
In summary, separate analysis of coarse- and fine-grained eye-movements for the balanced and English-dominant group did not confirm a hypothesis that slowed computation of contrast sets and late sensitivity to visual context, word order and prosody by the HL listeners as a whole was due to the language dominance factor, i.e., that English-dominant participants were responsible for the slowed processing of contrastiveness observed for the HL group as a whole.
General discussion
Our purpose in this article was to experimentally investigate the online processing of contrastiveness in Russian in two populations, monolingual and heritage Russian−English bilingual listeners. Our findings reveal that the HL listeners were not only substantially slower than the monolingual controls but also used the multiple sources of information critical for processing of contrastiveness in Russian differently. This confirms that, indeed, heritage language comprehension is less efficient than monolingual comprehension, and it is characteristic of not only English-dominant HL speakers but also of highly proficient, balanced HL ones as well. Our results are in parallel to the slowdown (33 ms) in production reported by Ivanova and Costa (Reference Ivanova and Costa2008) for bilingual Spanish–Catalan speakers in picture naming in their first and dominant language, Spanish (as well as in their weaker language, Catalan). We speculate that Russian HL speakers’ late and only partial integration of multiple sources of information in computing contrastiveness could be caused by a slowdown that originates at the level of spoken word recognition and then culminates at the interfaces, as was proposed in recent L2 research (Rothman, Reference Rothman2009).
The processing of contrastiveness by monolingual Russian listeners
Monolingual listeners showed early and robust ability to notice and compute contrast sets encoded in the visual context, as revealed by both coarse- and fine-grained eye-movement analyses. During the referentially ambiguous adjective–verb portion of the utterance they were already able to identify the Target referent when the 1-Contrast visual context supported the contrast set for color. The computation of contrast sets occurred early in the utterance, before the noun itself was heard. This timing was especially striking; it was made possible because of the Russian split-constituent word order that allowed us to demonstrate that certain cues to contrastiveness can have their effect much earlier than has been possible to test in other languages. In Russian, contrastive cues became available one second before the noun, while the effect in English (Ito & Speer, Reference Ito and Speer2008; Sedivy, Reference Sedivy2003; Sedivy et al., Reference Sedivy, Tanenhaus, Chambers and Carlson1999) and German (Weber et al., Reference Weber, Braun and Crocker2006) occurred during the first 200–400 ms after the onset of the noun. Thus, the referential implications of a split-constituent construction appear to be computed by monolingual Russian listeners ‘on the fly’, allowing them to anticipate a plausible referent in the contrastively supportive visual context without yet having heard the lexical head.
However, in contrast to the English and German studies in which resolution of referential ambiguity was facilitated by contrastive pitch accent on the adjective, and contrary to our expectations, the contrastive pitch accent placed early, on the adjective, had no facilitatory effect in the identification of the Target referent in Russian. One possible interpretation of this early lack of effect of prosody could be that prosodic cues to contrastiveness by themselves are not strong enough because prosody as a cue to structure and meaning is inherently ambiguous (e.g., Cutler et al., Reference Cutler, Dahan and van Donselaar1997; Snedeker & Trueswell, Reference Snedeker and Trueswell2003). It reflects multiple factors, including constraints on lexical stress, phrasal boundaries, constituent boundaries and reference. In comparison, contrastiveness is first and foremost driven by discourse demands which may or may not be encoded by grammatical means.
It is our belief, though, that a different explanation for the lack of anticipatory prosody effect is more plausible. For pitch accent to generate a strong contrastive interpretation, it may be critical that a contrast set is established overtly, by mentioning the Competitor; this should create direct discourse support for contrastiveness. Note that this is precisely what was done in the English and German eye-tracking experiments with contrastiveness in which the utterance with the Target referent was always second following a discourse-initial one-sentence preamble in which the Color Competitor was overtly mentioned (e.g., First, hang the green drum. Now, hang the BLUE drum in Ito & Speer's, 2008, study). In such a set-up, listeners' attention was deliberately directed toward the contrast set as salience of the Target referent was enhanced by the contrastive pitch accent it bears. In our experiment, however, there was no preamble instruction that overtly established the contrast set because the experimental instruction was always the first one, and the participants had to infer the contribution of pitch accent to the meaning of the experimental instruction while processing it. The critical manipulation to test this explanation for the lack of effect of prosody would be to add preambles to our experimental materials that are either mention the Color Competitor (discourse-mention) or do not (discourse-new); indeed, we are currently in the process of conducting such an experiment in our laboratory.
The processing of contrastiveness by bilingual heritage Russian listeners
A very different picture emerges for the comprehension of contrastiveness by HL Russian bilingual listeners. The most striking characteristic of this group's performance in comparison to that of the monolinguals was their overall slowness, both in performing the task and even in eye-movements, in line with the idea that bilinguals in general are slower in verbal tasks (Bialystok, Reference Bialystok2009). The HL listeners did not attempt to identify the Target referent as soon as the color adjective “red” became available in the spoken input; in fact, they did not even look at either of the two red objects at all during the referentially ambiguous adjective–verb portion of the utterance. They seemed to adopt the strategy of ‘wait-and-see’ and showed the first signs of taking the visual context into consideration only after the lexical head (e.g., the noun) was processed, on average 800 ms later than the monolingual listeners did. Not only was the effect of prosody delayed, but it also worked in reverse: the contrastive pitch accent facilitated the identification of the Target referent only when the noun was stressed and only after the participants had heard it. Thus, although the HL listeners eventually took notice of cues critical for the computing of contrastiveness, it happened only when the lexical identity of the noun made the task superfluous.
What are some possible explanations for both quantitative (slowdown) and, to a certain degree, qualitative (late and partial sensitivity to multiple sources of information) differences in how the monolingual and HL bilingual speakers process contrastiveness in Russian? We suggest that late and partial integration of visual context and prosody is a consequence of the quantitative differences: HL speakers start slow and, thus, are less efficient in spoken language comprehension of their heritage language. This finding fits well with overall disadvantage for all categories of bilinguals in such verbal fluency tasks as language production (Pontocarrero, Burright & Donovick, Reference Pontocarrero, Burright and Donovick2007), lexical retrieval (Bialystok, Reference Bialystok2009), and the reading of syntactically ambiguous sentences (Frenck-Mestre, Reference Frenck-Mestre, Heredia and Altarriba2002). Frenck-Mestre proposes to explain the slower reading times of bilinguals by their tendency to re-read, thus pointing to reanalysis as the probable cause. Bialystok suggests that the bilinguals’ slowness is caused by difficulties related to attentional control: even when bilinguals have to perform in one language, the second one is activated in parallel, and this creates covert competition between the two languages. Note that this disadvantage was found in our data even though there are significant methodological differences between these verbal fluency tasks and our experiment. The former are complex, and require metalinguistic skills and literacy, whereas our spoken language comprehension task was based on six-word instructions with simple and basic vocabulary items, and was not challenging at all.
In addition, separate analyses of the balanced and English-dominant groups revealed that the overall slowdown in the HL processing was not brought about by the difference in Russian proficiency among our HL participants. The balanced HL speakers were not any faster or more efficient than the English-dominant ones, although they did take advantage of the visual context earlier and were not confused by the early contrastive pitch accent on the adjective the way less proficient HL participants were. Their self-reported proficiency in Russian was correlated with the age of arrival to the US, suggesting that they are not the only language background variables that shape the HL processing system, but a more detailed exploration of the contribution of sociolinguistic variables is left for future research.
One alternative hypothesis is that the slowdown in the HL processing system is a reflection of a ‘wait-and-see’ strategy that HL speakers consciously follow when confronted with the heritage language when they do not want to commit themselves to an interpretation until they are fairly confident of it. This strategy could stem from HL speakers’ self-imposed perception of having ‘inferior’ knowledge of the heritage language: they may be risk-averse because they are uncertain of their ability to process the heritage language correctly. Although not implausible, this explanation is unlikely for our experiment for two reasons. First, we specifically asked our participants to perform the task as fast as possible, and they reported it as easy. Second, eye-movements are completely automatic and cognitively ‘cheap’; not looking at the target mentioned in the spoken input and present in plain view is much more cognitively demanding than looking at it, as studies that employ the anti-saccade task demonstrate (Bialystok, Craik & Ryan, Reference Bialystok, Craik and Ryan2006).
The second alternative hypothesis is, as was suggested by one of the anonymous reviewers of this article, that the HL participants were less familiar with the split-constituent construction in Russian and this lack of familiarity could have slowed them down in acting upon instructions with split constituents. While the controlled testing of this idea should be included in the future investigation of grammar and processing in bilingual Russian HL speakers, the anecdotal evidence from the first author's interactions with the participants in the laboratory suggests that the split-constituent construction could be elicited in their spontaneous production.
We suggest that this slowdown is not conscious but is a reflection of covert competition between the two languages of the HL speakers that starts at the level of spoken word recognition. McMurray and colleagues (McMurray, Samelson, Lee & Tomlin, Reference McMurray, Samelson, Lee and Tomlin2010) have recently found that the impaired language abilities of specific language impairment (SLI) adolescents caused decreased activation of the target and lingering interference from the competitor in the VWP study that used the Cohort effect (Allopenna, Magnuson & Tanenhaus, Reference Allopenna, Magnuson and Tanenhaus1998). Both effects occurred quite late in the time course, a finding not attested in previous eye-tracking studies that usually demonstrate rapid spoken-word recognition. The authors proposed that these changes in word recognition were caused by lexical decay in SLI listeners and hypothesize that “[v]ariation in how well the system settles on a single candidate will directly affect syntactic and semantic processes involved in sentence comprehension” (p. 32). The idea that word recognition in bilingual spoken-word recognition is affected by competition from both vocabularies has been investigated in several eye-tracking studies and remains robust despite the facts that experiments were conducted in the monolingual mode, the Target–Competitor phonetic overlap was only partial (Weber & Cutler, Reference Weber and Cutler2004), and the languages were typologically different (Marian & Spivey, Reference Marian and Spivey2003). It is modulated by the lexical status of the words in Target–Competitor pairs, the language proficiency of the participants (Blumenfeld & Marian, Reference Blumenfeld and Marian2007), with the dominant language exerting earlier and shorter lasting co-activation than the weaker language, and by the sociolinguistic circumstances surrounding the division of labor between the two languages. Weber and Cutler concluded that “[n]on-native listeners may be doomed to recognize spoken language less rapidly than native listeners” (2004, p. 18).
Setting aside methodological differences (i.e., different linguistic phenomena, tasks, measures, languages, participants) between these studies and our experiment, we propose to extend the hypothesized covert competition between the two languages of L2 bilinguals to HL bilinguals to explain the slowdown found in the HL processing of contrastiveness. The competition starts early, at the level of spoken-word recognition (McMurray et al., Reference McMurray, Samelson, Lee and Tomlin2010), continues to accumulate at the morphosyntactic level (cf. Bernolet, Hartsuiker & Pickering, Reference Bernolet, Hartsuiker and Pickering2007), and culminates at the interfaces and the lexical−semantic levels. Activation of English, the dominant language, despite the fact that our HL participants were performing a task in Russian, would result in cognitively taxing interference from English into Russian. Testing this hypothesis requires a series of rigorously controlled experiments that will systematically investigate every domain of the HL language processing system starting with spoken-word recognition and progressing through morphology and syntax to semantics to pinpoint the processing bottlenecks that cumulatively slow down bilingual L2 and HL language processing. This should be a step-by-step investigation of the contribution of grammatical features (gender, number and case) and syntactic structures (word orders, anaphoric relations and multiple dependencies). Finding interference from the dominant language into the heritage language (or from L2 into L1) in particular language domains of the bilingual language processing system will constitute the missing evidence for the cascading effects of competition responsible for slowing down bilingual in general and HL processing in particular. Ultimately, the entire HL processing system slows down and, with time, interference from the covert competition between the two languages becomes a major contributing force to heritage language attrition whose first signs, as in our experiment, can be so vividly captured by HL bilinguals’ eye-movements.
