Blend representations

Many psycholinguistic theories are framed within a spreading-activation or connectionist perspective (Rumelhart, Hinton & McClelland, Reference Rumelhart, Hinton, McClelland, Rumelhart and McClelland1986; see Goldrick, Reference Goldrick and Faust2012, for a recent review). In such theories, mental representations are graded, distributed patterns of activation, a numerical quantity associated with simple processing units. This allows for blends: representational states in which multiple representational elements occupy (to varying degrees) a single position within a linguistic structure.

For example, suppose a native Spanish speaker is producing a sentence in English: “Yesterday I went to the park to walk my dog.” While planning this utterance – in particular, while retrieving the appropriate final noun from memory – many psycholinguistic theories of bilingualism assume that the speaker's production system enters the state shown in Figure 1 (see, e.g., Kroll & Gollan, Reference Kroll, Gollan, Goldrick, Ferreira and Miozzo2014, for a review of such proposals). In this network, there are three types of representational units. The input to the system consists of semantic features along with a representation of the intended language of response. Activation spreads from these units to a set of units corresponding to lexical items (e.g., ‘lemmas’). Figure 1A shows the flow of activation through the connectionist network; Figure 1B provides an alternative view of the distribution of activation over the lexical items.

Figure 1. A. Depiction of psycholinguistic processing model during production of DOG. Thickness of circle denotes relative activation of unit. B. Alternative depiction of the state of this system, focusing on gradient activation at the lexical level.

In such a representation, the intention to produce a single lexical item (a single noun in the phrase ‘my ___’) results in the simultaneous co-activation of multiple mental representations. Lexical selection processes simultaneously consider the target (DOG), semantically related words within the same language (CAT), and non-target language words (PERRO). This can be seen in Figure 1A+B, where multiple representational elements have varying non-zero activation. The state of processing a single word is thus a blend of multiple linguistic representations. This representational hypothesis is often referred to as co-activation or parallel activation. We use the term blend to emphasize that the multiple elements are not simply simultaneously activated; they are co-present within a single position in the linguistic representation (e.g., head of a particular noun phrase; this is highlighted in the depiction in Figure 1B).

Empirical evidence for blend representations

Activation based representations do not require blends (one could assign 0 to all non-target representations in Figure 1, activating only DOG). It is also not immediately clear what functional motivation would require blend states. In a purely English utterance, why should one consider Spanish words? This makes it all the more striking that a substantial body of evidence supports such blend representations. Kroll and Gollan (Reference Kroll, Gollan, Goldrick, Ferreira and Miozzo2014) provide an extensive review of evidence from multilingual speakers (see Melinger, Branigan & Pickering, Reference Melinger, Branigan and Pickering2014, for a review of evidence from monolinguals). Here, we emphasize a few key recent studies that provide evidence of such representations during production of phrases and sentences.

A key prediction of blend representations is that the spread of activation will lead to the partial activation of non-target representations at other levels of processing. Following the example above, when producing target DOG, the partial activation of the lexical representation PERRO is predicted to lead to partial activation of representations of the /p/ sound. In contrast, when producing CAT, the PERRO representation should be less active, resulting in less activation of the /p/ sound. Consistent with this prediction, many studies have demonstrated that production is facilitated when there is a phonological relationship between the target utterance and non-target translation equivalents.

Spalek, Hoshino, Wu, Damian, and Thierry (Reference Spalek, Hoshino, Wu, Damian and Thierry2014) examined German–English bilinguals producing adjective-noun phrases. For both behavioral and electrophysiological measures, they found that second-language English production was facilitated when the English adjective shared phonological structure with the noun's German translation equivalent (e.g., “blue flower”: blue shares the initial sounds of the German translation equivalent Blume; contrast with “green skirt:” green shares no sounds with the German translation equivalent Rock). While significant, effects of non-dominant L2 English on production of dominant L1 German phrases were smaller, limited to electrophysiological measures. These results are consistent with the presence of blend representations, but suggest that the degree to which non-target representations are present in blends is modulated by the relative strength of each language (such that Blume is more active during processing of flower than vice versa).

Another set of studies arguing for co-activation of multiple representations has compared the production of targets that share phonological structure with their translation equivalent (e.g., English ANCHOR – Dutch ANKER) to those with no overlap (e.g., BOTTLE – FLES). The former are often referred to as ‘cognates’ in the psycholinguistic literature; however, no historical connection between the translation equivalents is required. The logic is that simultaneous activation of lexical representations in the two languages should facilitate processing of any shared phonological structure, producing a cognate facilitation effect (Costa, Caramazza & Sebastián-Gallés, Reference Costa, Caramazza and Sebastián-Gallés2000). For example, simultaneously activating ANCHOR and ANKER will serve to facilitate retrieval/planning of shared segments /ŋ/, /k/, /ə˞/. Starreveld, De Groot, Rossmark, and Van Hell (Reference Starreveld, De Groot, Rossmark and Van Hell2014) recently documented cognate facilitation during sentence planning. Dutch–English bilinguals read aloud sentences with an embedded picture (e.g., a picture of an anchor appeared in the position of the blank in the sentence “In the middle of the square was an ____ with a thick chain attached to it.”). When producing picture names in L2 English, participants showed cognate facilitation. Following the study reviewed above (Spalek et al., Reference Spalek, Hoshino, Wu, Damian and Thierry2014) as well as many other results, these effects were much stronger in L2 than L1 production. Furthermore, cognate effects were modulated by the sentence context. When the sentence placed greater constraints on the word that could fit in the space occupied by the picture (e.g., “Popeye the sailorman has a tattoo of an ____ on his arm.”), cognate effects were diminished. As with the preceding study, these results suggest that while blends are part of language production, the degree to which non-target representations are present is modulated not only by the relative strength of each language, but also by the degree to which context supports the retrieval of a specific target word.

Finally, some of the strongest evidence for blend representations has come from studies that have documented the literal co-production of multiple representations. The simultaneous co-presence of multiple linguistic representations during planning leads to the simultaneous production of actions associated with these representations. Pyers and Emmorey (Reference Pyers and Emmorey2008) examined the oral and manual productions of bimodal bilinguals: native speakers of a spoken language (English) and a manual language (American Sign Language; ASL). During conversations with non-signers – where the bimodal bilinguals intend to speak a single (oral) language – they simultaneously produced ASL and English grammatical markers. At rates much higher than non-signers (but lower than in their ASL productions), the bimodal bilinguals furrowed their brows while producing wh-questions (e.g., “How many siblings does she have?”). This occurred in spite of the fact that spoken English explicitly marks wh-questions (making double-marking unnecessary to express the intended message). Note that this gesture is pragmatically dispreferred in spoken English, where it conveys negative affect. Pyers and Emmorey argued that this modulated the rate of co-productions, as co-productions were much higher for conditionals (e.g., “If it rains, class will be canceled”; associated with raised brows). This provides further evidence for constraints on the degree of activation of non-target representations in blends.

In the bimodal bilingual case, the two languages are not competing for expression on the same communication channel. More subtle co-productions can be found during production of two oral languages. While co-activation enhances retrieval of shared phonological structure, the heightened activation of non-target language representations should increase cross-language phonetic interference – the intrusion of non-target language phonetic properties into bilingual productions. For example, while Spanish and English share a common set of voicing contrasts in initial stops (e.g., /b/ vs. /p/), the phonetic realization of this contrast is distinct in each language (pre-voiced vs. short-lag voice onset time in Spanish; short vs. long-lag in English). This conflict leads non-native speakers to produce these sounds with phonetic properties intermediate between the two languages (Flege, Reference Flege1991). Amengual (Reference Amengual2012) showed that this cross-language phonetic interference is enhanced for cognates. When reading sentences aloud, Spanish–English bilinguals produced initial stops in Spanish with more English-like properties in cognates vs. non-cognates. No such difference was found in the productions of Spanish–Catalan bilinguals (where the two languages have similar phonetic realizations of this contrast). Other results suggest that these cognate effects are not simply word-specific phonetic patterns in bilingual speech, but rather reflect dynamic properties of bilingual production. Olson (Reference Olson2013) and Goldrick, Runnqvist, and Costa (Reference Goldrick, Runnqvist and Costa2014) found that phonetic interference was increased when participants were required to unexpectedly code-switch during picture naming (vs. trials where participants did not switch languages). For voiceless stops, Goldrick et al. found that this context-specific phonetic interference effect is enhanced during production of single cognate vs. non-cognate words – suggesting that the cognate effect reflects the context-specific activation of target and non-target language representations.

Integration of grammatical principles in code mixing

Given the evidence for co-activation of the two languages in contexts where speakers intend to produce only one language, it is unsurprising that co-activation is a fundamental property of code mixing. Critically, speakers are not only uttering lexical items from both languages but are also integrating grammatical principles from each linguistic system.

The integration can be seen in cross-linguistic syntactic priming, where exposure to a structure in one language increases the probability that speakers will use a similar structure in another language (see Pickering & Ferreira, Reference Pickering and Ferreira2008, for a review). For example, Hartsuiker, Pickering, and Veltkamp (Reference Hartsuiker, Pickering and Veltkamp2004) found that when Spanish–English bilinguals heard a passive construction in Spanish, it increased the likelihood that they would produce a passive vs. active construction in English on a subsequent trial. Such priming does not only alter the probability of attested structures. In certain contexts, it can allow for the transfer of grammatical patterns from one language to another, reflecting the integration of knowledge of each language. For example, in many contexts Spanish does not allow for the word order adjective-noun, the typical word order pattern observed in English. Hsin, Legendre, and Omaki (Reference Hsin, Legendre, Omaki, Baiz, Goldman and Hawkes2013) found that in Spanish–English bilingual children priming could allow for transfer of this word order from English to Spanish.

Such integration also occurs in the context of intra-sentential code mixing. For example, Kootstra, van Hell, and Dijkstra (Reference Kootstra, van Hell and Dijkstra2010) elicited code mixed utterances from Dutch–English bilinguals. Participants described pictures by completing a Dutch sentence fragment that biased speakers to produce one of several word orders possible in Dutch (Subject-Verb-Object [SVO], SOV, or VSO). When cued to produce a mixed structure (i.e., using at least one English word to complete the Dutch fragment), participants preferred to use the word order common to both grammars (SVO). A similar preference for congruent grammatical patterns has been found in spontaneous mixing corpora (for reviews and discussion, see Deuchar, Reference Deuchar2005; Muysken, Reference Muysken, Milroy and Muysken1995; Myers-Scotton & Jake, Reference Myers-Scotton and Jake1995; Poplack, Reference Poplack1980).

Blends and co-production in code mixing

When speakers intend to mix lexical items and grammatical principles from two languages, we also observe blends. Some of the most dramatic examples come from bimodal bilingual code mixing. For ASL–English bilinguals the predominant type of code mixing is code blending: co-production of oral and manual elements (Bishop, Reference Bishop2010; Emmorey, Borinstein, Thompson & Gollan, Reference Emmorey, Borinstein, Thompson and Gollan2008). These cross-modal productions are typically semantically equivalent and synchronized in time. In examples (1) and (2), the English gloss of the sign production is shown in italics beneath the point in the sentence where the sign roughly occurred; underlining indicates the speech that co-occurred with that sign.

1. (1) And there's the bird. (Emmorey et al., Reference Emmorey, Borinstein, Thompson and Gollan2008: 48)

   bird

2. (2) Now I recently went back. (Emmorey et al., Reference Emmorey, Borinstein, Thompson and Gollan2008: 48)

   now I recently go-to

In unimodal bilinguals there is subtle evidence of co-activation in articulation. Analyzing a spontaneous code mixing corpus, Balukas and Koops (in press) found that phonetic interference effects in Spanish–English bilinguals increase at points closer to code switches. This suggests that co-productions are not unique to bimodal bilinguals.

Blends without co-production: Doubling constructions

Blending representations from two different languages can also yield non-simultaneous articulations. Languages differ in word order, mapping elements to different positions in the surface string. This raises the possibility that the grammatical principles from each language – both active during code mixing – could both be satisfied without yielding simultaneous articulation. For example, a bilingual's L1 has the word order verb-object and L2 object-verb. The string verb(L1)-object-verb(L2) satisfies the word order constraints of both languages; the L1 verb precedes the object, while the L2 verb follows the object. Although such strings might violate structural constraints on linguistic representations, they would not suffer from articulatory incompatibilityFootnote ¹ .

Patterns of attested doubling constructions: A review

Although such constructions are commonly discussed, only a few detailed references are devoted exclusively to them (Chan, Reference Chan, Bullock and Toribio2009; Hicks, Reference Hicks2010, Reference Hicks and Ross2012; Muysken, Reference Muysken2000: 104–6). They manifest in a variety of constituents, although preference seems to be given to the doubling of functional elements (syntactic elements expressing grammatical relationships; e.g., complementizers, determiners, prepositions, and auxiliary verbs) over lexical item (e.g., nouns). The following examplesFootnote ² provide an overview of the range of doubled structures. The doubled elements are underlined in each example.

1. (3) Complementizers: English–Japanese (Azuma Reference Azuma and Clancy1993: 199)

   if it goes three rounds datta ra ne

   if it goes three rounds was if TAG

   ‘If it goes three rounds.’

Note that in (3), if is located in its canonical place in English (appearing at the start of the dependent clause), and the Japanese ra in its expected location (were the utterance fully Japanese, ra would appear at the end of the dependent clause). Examples (4)–(7) illustrate similar doubling for various other elements, respecting the contrasting word orders.

1. (4) Adpositions: English–Finnish (Poplack, Wheeler & Westwood, Reference Poplack, Wheeler and Westwood1989: 405)

   mutta se oli kidney-sta to aorta-an

   but it was kidney-from to aorta-to

   ‘But it was from the kidney to the aorta.’

2. (5) Adverbials: English–Tamil (Sankoff, Poplack & Vannianiarajan, Reference Sankoff, Poplack and Vannianiarajan1990: 92)

   According to the schedule

   paDi oNNutaan irukkaNum.

   according to one only be must

   ‘According to the schedule, there must be only one.’

3. (6) Coordinating: conjunctions Spanish–Aymara (Stolz, Reference Stolz1996: 146, citing Porterie-Guierrez, Reference Porterie-Gutierrez1988: 355)

   pero sorro -sti wali astuturi -tajna. . .

   but fox -COO very keen -3.SG.PRT.EVI

   ‘But the fox was very keen.’

4. (7) Verbs English–Tamil (Sankoff et al., Reference Sankoff, Poplack and Vannianiarajan1990: 93)

   they gave me a research grant koɖutaa

   they gave me a research grant gave.3.PL.PAST

   ‘They gave me a research grant.’

Multi-word chunks can be doubled, as shown in (8) (verb + adverb) and (9) (verb + complementizer).

1. (8) Verb + Adverb English–Japanese (Nishimura, Reference Nishimura and Vaid1986: 139)

   We bought about two pounds gurai kattekita no

   We bought about two pounds about bought TAG

   ‘We bought about two pounds.’

2. (9) Verb + Complementizer English–Korean (Chan, Reference Chan2008: 800)

   everybody think that nay-ka yenge-lul cal

   everybody think C I-NOM English-ACC well

   hanta-ko sayngkakhayyo

   do-C think

   ‘Everybody thinks that I’m a good English speaker.’

While these types of code mixing utterances have been consistently documented in corpora, they are clearly marked; in general, these structures are largely avoided. Poplack et al. (1989: 405) report that these blends are “exceedingly rare”, citing that they only found 2 in their entire corpus; Furukawa (Reference Furukawa2008) found 7 examples in 5 hours of sociolinguistic interview data. However, Nishumura (Reference Nishimura and Vaid1986) and Backus (Reference Backus1992) suggest that these blends occur in roughly 3–5% of their corpus materials. The rarity of such productions is unsurprising. As noted above, the integration of grammatical principles from both languages yields a preference for congruent grammatical patterns in code mixing. The examples above violate this principle; they involve doubling of elements that are subject to conflicting grammatical patterns (e.g., verb-object vs. object-verb word order). Furthermore, research on non-code mixed productions suggests there are strong limitations on blends; the degree of co-presence within a blend is highly limited. If doubling constructions reflect blend representations, we also expect them to be strongly dispreferred.

Surveying the reported instances of these constructions, Hicks (Reference Hicks2010) identifies several cross-linguistic generalizations. First, as noted above, doubled elements locally respect the word order of the source grammars. Second, the doubled elements are typically heads; syntactic elements that define the syntactic properties of the phrase to which they belong. The doubled heads share a non-doubled complement; the other syntactic elements that belong to the phrase. For example, a verb phrase can be composed of a verb (the head) and an object (the complement). In the doubling construction V_L1O_L1V_L2, doubled verbs share a non-doubled object complement (see also Furukawa, Reference Furukawa2008). Thus, strictly local doubling (e.g., V_L1O_L1O_L2V_L2) is typically not observed. Finally, while some languages exhibit doubling in monolingual contexts (discussed further below), Hicks notes that doubling of elements from the same language (e.g., analogous to (7), *gave grant gave, *gave gave grant) are not observed during code mixing.

In sum, while doubling is rare, it is consistently observed across various sources; a variety of elements participate in intrasentential code mixing blends. Doubling is not mere repetition of elements; its occurrence is constrained by grammatical principles; doubled heads share non-doubled complements; and equivalence for grammatical features. This suggests that doubling constructions are not an “ad hoc production strategy” (Sankoff et al., Reference Sankoff, Poplack and Vannianiarajan1990: 92), but are rather coherent, syntactic objects that are governed by grammatical principles.

Overview of the proposal

Our account is based around 3 general principles. We first provide an overview of these and then examine in some detail how they can be applied to the empirical patterns of doubling constructions.

Relationship to other formal approaches to code mixing

Generative theories of code mixing – such as the one we further develop here – can be divided into two types. One set specifies grammars specific to code mixing. Rules or constraints refer specifically to code mixed structures, explicitly stating preferences for distinct types of code mixing (Belazi, Rubin & Toribio, Reference Belazi, Rubin and Toribio1994; Bhatt, Reference Bhatt1997; Di Sciullo, Muysken & Singh, Reference Di Sciullo, Muysken and Singh1986; Joshi, Reference Joshi, Dowty, Karttunen and Zwicky1985; Legendre & Schindler, Reference Legendre and Schindler2010; Muysken, Reference Muysken2013; Myers-Scotton, Reference Myers-Scotton1993; Poplack, Reference Poplack1980; a.o.). A classic example is Poplack's (Reference Poplack1980: 586) Equivalence Constraint: “Code-switches will tend to occur at points in discourse where juxtaposition of Ll and L2 elements does not violate a syntactic rule of either language.” Here, the grammatical principle refers directly to code mixing, distinct from (but related to) syntactic patterns in non-code mixed contexts. Similarly, from an Optimality-Theoretic perspective, Muysken (Reference Muysken2013: 715) makes use of constraints such as “*CSL = Don't switch between separate languages, either in their lexicon or in their grammar.”

An alternative approach assumes the grammatical principles of two languages are integrated during code mixing, and that this integration yields the patterns (Chan, Reference Chan2003, Reference Chan2008, Reference Chan, Bullock and Toribio2009; Lohndal, Reference Lohndal2013; MacSwan, Reference MacSwan1999, Reference MacSwan2000; Mahootian, Reference Mahootian1993; Woolford, Reference Woolford1983; a.o.). Since Mahootian (Reference Mahootian1993), this position is commonly referred to as the “null theory” of code mixing, according to which monolingual and bilingual grammars should be subject to identical representational/grammatical constraints and psychological principles.

Our approach incorporates elements of both perspectives. Following the null theory perspective, we assume that the features of code mixing reflect general principles of syntactic knowledge and sentence processing. Blend representations are a general feature of grammatical knowledge and processing; the emergence of doubling constructions in bilinguals is a consequence of the principles underlying these grammars. However, in contrast to the strongest version of the null theory (e.g., MacSwan, Reference MacSwan1999), we assume that grammatical principles can refer to language membership (e.g., distinguishing the well-formedness of lexical items in English vs. Tamil based not on syntactic features but on the language from which the item originates). This is critical to understanding attested doubling patterns. In such constructions, the doubled elements have (nearly) equivalent grammatical features (e.g., they match in agreement features (tense, aspect, case) and share argument structure requirements), yet surface in positions appropriate to the element's source language. If the grammar does not make reference to the source language, there is no means of capturing this restriction.

Weighted constraint interaction in stochastic generative grammars

Our theory utilizes the Gradient Symbolic Computation formalism (GSC; Smolensky et al., Reference Smolensky, Goldrick and Mathis2014). GSC is a constraint-based approached to generative grammar, building on work in Optimality Theory (Legendre, Reference Legendre, Legendre, Grimshaw and Vikner2001; Legendre, Putnam, de Swart & Zaroukian, in press-a; Prince & Smolensky, Reference Prince and Smolensky1993/2004) and Harmonic Grammar (Legendre, Miyata & Smolensky, Reference Legendre, Miyata and Smolensky1990, Reference Smolensky, Smolensky and Legendre2006; Pater, Reference Pater2009). Like other generative grammars, GSC defines a function that maps input structures (e.g., logical forms) to output structures (e.g., syntactic structures). In GSC (and Harmonic Grammar), the grammar is defined via a set of weighted violable constraints that assign a numerical well-formedness value (harmony) to each of the candidate outputs for a given input. GSC grammars are stochastic, generating a probability distribution over output forms (reflecting the relative harmony of the candidates).

To build up our theory, we begin by modeling monolingual grammars. Consider a simple input consisting of a subject and verb; as shown in Figure 2, this can be linearized using at least two surface syntactic structures.

Figure 2. Two alternative surface syntactic structures corresponding to the input goes (John). The text below each provides a bracket notation corresponding to the tree, with the subscript on each open bracket denoting the category of the constituent.

These surface syntactic structures reflect the assumptions of X-bar theory (see Carnie Reference Carnie2010: Chapter 7, for a historical overview). Briefly, X-bar theory places restrictions on traditional phrase structure grammars. Abstracting away from more complex phenomena, basic X-bar theory assumes that the basic structure of an extended projection consists of two syntactic phrases: XP, consisting of a specifier and an Xʹ phrase; and Xʹ, consisting of the head element X⁰ and a complement. As illustrated here, the extended projection of the verb consists of a verb phrase (VP) with specifier John, and a Vʹ phrase consisting of the head goes (and a null complement). This simplified notation is sufficient for capturing the basic facts about syntactic constituency and allows us to specify the patterns in word order variation underlying the doubling examples we consider. We believe that the insights of this analysis would generalize to more recent representational frameworks (e.g., Bare Phrase Structure), which retained many of the insights of this basic system (see e.g., Chametzky, Reference Chametzky2000).

To characterize the difference between languages that prefer the linear order subject-verb vs. verb-subject, we build on Grimshaw's (Reference Grimshaw1997, Reference Grimshaw2001) analysis. Grimshaw proposed constraints on the alignment of specifiers, heads, and complements to edges of extended projections in X-bar theoryFootnote ³ . The relative weighting of these constraints derives different word ordering preferences; i.e., determining whether a particular grammar prefers an SVO- vs. an SOV-ordering of arguments. We adapt these to develop a GSC analysis, using constraints on structural well-formedness (markedness constraints). A subset of these is shown below:

1. (10) HeadLeft: “Every X⁰ is leftmost in X-max.”

   For each X⁰ in candidate C, decrease C's harmony by 1 for each terminal node intervening between the X⁰ and the left edge of its XP.

2. (11) SpecLeft: “Every specifier is leftmost in X-max.”

   For each specifier in candidate C, decrease C's harmony by 1 for each terminal node intervening between the specifier and the left edge of its XP.

A pseudo-English weighting of these two constraints is shown below. The columns show the constraints. Cells in each column show the constraint's contribution to the harmony of each candidate (scaled by the weight of the constraint). Here, since SpecLeft has a stronger weighting than HeadLeft, the subject-verb candidate has a higher harmony value. The final column gives the probability of each candidate. As in Maximum Entropy grammars (Goldwater & Johnson, Reference Goldwater, Johnson, Spenader, Eriksson and Dahl2003; Hayes & Wilson, Reference Hayes and Wilson2008), the probability is an exponential function of its harmony relative to the other candidatesFootnote ⁴ . In this example, the harmony of the subject-verb order is so much higher than verb-subject that its probability is extremely close to 1.0. While the second candidate technically has non-zero probability, it is extremely small; less than 1 × 10⁻⁸. Thus, the grammar is essentially categorical.

If the ranking of the constraints shifts, the probability of different candidates will also shift. This can specify cross-linguistic variation; consider the grammar fragment in Table 2. Here, HeadLeft has a much stronger weighting than SpecLeft. This yields a language with post-verbal subjects.

In the above cases, the differences in harmony are quite large. However, as differences in harmony of candidates grow smaller, variation can result:

In these example fragments, the weighting of constraints has been arbitrarily decided. Our assumption is that such weightings are acquired by learners based on the probability distribution of forms in their linguistic experience (Goldwater & Johnson, Reference Goldwater, Johnson, Spenader, Eriksson and Dahl2003; Hayes & Wilson, Reference Hayes and Wilson2008). For example, an English learner would acquire many examples with the word order subject-verb; all else being equal, this would lead her to favor constraint weightings similar to Table 1 over those in Table 2 or Table 3. As this discussion is focused on exploring the basic principles of the theory, we do not undertake a detailed study of this acquisition process. The conclusions we draw below will not be dependent on the particular weight values used to illustrate our analysis.

Table 1. Grammar fragment for English word order

Table 2. Grammar fragment for verb-subject word order

Table 3. Grammar fragment for variable word order

To provide concrete weight values for the purpose of illustration, we utilized Goldwater and Johnson's (Reference Goldwater, Johnson, Spenader, Eriksson and Dahl2003) learning algorithm (as implemented in the MaxEnt Grammar Tool; Hayes, Reference Hayes2009; weights were rounded to yield integer values for ease of exposition). A weak uniform prior was used for each constraint (µ = 0; σ = 10⁷). The prior influences how constraint weights are updated during learning. This prior specifies a target value for each constraint weight (here, zero, so that constraint weights are as small as possible), along with a penalty for deviating from that target value (here, the very high variance implies an extremely small penalty). This reduces our examples to a single free arbitrary parameter: the variance on the prior. Using this training algorithm, this parameter (combined with the training data exemplifying a given word order) completely determines the constraint weights below.

Head-complement word order variation

For illustration purposes, the grammar fragments above are quite simple, considering only 2 possible candidate outputs and 2 constraints. In this section, we consider a somewhat more extended example, including two additional constraints and an explicitly defined space of possible output structures. This allows us to specify monolingual grammars that contrast in word order – specifically, subject-verb-object vs. subject-object-verb (again building on Grimshaw, Reference Grimshaw1997, Reference Grimshaw2001).

Modeling constructions including complements requires an additional markedness constraint, parallel to those proposed above:

1. (12) CompLeft: “Every complement is leftmost in X-max.”

   For each complement in candidate C, decrease C's harmony by 1 for each terminal node intervening between the complement and the left edge of its XP.

Extending the set of candidates, we consider not only those that vary in word order but also those that omit elements of lexical conceptual structure. These avoid violations of the constraints above by simply leaving out elements (a candidate with no complements cannot violate CompLeft). To insure that such candidates are dispreferred, we use a faithfulness constraint that assigns well-formedness based on the relationship between syntactic and semantic structure (after Legendre, Wilson, Smolensky, Homer & Raymond, Reference Legendre, Wilson, Smolensky, Homer, Raymond, Beckman, Urbanczyck and Walsh1995):

1. (13) Parse: “Lexical conceptual structure is parsed.”

   Decrease candidate C's harmony by 1 for each element of lexical conceptual structure that does not have a corresponding element in C's surface syntactic structure.

For this discussion, we assume candidate outputs are limited to those including X-bar trees that parse all elements of lexical conceptual structure or any possible subset of elements. We further assume that elements of lexical conceptual structure are parsed into the appropriate syntactic positions (e.g., subject is parsed into Spec). For an input with a verb, subject, and object, this yields the candidate set shown in Table 4. Constraint weights were determined by training on data reflecting the context-neutral English word order for this particular input: 100% subject-verb-object (e.g., “They gave a grant”; Berk, Reference Berk1999). Note that an English language learner might have somewhat different weightings for these constraints, as she would be exposed to different inputs (e.g., inputs with no object complement) and would have a much larger set of constraints.

Table 4. Grammar fragment: English subject-verb-object word order. Blank cells indicate the candidate does not violate the constraint. Note that since CompLeft has a weighting of 0, violations of the constraint do not decrease harmony. Probabilities are rounded; those less than 1 × 10⁻⁴ are represented as 0.

This training procedure yields a strong weighting to our faithfulness constraint. Although deleting elements of lexical conceptual structure allows many candidate outputs in Table 4 to avoid violations of the markedness constraints, they incur one, two, or three penalties from the faithfulness constraint, substantially lowering their harmony. The relative ranking of the markedness constraints determines which of the first four fully faithful candidates are selected. The second most-highly weighted constraint prefers specifiers occur to the left of the Vʹ projection, ruling out the third and fourth candidates. The third constraint, preferring that heads be leftmost, then rules out the second, yielding the subject-verb-object word order.

Languages like Tamil exhibit a contrasting context-neutral word order pattern, subject-object-verb (Sarma, Reference Sarma1999; Schiffman, Reference Schiffman1999). Training on these data for the same input yields a contrasting weighting.

The sole change to the weighting is the relative strength of HeadLeft and CompLeft. Now that the latter has a higher weighting, there is a reversal of the relative harmony of the first two candidates; object complements, not verbal heads, are leftmost in Vʹ. This yields the appropriate subject-object-verb word order.

Code mixing in constraint-based grammars

Having demonstrated that our formalism can represent cross-linguistic differences in word order, we consider the grammars utilized by bilinguals (e.g., an English–Tamil bilingual). As reviewed above, in intra-sentential code mixing bilinguals integrate grammatical principles from each linguistic system. In GSC, grammars are defined by the weighting of constraints. We therefore formalize this integration by having the weights of constraints in the grammar underlying code mixing reflect both linguistic systems.

We propose to associate each linguistic system present in a code mixed utterance (L1, L2) with an activation value (α_L1, α_L2; the sum of these values must be 1). This scales the amount each linguistic system contributes to the modulation of each constraint's violations. Specifically, violations of each constraint C are scaled by the sum of the C's ranking in each linguistic system, weighted by the activation of that system. This allows for interactions between the two linguistic systems (as each contributes to harmony for every element). This scaling value is additionally increased by the activation of a linguistic system if the constraint refers to an element in that system. This latter factor encodes a (violable) preference for linguistic elements to obey the properties of the source language.

For example, suppose HeadLeft has weighting –10 in L1 and –5 in L2; the activation of L1 is 0.75 and L2 0.25. An L1 head that has 1 position intervening between it and the edge of XP will incur a harmony penalty of 1 * [–10*(0.75+0.75) + 1*–5*(0.25)] = –16.25. An L2 head that has 1 position intervening between it and the edge of XP will incur a harmony penalty of 1 * [–10*(0.75) + 1*–5*(0.25+.25)] = –10.0. The L1 head incurs a slightly greater penalty because of the stronger weighting of this constraint within the source grammar.

Blends in grammatical representations

Weighted constraints are not a novel claim of GSC; these overlap with existing formalisms including Harmonic Grammar and Maximum Entropy models. A novel feature of GSC is the incorporation of blends. Specifically, GSC proposes that elements of symbolic grammatical representations are associated with activation values. This includes all elements of syntactic representations: the nodes of the tree (both terminal and non-terminal elements) as well as the links between nodes. This allows for the specification of blends; multiple representational elements that would occupy a single position or role in a discrete symbolic representation can be co-present, to varying degrees. In previous work, we have examined the role that blends play in monolingual language processing. For example, in phonological speech errors (mispronouncing bat as pat), there is evidence that target and error sound representations are co-activated (the onset of the error syllable pat is a blend simultaneously containing elements of both /b/ and /p/; Goldrick & Chu, Reference Goldrick and Chu2014; Smolensky et al., Reference Smolensky, Goldrick and Mathis2014). Here, we extend this very general representational principle to the domain of bilingualism, focusing on blends involving elements from two distinct source languages.

For our initial discussion, we focus on cases where multiple elements are co-present to the same degree; this suffices to illustrate the general analysis. We illustrate this with example (7), repeated below for convenience.

1. (14) Doubling: Verbs English–Tamil (Sankoff et al., Reference Sankoff, Poplack and Vannianiarajan1990: 93)

   they gave me a research grant koɖutaa

   they gave me a research grant gave.3.PL.PAST

   ‘They gave me a research grant.’

We analyze the input to the grammar in code mixing contexts as consisting of blends of semantic elements. For the example above, we analyze the input as the co-presence of two verbal elements – drawn from two distinct languages – which share multiple arguments (shown in (15) below). This representation instantiates a core claim of the blend analysis of doubling constructions: the simultaneous presence, in the input to the grammar, of the semantic representation underlying the doubled elements.

1. (15)

   

Blending in syntactic representations is a key part of our analysis of doubling constructions. In general, we analyze such constructions involving two Vʹ phrases, with distinct heads, that occupy the same position in the tree. These Vʹ-projections share two complements: an indirect (me) and a direct object (grant), with these elements simultaneously associated to both Vʹ. Figure 3 illustrates the representation hypothesized for (15). Note that we adopt a ternary-branching structure for the double object construction. Our analysis does not hinge on this assumption; a binary branching VP-shell structure (Larson, Reference Larson1988) could also include blends, and would yield similar results here.

Figure 3. Hypothesized blend structure for English-Tamil doubling construction they gave me (a research) grant koɖutaa. The text below each provides a bracket notation corresponding to the tree. Dashed lines highlight blended components of the representation.

In this example, there are two Vʹ (headed by gave vs. koɖutaa), sharing the complements me and grant. These two Vʹ simultaneously serve as the head of VP (a blend of nodes in the same position in the tree). This is a key part of our analysis, as it places each of the doubled elements in the same role within the syntactic structure. This sets up the structural relationships that ensure the doubled elements exhibit parallel tense, aspect, agreement, and case features.

Thus far, our discussion of blending has focused on the blending of elements within a structural position (e.g., two Vʹ projecting from the head position; two verbs occupying the same role in semantic structure). However, as each element of a representation can be associated with an activation value, blending is predicted to extend to the roles themselves. This is depicted in Figure 3, where the same arguments are shared across the two phrases. This represents a blending of positions or syntactic relationships; one node in the tree (e.g., the indirect object) has the same type of link to two distinct nodes (e.g., Vʹ headed by an English verb and the Vʹ headed by a Tamil verb). In our analysis, this sharing of the complements is critical, as it allows the blended structure to be linearized. If two lexical items simultaneously occupy the head of VP, there is no way to determine which one of them should occur first in the surface string; simultaneity means there is no precedence relationship between the phrases. However, because the precedence relationships within each phrase share a common element (gave precedes me and grant, me and grant precedes koɖutaa), a complete ordering of the terminals can be determined (by transitivity, gave must precede koɖutaa).

As reviewed in preceding sections, there is ample empirical evidence that blends are subject to both physical constraints (e.g., unimodal bilinguals cannot place articulators in contradictory positions) as well as cognitive constraints (e.g., affective/pragmatic constraints, relative strength of the two languages, grammatical context). GSC suggests a clear theoretical motivation for some of these cognitive constraints (Smolensky et al., Reference Smolensky, Goldrick and Mathis2014). In many cases, purely grammatical constraints will prefer blends that do not reflect the structural principles of the source grammars. For example, if constraints prefer that all elements be at the left edge (i.e., specifier) of XP, why not place all elements in that position simultaneously? This candidate would satisfy all the constraints above; in the absence of other principles, it would have the highest harmony. By allowing the grammar to avoid making choices between structures exhibiting different word orders, this blend representation would prevent GSC from capturing key properties of cross-linguistic variation. GSC, unlike many previous connectionist proposals, therefore incorporates an explicit dispreference for blend representations (Smolensky et al., Reference Smolensky, Goldrick and Mathis2014). For the purposes of this discussion (focused on simultaneous presence of equally active elements), we represent this as a constraint that simply refers to the presence vs. absence of blended elementsFootnote ⁵ :

1. (16) Quantization: “Candidates must be discrete symbolic structures.”

   For each blended structural element in candidate C, decrease C's harmony by 1.

It is important to emphasize that Quantization plays a key role in monolingual grammars. Although we have emphasized their role in bilingual language processing, blend representations are a ubiquitous feature of monolingual processing as well (Melinger et al., Reference Melinger, Branigan and Pickering2014). All grammatical computations – not only those involved in code mixing – must therefore evaluate representations where multiple elements occupy the same structural role (Smolensky et al., Reference Smolensky, Goldrick and Mathis2014).

With respect to the current discussion, it is important to note that Quantization is violable; other constraints can compel the presence of blends. The next section examines a situation in which this can occur.

Analysis of doubling constructions

Combining the results of the previous sections, we examine conditions under which doubling constructions can be produced. We limit ourselves to candidates that parse all elements of lexical conceptual structure or any possible subset of elements (in this code mixing case, this includes elements from multiple languages). Parsed elements are associated with appropriate syntactic positions. We extend this candidate set to include the blend structure depicted in Figure 3, simplifying the example by omitting the indirect object. Following the section on code mixing, the constraint rankings in Table 4 [English] and Table 5 [Tamil] are combined to determine the grammar used to evaluate code-mixed constructions. We include the Quantization constraint as a language-independent constraint, examining how its relative weighting affects the probability of blend constructions.

Table 5. Grammar fragment: Tamil subject–object-verb word order. Blank cells indicate the candidate does not violate the constraint. Note that since HeadLeft has a weighting of 0, violations of the constraint do not decrease harmony. Probabilities are rounded; those less than 1 × 10⁻⁴ are represented as 0.

The tableau in Table 6 illustrates one ranking that qualitatively approximates the empirical distribution of doubling constructions – non-zero, but relatively small probability.

Table 6. Grammar fragment: Doubling construction, Tamil–English code mixing (see appendix for full set of candidates). Blank cells indicate the candidate does not violate the constraint. Probabilities are rounded; those less than 1 × 10⁻⁴ are represented as 0.

The observed doubling construction (candidate a) violates several markedness constraints:

• • With respect to HeadLeft, it receives a total harmony penalty of 30 due to:

  • ○ Language general constraints: –24 = –6 * 4 violations (3 for koɖutaa and 1 for gave)

  • ○ English-specific constraints: –6 = –6 * 1 violation (for gave)

  • ○ Tamil-specific constraints: 0 = 0 * 3 violations (for koɖutaa)

• • For CompLeft the penalty is –12:

  • ○ Language general constraints: –12 = –6 * 2 violations (for grant)

  • ○ English-specific constraints: 0 = 0 * 2 violations (for grant)

• • Two violations of the Quantization constraint yield a penalty of –16 (for the two Vʹ simultaneously projecting from the head of VP, as well as grant occurring as a complement in both Vʹ).

However, unlike candidates that delete English or Tamil verbs (e.g., candidate c), the observed doubling constructions avoid violations of the faithfulness constraint Parse. The probability of the doubling construction relative to non-doubled candidates like (c) is therefore related to the weighting of faithfulness relative to the markedness constraints above.

Assuming faithfulness has a strong enough weight to compel the presence of doubling, the attested candidate (a) will be preferred to unattested candidate (b) due to the influence of language-specific constraints. The two candidates incur equal violations of the language general constraints, but (b) incurs extra violations of language-specific constraints (2 additional English-specific violations for gave). So long as these language-specific constraints have a non-zero weighting, the grammar will assign higher probability to the attested form.

Predicted limitations on doubling constructions

Our analysis above focused on an example that, following the empirical patterns of doubling, consisted of two heads with a shared complement. We assume that doubled complements – specifically, arguments of verbs – are unattested because such structures would violate Chomsky's Theta Criterion, which states that “each argument bears one and only one theta-role, and each theta-role is assigned to one and only one argument” (Reference Chomsky1981: 35; see also Chan, Reference Chan2003, Reference Chan2008, Reference Chan, Bullock and Toribio2009). For example, contrast the attested they gave_English grant gave_Tamil with the unattested they grant_Tamil gave grant_English . In the latter, the Theta Criterion is violated as both grant_Tamil and grant_English share the same role (theme). In contrast, in the attested example the theme is occupied by a single entity (grant_English , shared across the two verbs) so there is no violation. We assume that violation of this constraint either causes the grammar to categorically rule out such structures or, alternatively, greatly reduces their probability (if the Theta Criterion is realized via a strongly weighted constraint).

A novel prediction of our account is that different distributions of doubling should be observed for expletive vs. non-expletive elements. Expletive elements are those that appear solely for structural considerations and are semantically vacuous (e.g., in It's raining; the pronoun it does not actually refer to a specific agent). Our analysis attributes the presence of doubling to the co-presence of multiple elements in the input to the grammar. This is reflected by the crucial role of faithfulness; Parse provides an advantage for doubling constructions, in spite of their increased violations of alignment constraints and Quantization. This makes a novel prediction: we should not observe doubling of expletive elements (e.g., English do) alone. Following Grimshaw (Reference Grimshaw1997, Reference Grimshaw2001, Reference Grimshaw, Broekhuis and Vogel2013, a.o.), the occurrence of such elements can be attributed to structural (i.e., markedness) constraints rather than to the presence of expletives in the input to the grammar. As they only appear to satisfy the structural requirements of other elements, our account predicts that expletives should not be doubled in isolation.

To make this concrete, consider a case where doubling could be predicted. English and Korean both utilize do-support in negatives (Grimshaw, Reference Grimshaw, Broekhuis and Vogel2013), but exhibit contrasting word order. Like the verb, Korean negatives (and do) appear following the object, the opposite of English:

1. (17) Chelswu-ka ppang-ul mek-ci ani

   Chelswu-NOM book-ACC read-CI NEG

   ha-ess-ta (Hagstrom, Reference Hagstrom, Ahn, Kim and Lee1996: 169)

2. do-PAST-DECL

   ‘Chelswu did not read the book.’

An account that attributed the presence of doubling to contrasting surface word orders would predict that doubling of either the verb, negative morpheme, or do alone could occur in Korean–English code mixing. In contrast, our analysis predicts that the doubling of do alone should not occur. The presence of do here reflects structural well-formedness constraints, triggered by the presence of negation (Grimshaw, Reference Grimshaw1997, Reference Grimshaw, Broekhuis and Vogel2013). There is no independent motivation to include this expletive aside from this. Thus, doubling of do alone would violate constraints such as Quantization while providing no benefit with respect to constraints such as Parse. (In fact, insertion of expletive elements not present in the input may violate faithfulness constraints such as Grimshaw's FullInt.)

Bilingual doubling cannot be analyzed as movement

Having outlined our proposal, we briefly consider whether existing analyses of doubling could provide an alternative to our analysis. Doubling of elements in monolingual grammars has been a focus of recent generative research (see, e.g., the contributions in Barbiers, Koeneman, Lekakou & van der Ham, Reference Barbiers, Koeneman, Lekakou and van der Ham2008). This is typically analyzed as resulting from phonological realization of multiple links in a representational structure linking a syntactic element from its location in the surface syntactic structure to other distal locations in the syntactic tree (e.g., derivational chains; Jónsson, Reference Jónsson, Barbiers, Koeneman, Lekakou and van der Ham2008; Nunes, Reference Nunes2004). Such an analysis does not appear to be tenable for the attested examples of bilingual doubling. Consider the Tamil–English example analyzed above, where there is doubling of the verb gave. There is no clear motivation for such movement in the grammar of either English or Tamil. Even if we were to entertain such an analysis, it would violate a basic principle of the locality of head movement – the head of a projection (here, the verb) cannot undergo movement within that projection (Abels, Reference Abels2003).

Gradient co-activation and directions for future research

Gradient activation in blends – the key to accounting for the range of psycholinguistic data reviewed in the introduction to this paper – is clearly outside the scope of any traditional grammatical theory. In the GSC framework, such representations are possible inputs and outputs to the grammar, and are assigned Harmony values by constraints. Specifically, the violation of each constraint reflects the activation of the constituents referred to by the constraint. For example (c.f. (13), emphasis added to show contrast in definitions):

1. (18) Parse: “Lexical conceptual structure is parsed.”

   Decrease candidate C's harmony by the activation of each element of lexical conceptual structure that does not have a corresponding element in C's surface syntactic structure.

The incorporation of activation stems from principles of connectionist computation (Legendre et al., Reference Legendre, Miyata and Smolensky1990; Smolensky & Legendre, Reference Smolensky, Legendre, Smolensky and Legendre2006). GSC-representations are realized by real-valued activation vectors over simple processing units. Over the course of computation, activation spreads among these units via weights that implement grammatical constraints (Smolensky et al., Reference Smolensky, Goldrick and Mathis2014). Critically, the activation values are continuously updated; the network does not simply ‘jump’ from one gradient symbolic representational state to another. In order to insure that this continuous update respects the well-formedness conditions specified by the grammar, constraints must assign well-formedness values to the full range of intermediate, gradient representational states. Harmony therefore varies with the activation of each representational constituent.

In the context of code mixing, gradient activation of elements in the input will alter the relative probability of these elements appearing in the output. This is because violations of faithfulness constraints like Parse will be scaled by activation. For a given weighting of Parse, the harmony penalty incurred by deleting an element will be less if the element has a lower vs. higher activation value. Less active elements will therefore be more likely to be deleted. The tableaux in Table 7 and Table 8 illustrate this for a simple subject-verb sentence (here, we assume Quantization is strongly weighted, blocking the appearance of output blends).

Table 7. Effects of variation in input activation: Strong bias towards English vs. Tamil. Note that competitors involving additional deletion of input elements have been omitted (due to violations of Parse, they have very low Harmony and thus output probability near 0).

Table 8. Effects of variation in input activation: Weaker bias towards English vs. Tamil. Note that competitors involving additional deletion of input elements have been omitted (due to violations of Parse, they have very low Harmony and thus output probability near 0).

In these tableaux, violations of Parse are scaled by the activation of the input element. For example, in Table 7, the first candidate deletes koɖutaa. Language general constraints assign a violation of –10 = 0.4 * –25 and Tamil specific constraints assign a violation of –5 = 0.4 * –12.5. Compare this to the first candidate in Table 8. Here, the violation of language general constraints increases to –11 = 0.44 * –25 and Tamil-specific constraints to –5.5 = 0.44 * –12.5. As the activation of an input element increases, the cost of deleting it also increases – it becomes more critical to preserve the element (reflected in the shift in output probabilities across the two tableaux).

Gradient activation of elements in the output provides a mechanism for modeling the data reviewed in the first sections of this paper; gradient blends observed in phonological and articulatory processing in spoken and signed languages (e.g., the co-activation of <DOG> and <PERRO> during Spanish–English bilingual production, depicted in Figure 1). Clearly, gradient symbol structures have the expressive capability to represent such structures; throughout this discussion, we have assumed graded activation of elements of the input to the grammar. Our claim is that the degree of blending present in the output reflects grammatical computations.

Outside of numerical simulations, the final blend states of our first implementation of the Quantization constraint (Smolensky et al., Reference Smolensky, Goldrick and Mathis2014) cannot be determined. In more recent work (Tupper and Smolensky, in progress) we have therefore developed new realizations of this constraint that are more amenable to analysis. Using these methods, we can calculate the optimal blend state predicted by the grammar.

To illustrate this approach, we considered the scenario shown in Figure 1 – the coactivation of two nouns in the head of an NP consisting of a determiner and noun. For this computation, we simplified our grammar, focusing only on Quantization and Faithfulness (as Spanish and English agree on word order for nouns in these phrases). Following the scenario depicted in Figure 1, we assumed that English has greater activation than Spanish. As shown in Figure 4, this set of constraint weightingsFootnote ⁶ assigned highest Harmony to a blend state that is closest to <DOG> (reflecting the higher activation of English vs. Spanish), yet contains some partial activation of the translation equivalent <PERRO> (reflecting the relative weighting of Quantization). Critically, this degree of blending is not assumed, but is rather derived from the constraints of the grammar.

Figure 4. Relative Harmony (e^H ) of various blends of <DOG> (activation shown on X axis) and <PERRO> (activation shown on Y axis); lighter color indicates higher Harmony. The optimal blend (0.81 <DOG>, 0.15 <PERRO>) is marked with an x.

Given input activations and relative constraint weightings, a GSC theory will make predictions about multiple facets of code mixed productions: discrete (e.g., output probabilities of various structures) as well as gradient (e.g., coactivation of translation equivalents). To develop this account, it is important that we gain a more precise understanding of the factors that facilitate (and inhibit) the activation of representations within each of a bilingual's languages during sentence processing as well as how bilinguals learn the relative weightings of grammatical constraints. Critically, GSC provides us with a framework that can integrate these various influences on code mixing – allowing us to develop a unified account of discrete and gradient properties of bilingual linguistic knowledge and processing.