Introduction
Recent years have seen increasing research into bimodal bilingualism from a variety of paradigms such as bilingual acquisition, language processing, neural systems, and cognitive skills, with the underlying assumption that successful bimodal bilingualism entails the knowledge representations and processing of two grammars each of which via a distinct modality, auditory-oral versus visual-gestural. As such, it opens up an arena of cutting-edge research enabling comparisons of the linguistic and cognitive effects of monolingualism versus bilingualism, as well as unimodal bilingualism versus bimodal bilingualism.
Emmorey, Giezen and Gollan (Emmorey, Giezen & Gollan) discuss available studies on language processing and neural systems, mostly based on lexical retrieval in language production and comprehension. The findings suggest online co-activation of two lexical representations by bimodal bilinguals, as similar to what uniimodal bilinguals do. Yet, modality-specific properties of signed language and spoken language do result in qualitative differences in terms of processing cost, executive functioning skills and involvement of neural systems. One crucial difference is that bimodal bilinguals demonstrate a capacity for producing and comprehending code-blends as a preferred code-mixing strategy to code switching. This observation is further substantiated by results of psycholinguistic experiments showing that producing or switching from monolingual codes to code blends is less costly, if not ‘cost free’, in terms of inhibitory control, when contrasting the cost of inhibition in code switching with unimodal bilinguals. Also, tests of language comprehension reveal faster processing speed with code blends than sign alone or word alone conditions. Another insight is about the nature of co-activation, where the links between signs and words are observed to be more semantic than phonological, despite modality and articulatory differences between the two languages.
Other factors interacting with bimodal bilingual processing
Modality differences aside, another factor that may interact with co-activation and inhibitory control is the proficiency levels of the two languages of bimodal bilinguals. This factor may in turn vary as a function of the user's hearing status, or hearing level, age of language onset, as well as language dominance at different time points of bilingual development. Information about bilingual proficiency assessment as control in the investigation would shed more light on whether it is language modality per se, bilingual proficiency per se, or the mixed effects of language modality and bilingual proficiency that lead to some of the current observations. Understandably, as research on bimodal bilingual acquisition is just emerging (Donati & Branchini, Reference Donati, Branchini, Biberauer and Roberts2013, Fung, Reference Fung2012, Lillo-Martin, de Quarros, Chen Pichler & Zoe, Reference Lillo-Martin, de Quarros and Chen Pichler2014, van den Bogaerde & Baker, Reference van den Bogaerde and Baker2005), more information is needed regarding the nature of the linguistic representations of the two grammars, especially those of signed languages, and how they interact with each other in bimodal bilingual processing. From a bilingual acquisition point of view, these factors may determine what kinds of knowledge from the two linguistic systems are more accessible as well as how efficient the processing and inhibitory control are going to be at different points of bimodal bilingual development. As such, language dominance in bilingual acquisition and bilingual processing will play a role. Also, insights from this paradigm of research may shed light on the design of language processing research in future.
As said, most of the experimental studies compared the production or comprehension of monolingual codes and semantically congruent lexical blends, where the inhibitory control of code blends is found to be minimal. Nevertheless, research based on spontaneous productions also reported the less frequently observed semantically incongruent code-blends, which far exceeds explanatory adequacy at the lexical representation level. Emmorey et. al. (Emmorey et al.) predict that knowledge of syntactic structure of the non-target language must be inhibited under those circumstances, especially when there is conflict in syntactic structure between the signed language and the spoken language. Examples 5–10 of code blends as cited in the paper, produced by the ASL–English, NGT–Dutch and LIS–Italian CODAs, actually demonstrate interesting cases of merge of lexical signs with the heads of syntactic projections of a spoken language, sometimes referred to the base/matrix language in the code mixing literature (Myers-Scotton, Reference Myers-Scotton2002). The co-activation of two distinct syntactic structures in language processing, one from a signed language and the other from a spoken language, is also seen in Examples 11 and 12. These data give further proof that co-activation is not restricted to the lexical representations alone and may involve access to the syntactic representations of the two languages involved. A more detailed analysis of Examples 5–12 shows that there is a time course for lexical sign insertion into the syntactic structure of the spoken language. Example 8, for instance, shows the simultaneous merge of a demonstrative ‘this’ in English and a head noun ‘CONNECTION’ in ASL, presumably leading to the formation of a Determiner Phrase. In Example 9, the adverb ‘all-of-a-sudden’ in English is co-temporally produced with the verb “LOOK-AT”, and the latter fills the head position of a Verb Phrase presumably scoped by the temporal adverb in English. In Example 10, the subject “HUNTER” in LIS is co-temporally articulated with the object “con Biancaneve (with Snow White)” in Italian, and both constituents are preceded by a blended main verb “Parla/TALK” (Note that verbs may occur in the sentence initial position in Italian, assuming that it is the base language). All these examples show that code-blending is also governed by rules of syntactic structure of natural language grammar. If the analysis is on the right track, one may hypothesize that co-activation is the norm with bimodal bilinguals, who may exercise inhibitory control in various degrees subject to the nature of code-blending as well as demands of the retrieval process at different linguistic levels. Hence, Examples 11–12 may indicate minimal inhibitory control when the respective syntactic structure of the two languages are co-activated in the numeration. On the other hand, Examples 5–10 may demand different degrees of inhibitory control, subject to demands of merging different linguistic elements at specific sites of the syntactic structure. So far as the examples are concerned, it seems that suppressing the syntactic structure of signed language allows for the syntactic structure of the spoken language to serve as the base, while the lexical categories of the spoken language at certain syntactic nodes may be suppressed to allow for sign selection, as shown by the outputs like “this+CONNECTION” or “HUNTER+con Biancaneve (hunter with Snow White)”. In other words, processes of co-activation of linguistic representations and inhibitory control may be bidirectional, although their cause or cost in terms of language processing needs to be addressed in future research.
As said, different combinations of bilingual proficiencies or the backgrounds of bilingual acquisition with bimodal bilinguals may interact with the extent of co-activation of linguistic representations and degree of inhibitory control of two linguistic systems. Therefore, more research is needed in order to arrive at a better understanding of bimodal bilingualism or even the much debatable ‘bilingual advantage’. An alternative approach, which may be more fruitful, is to identify the qualitative differences in terms of language processing procedures, executive control skills and neurocognitive functioning, and examine their relative, if not unique, contributions to bilingual performance. Quite rightly put by Emmorey et. al. (Emmorey et al.), bimodal bilingualism brings with it the necessary cognitive skills required for processing signed language, like spatial memory, facial recognition, transforming mental images, and the like. Unless these skills are proven to be in conflict with the processing of some other neurocognitive functions, they also implicate ‘sign advantage’ for bimodal bilinguals, when compared with unimodal bilinguals or monolinguals. In fact, emerging brain imaging studies are beginning to show the extent of use of neural systems in language processing with monolinguals, unimodal bilinguals and bimodal bilinguals, suggesting that differences are bound to occur, especially in how the brain economizes the use of neural regions in speech and signed language processing, either sequentially or simultaneously. From an applied perspective, it is most encouraging to see evidence suggesting that modality differences do not block co-activation of signed language and spoken language at different linguistic levels, implying that deaf or hearing bimodal bilinguals have the flexibility of drawing linguistic resources from both modalities in their language development and language use. It offers insights to educators for the deaf in identifying ways and means to create a supportive educational environment that nurtures bimodal bilingualism.
Introduction
Recent years have seen increasing research into bimodal bilingualism from a variety of paradigms such as bilingual acquisition, language processing, neural systems, and cognitive skills, with the underlying assumption that successful bimodal bilingualism entails the knowledge representations and processing of two grammars each of which via a distinct modality, auditory-oral versus visual-gestural. As such, it opens up an arena of cutting-edge research enabling comparisons of the linguistic and cognitive effects of monolingualism versus bilingualism, as well as unimodal bilingualism versus bimodal bilingualism.
Emmorey, Giezen and Gollan (Emmorey, Giezen & Gollan) discuss available studies on language processing and neural systems, mostly based on lexical retrieval in language production and comprehension. The findings suggest online co-activation of two lexical representations by bimodal bilinguals, as similar to what uniimodal bilinguals do. Yet, modality-specific properties of signed language and spoken language do result in qualitative differences in terms of processing cost, executive functioning skills and involvement of neural systems. One crucial difference is that bimodal bilinguals demonstrate a capacity for producing and comprehending code-blends as a preferred code-mixing strategy to code switching. This observation is further substantiated by results of psycholinguistic experiments showing that producing or switching from monolingual codes to code blends is less costly, if not ‘cost free’, in terms of inhibitory control, when contrasting the cost of inhibition in code switching with unimodal bilinguals. Also, tests of language comprehension reveal faster processing speed with code blends than sign alone or word alone conditions. Another insight is about the nature of co-activation, where the links between signs and words are observed to be more semantic than phonological, despite modality and articulatory differences between the two languages.
Other factors interacting with bimodal bilingual processing
Modality differences aside, another factor that may interact with co-activation and inhibitory control is the proficiency levels of the two languages of bimodal bilinguals. This factor may in turn vary as a function of the user's hearing status, or hearing level, age of language onset, as well as language dominance at different time points of bilingual development. Information about bilingual proficiency assessment as control in the investigation would shed more light on whether it is language modality per se, bilingual proficiency per se, or the mixed effects of language modality and bilingual proficiency that lead to some of the current observations. Understandably, as research on bimodal bilingual acquisition is just emerging (Donati & Branchini, Reference Donati, Branchini, Biberauer and Roberts2013, Fung, Reference Fung2012, Lillo-Martin, de Quarros, Chen Pichler & Zoe, Reference Lillo-Martin, de Quarros and Chen Pichler2014, van den Bogaerde & Baker, Reference van den Bogaerde and Baker2005), more information is needed regarding the nature of the linguistic representations of the two grammars, especially those of signed languages, and how they interact with each other in bimodal bilingual processing. From a bilingual acquisition point of view, these factors may determine what kinds of knowledge from the two linguistic systems are more accessible as well as how efficient the processing and inhibitory control are going to be at different points of bimodal bilingual development. As such, language dominance in bilingual acquisition and bilingual processing will play a role. Also, insights from this paradigm of research may shed light on the design of language processing research in future.
As said, most of the experimental studies compared the production or comprehension of monolingual codes and semantically congruent lexical blends, where the inhibitory control of code blends is found to be minimal. Nevertheless, research based on spontaneous productions also reported the less frequently observed semantically incongruent code-blends, which far exceeds explanatory adequacy at the lexical representation level. Emmorey et. al. (Emmorey et al.) predict that knowledge of syntactic structure of the non-target language must be inhibited under those circumstances, especially when there is conflict in syntactic structure between the signed language and the spoken language. Examples 5–10 of code blends as cited in the paper, produced by the ASL–English, NGT–Dutch and LIS–Italian CODAs, actually demonstrate interesting cases of merge of lexical signs with the heads of syntactic projections of a spoken language, sometimes referred to the base/matrix language in the code mixing literature (Myers-Scotton, Reference Myers-Scotton2002). The co-activation of two distinct syntactic structures in language processing, one from a signed language and the other from a spoken language, is also seen in Examples 11 and 12. These data give further proof that co-activation is not restricted to the lexical representations alone and may involve access to the syntactic representations of the two languages involved. A more detailed analysis of Examples 5–12 shows that there is a time course for lexical sign insertion into the syntactic structure of the spoken language. Example 8, for instance, shows the simultaneous merge of a demonstrative ‘this’ in English and a head noun ‘CONNECTION’ in ASL, presumably leading to the formation of a Determiner Phrase. In Example 9, the adverb ‘all-of-a-sudden’ in English is co-temporally produced with the verb “LOOK-AT”, and the latter fills the head position of a Verb Phrase presumably scoped by the temporal adverb in English. In Example 10, the subject “HUNTER” in LIS is co-temporally articulated with the object “con Biancaneve (with Snow White)” in Italian, and both constituents are preceded by a blended main verb “Parla/TALK” (Note that verbs may occur in the sentence initial position in Italian, assuming that it is the base language). All these examples show that code-blending is also governed by rules of syntactic structure of natural language grammar. If the analysis is on the right track, one may hypothesize that co-activation is the norm with bimodal bilinguals, who may exercise inhibitory control in various degrees subject to the nature of code-blending as well as demands of the retrieval process at different linguistic levels. Hence, Examples 11–12 may indicate minimal inhibitory control when the respective syntactic structure of the two languages are co-activated in the numeration. On the other hand, Examples 5–10 may demand different degrees of inhibitory control, subject to demands of merging different linguistic elements at specific sites of the syntactic structure. So far as the examples are concerned, it seems that suppressing the syntactic structure of signed language allows for the syntactic structure of the spoken language to serve as the base, while the lexical categories of the spoken language at certain syntactic nodes may be suppressed to allow for sign selection, as shown by the outputs like “this+CONNECTION” or “HUNTER+con Biancaneve (hunter with Snow White)”. In other words, processes of co-activation of linguistic representations and inhibitory control may be bidirectional, although their cause or cost in terms of language processing needs to be addressed in future research.
As said, different combinations of bilingual proficiencies or the backgrounds of bilingual acquisition with bimodal bilinguals may interact with the extent of co-activation of linguistic representations and degree of inhibitory control of two linguistic systems. Therefore, more research is needed in order to arrive at a better understanding of bimodal bilingualism or even the much debatable ‘bilingual advantage’. An alternative approach, which may be more fruitful, is to identify the qualitative differences in terms of language processing procedures, executive control skills and neurocognitive functioning, and examine their relative, if not unique, contributions to bilingual performance. Quite rightly put by Emmorey et. al. (Emmorey et al.), bimodal bilingualism brings with it the necessary cognitive skills required for processing signed language, like spatial memory, facial recognition, transforming mental images, and the like. Unless these skills are proven to be in conflict with the processing of some other neurocognitive functions, they also implicate ‘sign advantage’ for bimodal bilinguals, when compared with unimodal bilinguals or monolinguals. In fact, emerging brain imaging studies are beginning to show the extent of use of neural systems in language processing with monolinguals, unimodal bilinguals and bimodal bilinguals, suggesting that differences are bound to occur, especially in how the brain economizes the use of neural regions in speech and signed language processing, either sequentially or simultaneously. From an applied perspective, it is most encouraging to see evidence suggesting that modality differences do not block co-activation of signed language and spoken language at different linguistic levels, implying that deaf or hearing bimodal bilinguals have the flexibility of drawing linguistic resources from both modalities in their language development and language use. It offers insights to educators for the deaf in identifying ways and means to create a supportive educational environment that nurtures bimodal bilingualism.