2.1 Cognitive and social-psychological affordances of (online) gaming worlds

There has been considerable research on the structure, organization, and forms of social practice that play out in MMOs, and particularly WoW (e.g., Gee, Reference Gee2003; Grimes & Feenberg, Reference Grimes and Feenberg2009; Nardi & Kallinikos, Reference Nardi and Kallinikos2010; Steinkuehler & Duncan, Reference Steinkuehler and Duncan2008; Thomas & Brown, Reference Thomas and Brown2009). These studies have uniformly assessed WoW and other MMOs as developmentally rich environments for the learning of literacy, scientific reasoning, problem solving, leadership, and collaboration. Aspects of game world design have also been argued to be relevant to rethinking educational cultures and environments, for instance the fact that players of differing ability levels share a common space and frequently interact with one another. In most game worlds, knowledge is freely shared because everyone benefits when there are better players to collaborate with, and over time, all players have the prospect of progressing to higher levels of accomplishment. Gee (Reference Gee2005) has described such dynamics using the term “affinity spaces” and has argued that formal education could well be improved by adopting the pedagogical and developmental cultures that are common to multiplayer online games.

It is relevant to note that other commentators and researchers have expressed more negative views of cognitive (e.g., Carr, Reference Carr2010) and social-psychological (Turkle, Reference Turkle2011) processes that pertain to Internet environments as well as to single and multiplayer online games. In regard to social media, for example, Turkle, who in the 1990s positively assessed the many identity construction opportunities in Internet-based virtual and gaming environments (e.g., Turkle, Reference Turkle1995), has more recently condemned social media, arguing that immersion in the cacophony of social media networks and ‘friendships’ may result in a sense of increased isolation (Turkle, Reference Turkle2011; though see Steinfield, Ellison & Lampe, Reference Steinfield, Ellison and Lampe2008, for a longitudinal study that suggests an opposite effect). In relation to L2 learning and video games, deHaan (Reference DeHaan2005) reviewed several single player and console based game genres and concluded that they were limited as environments for L2 learning. deHaan specifically faulted such games for their contrived and genre-limited language content (sports and ‘virtual pet’ games), the relative irrelevance of language to game play (simulation games), and in games that are language-intensive, for instance those in the role playing and action adventure genres, that the gaming environment does not allow learners to revisit, pause, or otherwise moderate the highly complex narratives and linguistic input in a way that would assist with comprehension and L2 development, especially for lower level language learners (though see Piirainen-Marsh & Taino, Reference Piirainen-Marsh and Tainio2009, for a contrary assessment). Given the large volume of time that many gamers spend in MMO settings, the criticism that online activity displaces (arguably) more meaningful face-to-face engagement, or presents learners with ‘inauthentic’ or overly complex language, are important issues that warrant continued exploration (for further discussion, see Thorne & Fischer, Reference Thorne and Fischer2012).

In other research within the L2 literature, Peterson (Reference Peterson2010) produced a meta-analysis of L2 research on gaming that outlined a number of positive attributes, among them high motivation, the need and opportunity for collaboration, and reduced that communicative inhibition. In a study examining an intercultural encounter in WoW, Thorne (Reference Thorne2008) analysed the transcripts of text-chat communication between an American and a Ukrainian player. His analysis illustrated a number of positive assets for language learning, such as natural and unscripted interaction, reciprocal alterations in expert status, explicit self- and other-correction at the level of linguistic form, extensive repair sequences, development of a positive affective bond, and exhibited motivation by both parties for learning the other's language. While existing research describes the considerable potential of MMOs as sites for L2 learning (e.g., Reinders, Reference Reinders2012; Sykes et al., Reference Sykes, Reinhardt and Thorne2010), there remains an outstanding need for meticulous empirical studies that critically evaluate the environments games provide for L2 learning.

2.2 The Anatomy of an MMO: World of Warcraft

As described above, WoW is a commercial massively multiplayer game, produced by Blizzard Entertainment, within which thousands of people simultaneously engage in solo play, collaborative play, match competitions and duels, commercial activity (making, buying, and selling items in an in-game economy), and participation in social interaction more generally. Game play involves controlling a digital avatar within a 3-D virtual world and completing game-generated “quests” or goal-oriented tasks to advance one's character to higher levels. As will be further described below, quests are presented to players as pop-up text windows that are approximately 100 words in length. In-game communication includes a variety of synchronous chat channels, an asynchronous email system, and a number of voice-communication tools that enable audio communication between players. Players also frequent game-external websites for advice regarding character development, for assessing the relative value of various items in the game (e.g., armor and weapons), to glean strategy for completing individual quests and more challenging group play (called instance runs and raids), and to pursue interests in the narrative, lore, and popular culture outgrowths of the game (e.g., machinima, fan fiction).

2.3 Global objective and procedures for the study

The study presented here includes questionnaires, completed by 64 gamers in total, and follow-up oral interviews with ten of the respondents. The responses provided by these players helped to identify and inform the selection of texts that were analysed for their linguistic complexity. By analysing the linguistic complexity of the texts that WoW players frequently engage with, this study aims to empirically document the quality of the linguistic ecology comprising a representative and widely played MMO, and to then describe its affordances and limitations in view of L2 development.

3.1 Method: Bi-national questionnaire and interviews with WoW players

In order to establish the routine practices and experiences of WoW gamers, in the Spring of 2011, the authors created a questionnaire instrument. The questionnaire was targeted toward Dutch players living in the Netherlands, for whom English was an L2, and American players living in the United States, in order to gain an international perspective of WoW gaming experience. The questionnaire consisted of 40 questions, beginning with demographics and background (e.g., age, gender, education, nationality); frequency of play; use of external websites and resources; the nature of their specific WoW experience including the official language of the realm they play on, membership within guilds (akin to structured clubs or teams) and the balance of solo to collaborative play; patterns of socializing and communication, and preferred communication tools. These topics were then further explored through self-selected and voluntary follow-up interviews with ten of the questionnaire respondents. These interviews were carried out via video conferencing (Skype), and in one instance, in a face-to-face format.

Only a subset of the questionnaire and interview data will be addressed here, namely the ways that players described interacting with quest texts and their use of websites that are external to the game (for analyses and discussion of other aspects of this project, see Thorne & Fischer, Reference Thorne and Fischer2012).Footnote ² As a caveat, we acknowledge that questionnaire data have the potential to be inaccurate, but numerous studies have also found high levels of congruency between self-reported data and objective assessments (e.g., Crockett, Schulenberg, & Petersen, Reference Crockett, Schulenberg and Petersen1987). Additionally, responses to the questionnaire regarding the focal themes explored here, namely the reading of quest texts and the use of external websites, were highly consistent across the bi-national sample of respondents. An additional ‘in-group member check’ is possible since two of the authors (Thorne and Fischer) have collectively played WoW for hundreds of hours, including attaining the maximum level caps for two earlier iterations of the game, and our play experiences allow us to further corroborate the validity of the questionnaire results.

3.2 Demographic profile of the questionnaire respondents

Dutch and English versions of the questionnaire were made available and it was distributed through various social networks (including the WoW Facebook page), emailed directly to Dutch and American players known by the authors, and posted to online WoW community websites with the encouragement for initial respondents to redistribute the questionnaire to other players. There was an equal number of Dutch (N = 32, 16 females, 16 males) and American (N = 32, 11 females, 21 males) respondents. On average, the Dutch participants were younger (M = 26.6, SD = 8.9) than the Americans (M = 33.1, SD = 9.2). None of the questionnaire respondents were playing WoW specifically to learn a foreign language, but all of the Dutch and American participants reported playing on English language realms. The respondents from the Netherlands all indicated that Dutch was their L1 and that they had learned English as an L2. 93% of the Dutch players indicated that they spoke English at an advanced level. In contrast, 31 of the 32 Americans were playing in their L1 of English.

3.3 Game play experience

The Dutch and American groups were equally, and generally highly, experienced in playing MMOs. On a scale of 1-5 for experience, the Dutch had an average score of 4.3 (SD = .99) and the American participants an average score of 4.6 (SD = .71). An independent samples t-test showed no significant difference between the two groups (t(62) = −1.3; p > .05). In both groups, most participants had been playing for more than three years (Figure 1), between one and four times a week (Figure 2), for an average of three hours at a time (Figure 3).

Fig. 1 Length of time Dutch and American participants have been playing WoW

Fig. 2 Frequency with which Dutch and American participants have been playing WoW

Fig. 3 Number of hours Dutch and American participants play WoW at a time

3.4 Identification and use of external WoW-related websites and resources

In both the Dutch and American questionnaire and interview groups, all respondents reported that they often used external websites relating to WoW. The results demonstrate that WoW players looked up information on how to find certain items, how to complete quests, information on the different player classes, strategy issues, and sites related to WoW lore and mythology, to name only the most frequently mentioned reasons.

Three websites were designated as the most frequently visited. The first was wowhead.com, used to look up quest strategy and information about items such as armor and weaponry. Second was wowwiki.com, which serves as a repository for WoW lore, background and history. The third most frequently used site was elitistjerks.com, visited mainly for information regarding game strategy. These sites were reported to have been accessed in English by all respondents, both Dutch and American. This is worth mentioning since WoW-related websites and online communities exist in many languages. For example, wowwiki.com is available in 26 different languages, including Dutch, and wowhead.com is available in German, English, Spanish, French and Russian. Despite the possibility of looking up information in many different languages, all participants reported using only the English versions.

3.5 Quest texts: Are they important to read?

The ten self-selected WoW players who agreed to be interviewed as part of this project were asked to discuss the importance of reading quest texts and, in the sub-section below, how and when they used external websites. Changes to the WoW interface to accommodate what are sometimes referred to as ‘casual gamers’ include enhanced map features, enhanced tracking, and spatial location assistance. This has made determining the purpose and location of the quest elements less reliant on close reading of the quest texts themselves. However, as the interviewees report below, quest texts remain an omnipresent and important source of information in the game. Note that all names are pseudonyms created using the name generating function within WoW.

Gláwen:

Glakela:

3.6 Importance of external websites

The importance of external websites was underscored by the all interviewees, with each reporting that they used external websites before, during, or after the gaming sessions. For this reason, we propose that external websites are an integral part of the WoW gaming experience. This observation is borne out in the following transcribed portions of the interviews.

Toclya:

Moonpunisher:

Glakela:

Based on the interviews it became clear that using external websites was not only a preparatory or post hoc evaluative process, but also an integral part of the moment-by-moment gaming experience.

4.1 Method and rationale

A key use of basic research on text analysis is its application to the operationalization and measurement of linguistic complexity. Of course, experienced educators and MMO players know intuitively that some texts are more difficult than others, but the goal of this research is to accurately and objectively assess the complexity of texts that gamers most frequently engage with.

Linguistic complexity can be broadly defined as the range and sophistication of language forms and structures (e.g., Ortega, Reference Ortega2003). Precise measurement of linguistic complexity has proven challenging and numerous algorithms have been developed. In this research, we assess the readability, lexical sophistication, lexical diversity, and syntactic complexity of the texts using four indices that have been shown in previous research to be useful measures of linguistic complexity. As will become clear in the discussion, these measures come with certain limitations, particularly as a result of conflating sentential variability to mean scores, but each also provides a useful vantage point from which to analyze and describe complexity.

4.2 The linguistic complexity measures

A number of readability formulas are used for gauging or designing the grade level appropriateness of reading materials. Criticism has been levelled at the use of readability formulas in that they tend to focus on character, word and/or sentence length as a proxy for actual lexical or structural analysis (e.g., Davison & Kantor, Reference Davison and Kantor1982). However, we are including one readability measure here, the Coleman-Liau Index (CLI) (Coleman & Liau, Reference Coleman and Liau1975), as it is a valued representative of the many readability measures available. The CLI numeric output correlates with predicted grade level reading abilities based on the American educational system (e.g., a numeric score of 7.0 would indicate a 7^th grade reading level, appropriate for twelve-to- fourteen-year-olds). The formula to calculate CLI, in which C, W, and S stand for the total number of characters, words and sentences respectively, is: 5.89 * C/W – 30 * S/W−15.8.

To determine lexical sophistication (LS), the proportion of sophisticated words in a text is measured. For analysis of the WoW corpora, we adopt a frequency-based approach and consider a word sophisticated if it is not on the list of the 2,000 most frequent words in the American National Corpus (ANC; see Reppen & Ide, Reference Reppen and Ide2004). The specific measure used here is the ratio of sophisticated lexical word types, calculated as the number of sophisticated lexical word types divided by the number of lexical word types. Following Reference LuLu (in press), we define lexical words as nouns, adjectives, verbs (excluding modal and auxiliary verbs), and adverbs with an adjectival base.

The measure used for lexical diversity is the Mean Segmental Type-Token Ratio (MSTTR) (Johnson, Reference Johnson1944). Lexical diversity measures the range of vocabulary displayed in a text. Type-token ratio, which stands for the ratio of word types to word tokens, is a widely used measure for lexical diversity, but has been shown to decrease as the length of a text increases. The use of MSTTR corrects for this problem. MSTTR is calculated by dividing the text into segments of 50 words and then calculating the average TTR for each segment.

Syntactic complexity is here measured using the Developmental Level (D-Level) scale (Covington, He, Brown, Naçi & Brown, Reference Covington, He, Brown, Naçi and Brown2006; Lu, Reference Lu2009; Rosenberg & Abbeduto, Reference Rosenberg and Abbeduto1987). This measure uses sentences as a unit of analysis and then assigns individual sentences to one of eight developmental levels that ascend in complexity from 0 to 7. The D-Level scale was developed using child language acquisition research and has been positively evaluated in application to both written and spoken data (e.g., the CHILDES database, see MacWhinney, Reference MacWhinney2000). The D-Level scale aims to measure syntactic and structural complexity, and not merely proxy measures such as sentence length or word length. Examples of different D-Level sentence types that come from our text samples, accompanied by linguistic annotation, are presented below (see Table 6).

Table 1 Overview of the measures chosen to determine the linguistic complexity of the text samples under investigation

5.1 Description of corpora

Within the game itself, a primary source of game-generated language is in the form of quest texts. Quests guide players through the game and provide instructions for carrying out specific tasks. There are currently more than 9,500 quests in WoW and most involve finding resources, vanquishing foes, transporting messages or goods, and exploration. Quest texts are typically presented to a player by a non-player character (or NPC, which refers to non-human computer-generated in-game characters). Upon completion of a quest, the player receives rewards that include gold (the currency in WoW), useful items, reputation, and/or experience points.

To insure that the quest text samples are representative and likely to have been seen by most players, two separate corpora were assembled, one comprising quest chains and a second built from quests that are specific to one of the ten different classes (or character types) that can be played. The rationale for these choices is that quest chains are serially linked together and it is highly likely that all players will complete a number of quest chains during routine play, while class-specific quests involve gaining necessary skills for one's character and thus all players seek out such quests in order to progress in the game.

The quest chains were selected randomly, with one for every letter of the alphabet (based on the first non-article word in title). This resulted in 22 quest chains (with none for the letters J, Q, X, or Z) comprising 178 individual quests and totaling 18,538 words. For class-specific quests, twenty quests were collected per class, with the exception of the Death Knight class (a hero class that can only be played at higher levels), for which five were included. This resulted in 185 individual quests comprising a total of 18,318 words. Player level was taken into account. There were more class-specific quests at the lower levels and this resulted in a distribution of four quests for each of the following level bands: 0-5, 6-10, 11-30, 31-50, and 50-85. All of the quest texts were obtained from wowwiki.comFootnote ³. See Table 2 for an overview.

Table 2 Overview of the collection of quests

5.2 Results and discussion: Quest texts

The analysis of the two corpora of quest texts revealed that they are closely comparable for all measures used (see Table 3 for an overview of the linguistic analyses). This suggests that all the quest texts, regardless of being part of a chain or being class-specific, are similar in terms of readability and complexity.

Table 3 Overview of results of the linguistic complexity analysis of quest texts

The CLI readability measure suggests that both text samples are ranked as appropriate for 7th and 8th US grade level students, which is middle school level. This indicates that quest texts are suitable for children between the ages of twelve and fourteen. Contrasting somewhat with the readability measure, the LS, measuring the ratio of sophisticated lexical word types, was greater than .60 for both texts, which indicates that many of the lexical items appearing in the quest texts are lower-frequency and potentially more difficult or sophisticated words. The lexical diversity for both texts was also high, with the MSTTR analysis showing .78 (1.0 is the maximum score for this measure, which would mean that all words in all 50-word text segments were unique). This indicates that the quest texts show a high proportion of unique (non-repeated) word types per 50-word sample. Finally, the D-Level scale indicates a mean score of 4.4, which reveals the presence of complex syntactic structures that include a variety of conjoined, relative clause, and comparative constructions.

To sum up, the analysis of the quest texts revealed that for all measures used, these two corpora of texts are lexically and syntactically comparable. Furthermore, quest texts show a high degree of lexical sophistication, lexical diversity, and syntactic complexity. However, the readability level as measured by CLI was quite modest, a finding that contrasts with the lexical and syntactic complexity measures. For this reason, we looked more closely at the output of the D-Level analysis. As discussed above, this syntactic complexity measure assigns individual sentences to one of eight developmental levels, as described in Table 4.

Table 4 Convington et al.'s (2006) revised D-Level scale

The mean D-Level for all sentences in both text samples is 4.4. In addition to providing a mean score, the D-Level analyzer categorizes sentences and thus allows for an examination of the distribution of sentence types within a sample. This analysis shows a very interesting pattern, namely that there are high numbers of both the simplest and most complex sentence types. This data is presented in Table 5.

Table 5 D-Levels of the quest texts

Graphically representing the distribution of D-Level sentence types shows a right-skewed U-shape pattern, as depicted in Figure 4.

Fig. 4 D-Levels of the quest texts

As was illustrated in Table 5, the proportion of sentences for both the d-0 level and the d-7 level are high, showing a strong polarisation between simple and difficult sentences. This indicates that there is considerable variability in sentence complexity levels within the texts, with the most complex level of sentences (d-7) occurring with greatest frequency. This demonstrates that in terms of syntactic complexity, mean scores present a homogenizing view of the actual data as a result of internal variability. In Table 6, specific examples of the d-0 and d-7 sentences are provided. In the case of the d-7 sentences, a linguistic annotation indicates the specific constructions that are present.

Table 6 Examples of the most simple (d-0) and complex sentences (d-7) in the quest text samples

A qualitative examination of the d-7 sentences illustrates the lexical and syntactic complexity that confront players on a frequent basis. Triangulating among these various measures of readability, lexical sophistication, and syntactic complexity with the D-Level scale, we conclude that as far as the quest texts are concerned, WoW presents to players an environment that includes a substantial volume of highly complex linguistic input.

6.1 Description of corpora

Based on the results from the questionnaire and follow-up interviews, it became clear that several external sources were frequently read by almost all respondents. Text samples were collected from these sites in order to carry out linguistic complexity analyses on their contents. The sites included:

• • http://www.wowhead.com: Topics include strategy for completing quests and information on items in the game (e.g., armor, weapons, etc.).

• • http://www.wowwiki.com/Portal:Main: Lore and mythology of the WoW universe; information related to different classes (or types) of online character.

• • http://elitistjerks.com/: Game related strategy and general topics related to WoW.

As indicated by the questionnaire and interview data, wowhead.com is used by nearly all respondents to look up information about items in the game and for strategy to successfully complete quests. Two text samples were collected from this site, one focused on item information and the second on quest strategy. Wowwiki.com is used mainly to learn about WoW lore and information about the different player classes. Again, two text samples were assembled for each of these themes. Elitistjerks.com is used mainly for in-depth discussion of game strategy. The specifics of these corpora are described in Table 7.

Table 7 Overview of the collection of text samples from external sources

6.2 Results and discussion

Results of the linguistic complexity analysis of the most visited external resources are presented in Table 8. The readability measures demonstrate that there is a small difference between the wowhead items and quest text samples. The former shows a CLI of 6.7 and the latter a CLI of 7.6, suggesting that the texts are suitable for children between the ages of twelve and fourteen. The other samples are ranked as more complex and range between 9th to 11th US grade levels, which correlates with the readability level of young people between the ages of fourteen and seventeen. There is, however, a high degree of lexical sophistication with the ratio of sophisticated lexical word types greater than .60 for all texts, and a high degree of lexical diversity for all texts, with the MSTTR varying between .76 and .79. The D-Level analysis of syntactic complexity registered at the extreme high end of the scale, with mean scores greater than 6 for all corpora with the exception of the wowhead quest sample.

Table 8 Overview results linguistic complexity analysis of external websites

As was the case with the quest texts analysed earlier in section 5, the mean score readability outcome for the external sites runs contrary to the syntactic and linguistic complexity measure outcomes. However, further examination of the distribution of sentences across the D-Level categories shows that the game-external website texts are comprised of predominantly d-7 sentence types, with the single exception of wowhead's quest texts, which were characterized by a symmetrical distribution of d-0 and d-7 sentences (see Table 9).

Table 9 D-Levels of the external site text samples

Represented graphically, wowhead's items text sample shows a right-skewed peak at the d-7 level, whereas a symmetrical U-shape pattern is revealed for the wowhead quest text sample (see Figure 5).

Fig. 5 D-Levels of the text samples from wowhead

The strong polarisation in the quest text sample indicates internal variability in complexity level (see Table 10 for examples). It is relevant to note that the wowhead quest text corpus comprises player-to-player texts, written by gaming enthusiasts, which we will further discuss below.

Table 10 Examples of the most simple (d-0) and complex sentences (d-7) in wowhead's quest text sample

Figures 6 and 7 illustrate that the text samples from wowwiki and elitistjerk are similar to the wowhead items sample in that they show a right-skewed peak at the d-7 level.

Fig. 6 D-Levels of the text samples from wowwiki

Fig. 7 D-Levels of the text sample from elitistjerks

The right-skewed data in Figures 6 and 7 indicate that these texts are more uniformly complex in terms of the distribution of sentences. The wowhead quests text corpus (Figure 5), in contrast, is similar to the in-game quest text corpora (see Figure 4) as it shows a strong polarisation between simple and difficult sentences, with the most complex sentences (d-7) occurring at approximately the same frequency as very simple sentences, a fact that is not apparent when one only looks at the mean D-Level scores.

The difference in the distribution of complex sentences between the game-generated quest texts and wowhead quest text sample on the one hand (Figures 4 and 5), and the wowhead item text sample and the text samples from wowwiki and elitistjerks on the other (Figures 6 and 7), is probably due to differences in genre – that the former are interpersonally oriented texts (e.g., many sentences using second person address forms) while the latter are cases of expository or informational texts. Genre theorists Knapp and Watkins (Reference Knapp and Watkins2005) note that third person texts are stylistically more objective in tone while the use of personal pronouns tends to dialogically engage the reader (see also Kamberelis, Reference Kamberelis1999). In the case of the in-game quest texts, they are written as if the NPC and the player are talking to each other, and thus are conversational in genre. Similarly, the player-authored wowhead quest text strategy commentaries are posted on an interactive discussion forum and are written as dialogical discourse with expectations for response and debate. Additionally, these texts show high levels of phatic communication, indicating that interpersonally oriented and community building communication is interwoven with the conveyance of information. The presence of clause level interactive discourse features is visible in the d-0 and d-7 sentence types shown in Table 9. Corresponding examples of dialogically oriented discourse from the game-generated quest corpora and the player authored wowhead quest strategy forums are displayed in Table 11.

Table 11 Examples of the conversational simple sentences in the interpersonal text samples