1 Introduction
Doing research, whether formal or informal, primary or secondary, has the potential to positively influence teaching practice and learning outcomes. Classroom research helps bridge the gap between theory and practice as researchers try to make sense of what is going on in different teaching contexts in order to inform new practices. But what research approaches have been used to investigate classroom-based interaction in both physical and virtual settings, particularly in the context of language teaching and learning in and around virtual worlds? What are the benefits and limitations of these approaches? The purpose of this article is to survey the types of classroom-based research approaches used in physical and virtual contexts to guide research in three-dimensional virtual learning environments (3DVLEs).Footnote 1 Of interest is investigating how interaction occurs among users, and between users and the virtual space. This focus is motivated by personal experiences with my own learners and the desire to systematically document what was going on as students were observed interacting with and in these 3DVLEs. These early experiences prompted me to survey research methods, not necessarily findings, in second language acquisition (SLA) and computer-assisted language learning (CALL) literature that could help in research design. The ever-growing popularity of 3DVLE technologies for language learning and teaching offers an exciting area for research opportunities across multiple domains, including CALL and second language methodology; thus, exploring effective research design is timely and important. This article explores methodologies used in classroom-based interaction studies to understand which methodologies may best apply to research in 3DVLEs. As empirical research in 3DVLEs develops, this article includes an overview of research approaches and methods in the area of second language classroom interaction in both physical and virtual contexts to inform research practice.
To begin, I provide a description of both the physical and virtual classroom context and classify the types of interaction more commonly found in second language studies and as highlighted in Gass and Selinker (Reference Gass and Selinker2008). This is followed by a historical overview of research approaches and methods. Next, I briefly survey certain studies in the role of physical classroom settings in classroom interaction, followed by studies in virtual classroom settings. The purpose of this literature review is to identify common research approaches, methods, analytical tools, data collection techniques and types, and their benefits and limitations in physical language classroom interaction studies in SLA and to compare them to those approaches used in CALL 3DVLE studies.
2 Definitions
This section defines the classroom context and specifies the types of classroom interaction typically investigated in second language learning and classroom interaction studies.
2.1 Classroom context defined
Research in the area of language teaching and learning and classroom interaction typically investigates what contributes to successful learning outcomes; however, many other variables can be the focus of classroom interaction research, such as the beliefs, attitudes, and personalities of participants, or the social contexts in which the research takes place (Brown & Rodgers, Reference Brown and Rodgers2002; Chaudron, Reference Chaudron1988). The article considers the online and virtual classroom space as a newly defined social context in which learning and, subsequently, research take place.
In his book Second Language Classrooms: Research on Teaching and Learning, Chaudron (Reference Chaudron1988) addresses the complexity and breadth of the second language learning context. Broadly, he identifies the foreign language (FL) and the second language (L2) context as variables. He describes the FL context as one in which the learner is assumed to have little exposure to the target language outside the classroom, compared to the L2 context in which the target language is the content and medium of instruction. Chaudron does not specify in which context an online or virtual classroom best fits; however, based on his explanation of FL and L2 contexts, one might infer that an online context is a context variable. Although these context variables are necessary as measurable variables, as described, they fail to explain what’s going on in the classroom context, such as behavior. These behaviors, such as interaction, are what Chaudron describes as process variables and are important elements in qualitative research approaches.
As such, based on the above description, the classroom is a context in which interaction is a process that can be observed qualitatively, and learning is an outcome or product that can be measured quantitatively. Yet this interpretation may not accurately depict an understanding of the online or virtual classroom as a necessary component of the process variable and not simply the context variable, as I experienced with my own students in a 3DVLE. For instance, Chaudron (Reference Chaudron1988) writes that the process variable includes observable behaviors of both the teacher and the learner; this description does not account for the different kinds of interactions that occur in a virtual space compared to a physical space (Hartwick, Reference Hartwick2015). Chaudron’s description of process variables also fails to account for the multidimensional, fluid interactions that often occur in 3DVLEs or other online spaces and which are rich sources of data for qualitative, quantitative, and mixed-method approaches. Thus, although conversations and instructional interactions in a target language of instruction are commonly researched qualitatively, it remains unclear in such studies what role interaction with space plays in influencing learning outcomes. Consequently, understanding space as both a context variable and a process variable has the potential to enrich research, because these can be explored qualitatively and/or quantitatively. This understanding necessarily frames my rationale and perspective throughout the rest of this article.
2.2 Interaction defined
The significant role of classroom interaction is well known in SLA and development studies and has prompted research focused on constructs such as negotiation of meaning, comprehensible input, recasts, repetition, instruction, and feedback (Brown & Rodgers, Reference Brown and Rodgers2002; Chaudron, Reference Chaudron1988; Ellis, Reference Ellis1988; Gass & Selinker, Reference Gass and Selinker2008; Hall & Verplaetse, Reference Hall and Verplaetse2000; Long, Reference Long1996). Gass and Selinker describe the interaction approach to classroom research as “learning through input (exposure to language), production of language (output), and feedback that comes as a result of interaction” (2008: 317). Similarly, Ellis (Reference Ellis1988) suggests that language classrooms must facilitate a need to communicate while simultaneously providing opportunities for comprehensible input and practice. At the same time, exposure, feedback, and interaction contribute to SLA, and SLA research considers the many constructs of interaction, which can be clearly delineated according to categories of classroom interaction, such as teacher–learner, learner–learner, and learner–text (Brown & Rodgers, Reference Brown and Rodgers2002). In addition to these categories and based on an unpublished exploratory study I conducted, I propose a new category, learner–space, stemming from the observed affordances of virtual worlds. This category of interaction considers how teachers and learners and/or participants interact with each other and the learning context, space, or objects in the space in which they are engaged.
3 Historical overview of research approaches
The practice of observing teachers’ and learners’ behavior is a long-standing method for collecting data and analyzing constructs in classroom-based studies (Brown & Rodgers, Reference Brown and Rodgers2002; Chaudron, Reference Chaudron1988). This practice is suitable for qualitative studies wherein the researcher is very often an active participant in the research process (Creswell, Reference Creswell1998). Yet, as research perspectives and trends in CALL (Peterson, Reference Peterson2006) and second and foreign language teaching and learning (Long, Reference Long2015) continue to develop, research design needs to be clearly defined according to a specific tradition, perspective, and context (Brown & Rodgers, Reference Brown and Rodgers2002). CALL researchers advocate for continued exploration of methods in computer-mediated contexts in order to truly understand the complexity of these innovative spaces for language learning (Blake, Reference Blake2008; Chapelle, Reference Chapelle2000). Ultimately, as stated by Chaudron, “the fundamental goal of most such research has, of course, been to determine which variables best, or more frequently, lead to academic achievement” (Chaudron, Reference Chaudron1988: 1). Consequently, selecting the best research approach, method, and tools should be considered with respect to research purpose, research questions, and context, along with a good understanding of potential limitations. A historical overview of similar research processes and contexts is a first step.
In the last 50 years, research perspectives in the domain of SLA have shifted from mostly quantitative, behaviorist, and product-only studies to more of an appreciation for the role of communication and interaction – more of a focus on process has meant that research became more qualitative in nature (Antón, Reference Antón2015; Chaudron, Reference Chaudron1988; Long, Reference Long2015). Understandably, this varies according to the theoretical perspective of the researcher, as their perspective, or frames of reference, determines how data are to be classified, coded, and subsequently understood (Green, Castanheira, Skukauskaite & Hammond, Reference Green, Castanheira, Skukauskaite and Hammond2015). Similarly, Gass and Selinker (Reference Gass and Selinker2008) note how learning theories help to contextualize the frame of reference.
In Chapter 4 of The Handbook of Classroom Discourse and Interaction, Long (Reference Long2015) provides a historical overview of language classroom interaction studies from the early 1960s to the present. He identifies four sequential and overlapping phases of research. Early studies in the first phase were comparative and quantitative and focused on the product or outcome as opposed to analyzing teaching and learning processes. Conversely, the second phase of studies in the 1970s and 1980s were more descriptive as researchers focused on teaching and learning observations – a qualitative trend that continues today (Antón, Reference Antón2015; Long, Reference Long2015). According to Long, descriptive studies, although data rich, can be time consuming (e.g. transcribing data such as classroom talk) and overwhelming in terms of segmenting and coding data. According to Long, the problem with this second phase was that this research lacked both focused and contextualized research objectives as well as clearly articulated and anticipated learning outcomes. This prompted a third phase, which Long refers to as “process–process, descriptive and correlational” (2015: 63). In this phase, research tended to focus on how the teacher’s behavior directly correlates with the quality and quantity of student learning. This gave way to the fourth and most recent phase, which Long considers to be focused on the impact of variables such as language use on learning outcomes. Studies in this so-called “process, product, quasi-experimental and experimental” phase (Long, Reference Long2015: 64) tend to have fewer participants, be shorter in duration, and have a more limited scope; however, they can also be more easily duplicated and generalizations can be made based on multiple studies. Further, these types of studies allow for greater control of variables, such as context, learner, and type of interaction. This latter phase shares the characteristics of mixed-method research as defined by Creswell and Plano-Clark (Reference Creswell and Plano Clark2011).
These historical approaches to SLA research focus mainly on types of interaction and the discourse that evolved from these interactions, arguing that L2 acquisition was a direct result of these interactions (Hall & Verplaetse, Reference Hall and Verplaetse2000). However, these approaches may not adequately account for interactions in non-traditional classrooms spaces, such as 3DVLEs. This shortcoming is an important consideration for current research studies, as an understanding of interactions in an online space might help to explain how or, in fact, if this newly defined context for interaction is related to SLA. Antón explains how more contemporary social approaches to language learning have motivated newer trends in research, including dynamic assessment, which is “… inherently interactive” (2015: 79). This trend focuses on classroom-based, formative assessment and includes observation of collaborative interaction and learner behavior. However, it still does not adequately account for interaction within what Phillips, McNaught and Kennedy (Reference Phillips, McNaught and Kennedy2012) describe as an interactive learning system and the potential impact on SLA.
Empirical research in CALL and 3DVLEs is an exciting new research domain that has been met with both skepticism and praise. In 2000, Kern and Warschauer considered computer-mediated communication tools, like 3DVLEs, one of the newest areas in CALL research reported to promote social and engaging learner interactions (Blake, Reference Blake2008; Chapelle, Reference Chapelle2000). Over a decade later, Peterson (Reference Peterson2012a) commented on how 3DVLEs are pushing the boundaries of contemporary language education, as digital learning platforms drive new CALL research. Similarly, Reinhardt and Sykes (Reference Reinhardt and Sykes2012) point out the transdisciplinary nature of this research area, which depends on a clear description of the intended outcomes, pedagogy, and methodology. Owing to the affordances of these spaces in language learning contexts, research in this area of CALL is growing exponentially and investigates a myriad of topics, including 21st century competencies, teacher education, and promotion of language fluency (Dooly, Reference Dooly2015).
The next sections survey classroom interaction studies in both physical and online classroom contexts. The purpose is to uncover the various research methods, tools, and analytical processes used in understanding physical and virtual classroom interaction and SLA.
3.1 Researching interaction in second language studies in the physical classroom
Simply, classroom-based research traditionally investigates the ways in which students and teachers interact to hypothesize what contributes to successful language learning. Due to the enormity of empirical research in second language classroom interaction, studies referenced herein were selected based on studies cited or understanding garnered in Chaudron (Reference Chaudron1988), Markee (Reference Markee2015), Long (Reference Long2015), Gass and Selinker (Reference Gass and Selinker2008), and Brown and Rodgers (Reference Brown and Rodgers2002). These five were the primary sources for this article – further review is outside the current scope.
Research methods, tools, and analytical processes used in classroom interaction SLA research vary tremendously. A brief survey of research studies about second language classroom interaction reveals that research approaches range from experimental and quasi-experimental (Spada & Tomita, Reference Spada and Tomita2010), to naturalisticFootnote 2 (Bailey & Nunan, Reference Bailey and Nunan1996), to case studies and action research (Tsui, Reference Tsui1996), to simulated classroom data and discourse analysis (Ellis, Reference Ellis1988), to stimulated recall (Gass & Mackey, Reference Gass and Mackey2000; Nunan, Reference Nunan1996). These approaches vary from purely qualitative or quantitative to mixed-method studies. Yet Chaudron (Reference Chaudron1988) claims that the research is not about a quantitative “versus” qualitative paradigm, but rather about finding complementary approaches to scientific inquiry. For example, studies investigating learning outcomes, correlation, and inference are typically quantitative, whereas descriptive and observational style studies are often more qualitative in nature (Chaudron, Reference Chaudron1988).
Regardless of approach, defining a research process is what is important, including specifying conventions such as research tools and data analysis techniques. Although not an exhaustive list, the research tools, techniques, or data sources in classroom interaction studies include surveys and questionnaires, transcripts, pre- and post-tests, participant interviews, observations, and field notes (Bailey & Nunan, Reference Bailey and Nunan1996; Brown & Rodgers, Reference Brown and Rodgers2002). The breadth, variety, and range of data collection techniques help determine the approach, and quantitative measurements necessitate clearly defined units of analysis, such as grammatical (word level) or interactional (turn-taking) features. Chaudron suggests that the “unit of analysis is a crucial aspect of observational instruments, in that the specification of a period of time, or of an analytical linguistic or pedagogical unit, involves basic assumptions about the nature of the classroom interaction” (1988: 20).
Units of analysis in quantitative classroom interaction studies include, but are not limited to, language-related episodes (LREs), time units, private speech, and moves (Chaudron, Reference Chaudron1988; Ohta, Reference Ohta2000; Swain & Lapkin, Reference Swain and Lapkin1998). Swain and Lapkin (Reference Swain and Lapkin1998) describe LREs as instances where learners use language to talk about or reflect upon their own or others’ language use, hypothesizing that these episodes of interaction will lead to greater SLA. They further classify the LREs according to lexical items (vocabulary) and form (structure) at the discourse level. Another study by Kim (Reference Kim2013) uses LREs as a unit of analysis to understand the importance of task design factors, such as sequencing and task repetition. Discourse used in analysis depends on units such as the utterance, turn, or function, quotes from dialogue, features of language, repetitions, type of feedback, teacher talk, and types of strategies (Chaudron, Reference Chaudron1988; Hall & Verplaeste, Reference Hall and Verplaetse2000).
Whereas Tsui (Reference Tsui1996) transcribed interactional extracts to improve teaching strategies, Ohta’s (Reference Ohta2000) qualitative study analyzed the private speech utterances as a unit of analysis. In this discourse-analytic approach, Ohta studied the effect of corrective feedback based on student to teacher interaction in the form of recasts. Like Tsui, she used audio and video recordings to collect and later transcribe data; however, data were analyzed through discourse analysis as opposed to teachers’ interpretations of selected extracts of interaction. Further, and equally important to most interaction studies, is that “what is ultimately salient, or what ultimately becomes the focus of attention, is likely to differ significantly from learner to learner” (Ohta, Reference Ohta2000: 67).
As noted earlier, although classroom observation is a common method for collecting rich data about types of interaction, transcription is time consuming and results are hard to compare due to the range and type of instrumentation or tools used in studies of classroom interaction (Brown & Rodgers, Reference Brown and Rodgers2002; Markee, Reference Markee2015). As such, researchers should consider how they will proceed with data analysis.
In addition to observation, other instruments used in second language classroom-based research include journals, field notes, interviews, questionnaires, checklists, etc. (Brown & Rodgers, Reference Brown and Rodgers2002). These are often combined with data results from observational data. Moreover, to understand the impact of interaction on learning outcomes, some classroom interaction studies employ pre- and post-tests to quantitatively measure change in performance. For example, to show the importance of language and interaction linguistically and cognitively, and in relation to task completion and learner proficiency measurements, Swain and Lapkin (Reference Swain and Lapkin1998) administer pre- and post-tests in addition to videotaping and transcribing specific language features. This process points to the complexity, challenges, and limitations of classroom-based interaction studies of which the researcher must be fully aware.
As reported by Chaudron (Reference Chaudron1988), other methodological limitations are sample size, inability to generalize, laboratory versus classroom contexts, and variance in factors like teacher behavior, learner behavior and characteristics, and types of interaction, or unreliable and incomplete measures of learner development. Further confusion stems from conflicting procedures, definitions, and analytical constructs or theoretical perspectives (Ellis, Reference Ellis1999). Ellis also lists some of the limitations of research with respect to interaction and second language learning, including a tendency to measure explicit knowledge of language in use rather than demonstrated acquisition. Interestingly, he further suggests that research fails to examine the “… kind of learning (implicit or explicit)” (1999: 237) that is a result of opportunities created to interact in L2. This is relevant to research in virtual worlds if we consider that 3DVLEs may provide an opportunity to interact in ways that otherwise might not have been possible.
3.2 Researching interaction in second language studies in the 3DVLE classroom
As highlighted in the previous section, traditional language classroom interaction studies analyzed the oral language data derived from learner or teacher production or output resulting from the process of negotiating, typically to complete a task. Frequently, language was analyzed according to specific constructs, features of the language, or functions. This section surveys research approaches, tools, and data analysis techniques used in 3DVLE interaction studies.
In addition to the studies already discussed, six additional studies were considered (Collentine & Collentine, Reference Collentine and Collentine2015; Jauregi, Canto, de Graaff, Koenraad & Moonen, Reference Jauregi, Canto, de Graaff, Koenraad and Moonen2011; Liou, Reference Liou2012; Mroz, Reference Mroz2015; Peterson, Reference Peterson2010, Reference Peterson2012a). These six studies were selected according to the following process. First, the keywords interaction and 3D and language were used to search abstracts in Scholars Portal and Academic OneFile library repositories. The search was limited to articles published after January 1, 2010, and to these journals: Computer Assisted Language Learning (CALL), ReCALL (the journal of the European Association for Computer Assisted Language Learning), and CALICO (Computer Assisted Language Instruction Consortium) Journal. A total of 16 studies were found, two from each of CALL and ReCALL, and 12 studies in CALICO. As it was not possible to restrict the keyword search in CALICO to the abstract, all 12 abstracts and introductions were scanned using the same keywords, resulting in a total of two articles. Although this process has certain limitations, the selections are motivated to better understand 3DVLE interaction research design, especially pertaining to methods and analytical procedures that may lead to a better understanding of the role of space in 3DVLEs in relation to language learning.
Empirical research studies related to 3DVLEs and specific to interaction as a feature of language learning are emerging; however, research pertaining to 3DVLEs in CALL has been criticized as being largely under-theorized, anecdotal, exploratory, and descriptive (Peterson, Reference Peterson2012b; Twining, Reference Twining2010). The survey of research approaches for this section includes empirical studies in motivation, vocabulary development, collaboration, and task design (Berns, Gonzalez-Pardo & Camacho Reference Berns, Gonzalez-Pardo and Camacho2013; Chung, Reference Chung2012; Ibáñez, Garcia, Galan, Maroto, Morillo & Kloos, Reference Ibáñez, García, Galán, Maroto, Morillo and Kloos2011; Milton, Jonsen, Hirst & Lindenburn, Reference Milton, Jonsen, Hirst and Lindenburn2012; Peterson, Reference Peterson2006, Reference Peterson2012a; Zhang, Reference Zhang2012), as well as the six interaction studies identified above. Case study design is commonly reported in these contexts (Berns et al., Reference Berns, Gonzalez-Pardo and Camacho2013; Jauregi et al., Reference Jauregi, Canto, de Graaff, Koenraad and Moonen2011; Liou, Reference Liou2012; Peterson, Reference Peterson2010; Zhang, Reference Zhang2012), and Second Life is the most featured platform used in approximately 60 percent of studies (Reisoğlu, Topu, Yılmaz, Yılmaz & Göktaş, Reference Reisoğlu, Topu, Yılmaz, Yılmaz and Göktaş2017). Peterson’s (Reference Peterson2012b) meta-analysis of three learner-based studies was useful in that it investigates three constructs of interaction, including negotiation of meaning, strategies used during task-based interaction, and participation patterns during voice chat.
Whereas physical classroom interaction studies often focus on language output during interactions in terms of features and function, earlier studies related to language learning in 3DVLEs often hypothesized that the affordances of the space are what contributed to language acquisition. However, these claims generally assumed language acquisition was achieved because of interaction with native and non-native speakers and meaningful engagement with a task (Molka-Danielsen, Mundy, Hadjistassou & Stefanelli, Reference Molka-Danielsen, Mundy, Hadjistassou and Stefanelli2012; Peterson, Reference Peterson2012b). Earlier studies suffered from low number of participants. For instance, Carter and Elseth’s (Reference Carter and Elseth2009) descriptive and qualitative study focused on the vocabulary development of only three beginner learners of German engaged in simulated field trips in Second Life. Similarly, Zhang’s (Reference Zhang2012) exploratory study, which investigated the barriers participants faced while attending to learning activities in Second Life, was limited to 10 participants. However, in a meta-analysis of studies in 3D virtual worlds, Reisoğlu et al. (Reference Reisoğlu, Topu, Yılmaz, Yılmaz and Göktaş2017) report a growing trend in sample sizes. In 167 studies analyzed, 32 employed between 51 and 100 participants. These authors speculate that low participation rates contributed to higher incidences of case study in past 3DVLE research.
In addition to low participant numbers, studies were often limited in terms of duration. For example, Peterson’s (Reference Peterson2006) study investigated interaction management patterns of 24 EFL students across three separate tasks. This two-phase study took place over a five-week period, but was limited to only three one-hour sessions, making results tentative and largely ungeneralizable. For this study, Peterson collected and coded text chats according to 11 specific discourse management strategies (e.g. requests for clarification). He then used an online concordancer to search for instances of the coded strategies. In lieu of a pre-test, participants were selected according to a pre-specified TOEFL score. Field notes and pre- and post-questionnaires were also used; however, no quantitative measurement was applied. A recent study by Liou (Reference Liou2012) examined the extent to which affordances of Second Life are perceived by students as beneficial in terms of language learning. This case study of 25 participants was longer and spanned 18 weeks.
Methodologically, research around 3DVLEs has since expanded to employ mixed-methods approaches, resulting in more robust discussions. Although the studies surveyed above were largely qualitative and descriptive, Chung (Reference Chung2012), Berns et al. (Reference Berns, Gonzalez-Pardo and Camacho2013), Lan (Reference Lan2015), and Mroz (Reference Mroz2015) used mixed methods in their research design. Specifically, Berns et al.’s (Reference Berns, Gonzalez-Pardo and Camacho2013) case study was designed to investigate the role of game-like scenarios in vocabulary development while using a 3D platform called VirtUAM. This mixed-method study used several data collection instruments, including a questionnaire, and pre- and post-test results. The questionnaire was used to qualitatively assess students’ perceptions of learning and levels of motivation, while researchers used a Wilcoxon test (Berns et al., Reference Berns, Gonzalez-Pardo and Camacho2013) to analyze test scores to measure the impact of selected games on learning. In much the same way, Chung’s (Reference Chung2012) quasi-experimental study compared achievement outcomes in vocabulary, grammar, and reading achievement. Control group participants were exposed to the same teaching materials in a traditional classroom as their experimental group counterparts, who participated in Second Life with the added interaction of virtual characters. Test scores and questionnaire results were analyzed and pointed at a positive correlation between the learning context and levels of motivation, and the context and test scores. Finally, Lan’s (Reference Lan2015) study succeeded in observing the behavior of 132 primary school students in Taiwan in a physical and virtual teaching context. Her comparison of teaching spaces helped validate the affordance of space in promoting conversational skills. This larger scale study demonstrates the importance of observing participants’ learning behaviors in addition to using pre- and post-tests.
Further, 3D research is beginning to investigate the role of the online space through recorded observations. Deray and Simoff (Reference Deray and Simoff2009) investigate ways in which interactions unfold in real time during the learning process of 3D design students to representing interactions visually to provide feedback. While the participants are not language learners, the study is useful for 3DVLE research in that it seeks to measure interaction processes by recognizing that interaction goes beyond the types described in Section 2.2 and which are common to traditional classroom-based interaction studies. Hence, although Deray and Simoff recognize the importance of interaction between learners, they also understand the role of the environment as an alternate channel of input.
Considering the motivation for this article was to understand how interactional behavior in and with 3DVLEs was captured, documented, and analyzed qualitatively and/or quantitatively, Jauregi et al.’s (Reference Jauregi, Canto, de Graaff, Koenraad and Moonen2011) case study was particularly relevant. This study used recorded observations to analyze how task design and affordances contributed to authentic social and intercultural interactions in Second Life compared to affordances of a 2D environment. Researchers used the recorded data from Second Life interactions to understand the type of interactions elicited by the different tasks and to see if tasks adequately made use of the affordances of space. Mroz’s (Reference Mroz2015) mixed-method study also used video recordings of interactions and student logs as secondary data to triangulate primary data sets of student chat logs in Second Life. These recordings helped researchers interpret interactions such as turn-taking.
To account for limitations of this review, I include relevant results from a meta-analysis of 167 studies intended to identify design and research trends in 3DVLEs (Reisoğlu et al., Reference Reisoğlu, Topu, Yılmaz, Yılmaz and Göktaş2017). This analysis, based on studies found in two databases considered pioneers in educational technology, used keyword searches and word combinations relating to 3D virtual worlds. The researchers’ analysis reveals the following trends to be considered in future research design: descriptive research tends to be more popular than experimental; research contexts are largely in Second Life; research objectives include learning support, simulation, and social interaction; environments are generally designed to support collaborative, explorative, and task-based learning; language learning topics are the most heavily researched in 3DVLEs, followed by science and health; and case study, quasi-experimental, descriptive, and mixed method are the most commonly reported methods.
Much like physical classroom contexts, surveys, questionnaires, field notes, and recorded observations continue to be used for data collection in these online spaces, yet newer techniques have evolved. One such technique is called dataveillance,Footnote 3 which can quantitatively measure and track a user’s actions. Although Ross, Castronova and Wagner (Reference Ross, Castronova and Wagner2012) define this term in the context of game-based scenarios, it could be extended to studies in 3DVLEs. However, the problems associated with this technique are “… managing the sheer quantity of data captured and the need to develop an elastic yet rigorous structure in which to organize and analyze it” (2012: 285). Ross et al. comment on the importance of identifying categories and setting parameters in advance of data collection to avoid the onerous task of post-hoc data organization. This process can also be time consuming; learning how and which data to mine is necessary for researchers wishing to take advantage of this rich data source. One way to mitigate these challenges is by using an observation matrix,Footnote 4 as in Table 1, to better understand the categories and to facilitate managing the sheer quantity and breadth of data. The next section considers possible methodological approaches, data collection techniques, and units of analysis as previously discussed.
Table 1 Observation matrix
4 Implications for research design in virtual worlds
The process of researching for this article has helped illuminate an important trajectory for future research agendas while stressing the continued need for research in these new learning contexts. This includes an understanding of the implications of the features of space in addition to the features or constructs of language. Indeed, future research in this area may consider both to be important.
As previously noted, earlier research in CALL and specifically 3DVLEs tended to be highly descriptive and qualitative and lacking in experimental studies (Genc-Ersoy & Ersoy, Reference Genc-Ersoy and Ersoy2013). Peterson remarked on how “the new forms of interaction made possible by virtual worlds remain, to a significant degree, unexplored” (2012b: 78), but the potential for exploratory research in this area is exciting. As noted by Ross et al., “the use of virtual worlds as experimental environments, or even as platforms for observation, coupled with dataveillance and survey tools has such powerful implications” (2012: 307). In an earlier article, Chapelle (Reference Chapelle1998) warns researchers of the strong data collection capacity of CALL platforms like 3DVLEs at the research design stage, and states to proceed with caution as the process for describing and interpreting data is still in development. Similarly, although 3DVLEs are potentially rich learning contexts, the challenge is knowing how and what to analyze regarding the interaction process in these ever-changing spaces (Deray & Simoff, Reference Deray and Simoff2009).
Understanding whether and how 3DVLEs as classroom contexts promote and facilitate interaction in a second language assumes that social interaction leads to developing language proficiency (Ellis, Reference Ellis1999). Further, it assumes that learning is a result of engagement and experience in an environment or context (Twining, Reference Twining2010). As such, observing learner interaction, as mediated by the task, with the space itself, the tools and objects in the space, and with other users in the space is likely important. The role and type of the task is important as these help determine how learners engage with the environment; the learners’ behavior as a consequence of the task becomes the learning process that could be observed qualitatively (Phillips et al., Reference Phillips, McNaught and Kennedy2012).
User interactions in these new contexts are perhaps best observed through screen-captured and recorded observations as an important component of the research design. These recorded observations capture participant behavior within and because of the space, although screen-capture and recording software should be rigorously tested to avoid faulty recordings. Additionally, recordings should ideally be done from multiple views in the space and in shorter segments to avoid missed perspectives or views and uploading complications. This is especially important, as observations ideally should capture the behavior of all simultaneous user actions, regardless of the location in the space. Further, multiple views will provide rich data in which analysis can investigate interaction in terms of features of the space, such as location and tools used, as opposed to just the features, constructs, and functions of spoken language analyzed from text-chat transcripts (Garrison, Anderson & Archer, Reference Garrison, Anderson and Archer1999). Thus, depending on the research problem, in terms of language acquisition the features of space may be a dependent variable in a mixed-method design.
These observed interactions with the space could be first analyzed qualitatively according to an observation matrix, such as the model in Table 1. Next, these results can be further enriched qualitatively through observer field notes and participant questionnaire results in order to understand participant perspectives. Similarly, with the development and improvement of data or engagement analytics built into many of the 3DVLE platforms, quantitative data might be readily available in terms of measuring frequency and duration of interactions according to the same categories established on the matrix. These categories may include actual location in space or use of a specific tool, like collaboration surface or teleporting function. This process could help triangulate data with observation matrix categories and better clarify the role of 3DVLE spaces in promoting user interaction.
Regarding a designated unit of analysis for future 3DVLE studies, the use of non-traditional units of analysis in interaction studies, such as location in space, might contribute to a better understanding of SLA in these contexts. In her research, Nocchi (Reference Nocchi2017) classifies the actual language task as a unit of analysis, whereas Mroz (Reference Mroz2015) identifies units of meaning realized in analysis of conversation. This decision depends on the exact focus of study as it develops during the design phase. However, based on earlier exploratory research (Hartwick, Reference Hartwick2015), units of time, such as time spent interacting in a specific location or with a specific object in the space, might be a logical choice.
It is difficult to label the exact research approach. Whereas Bailey and Nunan suggest that a naturalistic-ecological perspective for analyzing behavior is best and state that “… as its central tenet, the belief that the context in which the behaviour occurs has a significant influence on that behaviour” (1996: 2), identifying a specific research approach for 3DVLE research remains challenging. As the study of interaction in virtual worlds is clearly multifaceted, perhaps adopting a more pragmatic approach is reasonable. As a world view, pragmatism is an approach to inquiry that believes in doing what works best for the given context (Creswell & Plano Clark, Reference Creswell and Plano Clark2011; Teddlie & Tashakkori, Reference Teddlie and Tashakkori2010). Characteristically, pragmatism views the role of experience, agent or organism, environment, and interaction as highly important; further, this view recognizes the benefit of drawing on multiple theoretical lenses to best account for what is going on (Johnson & Onwuegbuzie, Reference Johnson and Onwuegbuzie2004). Pragmatic research is just that: a pragmatic, sensible, workable approach to inquiry. A pragmatic, mixed-method approach could help to triangulate multiple data sources to better interpret the research phenomenon, specifically by combining the strengths and offsetting the weaknesses of both qualitative and quantitative methods, providing more and more diverse forms of evidence, and helping to address more complex research questions that could not have been addressed as effectively by one method alone (Creswell, Reference Creswell2015; Creswell & Plano-Clark, Reference Creswell and Plano Clark2011; Dörnyei, Reference Dörnyei2007). Further, mixed-method approaches may enhance the researcher’s understanding of a relatively new phenomenon, as in virtual worlds, and strengthen their theoretical stance or understanding of a concept.
Regardless, conceptualizing the design framework or agenda is an important first step for virtual world research. Creswell (Reference Creswell2015) suggests a convergent design in which both qualitative and quantitative data are analyzed purposefully with the intention of comparing results. Accordingly, mixed-method design should begin with a specified theoretical understanding of language learning and learning in general. Further, an analytical framework, such as the learning environment, learning processes and learning outcomes (LEPO) framework proposed by Phillips et al. (Reference Phillips, McNaught and Kennedy2012) could help to capture the virtual environment, 3D or otherwise, as a necessary component of learning.
5 Conclusion
Although this review of literature is based on only a sample selection of studies, it might help demonstrate the changing landscape of research methodologies in 3DVLEs and CALL. In future studies, researchers should consider broadening the search criteria to include terms like virtual and include a range of databases and conference proceedings from a breadth of locations. It is an interesting and fulfilling time to be involved in research about 3DVLEs as an innovative digital and online learning space. There are many unanswered questions and different research methods and approaches that need applying. The use of analytics in combination with observational data in future studies could help clarify the role of space through the lens of different learning theories. Research should appreciate and understand the role of the environment as a mediator in the learning process and use 3DVLE analytics to measure duration and frequency of interaction to understand the impact of space on performance. Similarly, new studies might continue to include elements such as requests for turn-taking and impromptu dialogues as valuable data. Further analyses could consider the function of language and the specific dialogue generated solely for these purposes; in this case, discourse analysis might be a worthwhile approach. Ultimately, these studies have simply scratched the surface for this new area and serve as important prototypes for ongoing research.
About the author
Peggy Hartwick is an instructor in the School of Linguistics and Language Studies and a PhD candidate at Carleton University, Ottawa, Canada. As a recipient of the 2015 Brightspace Innovation Award, she is dedicated to teaching and learning. Her research includes the use of 3D virtual learning environments and ePortfolios in language teaching.
