1 Introduction and background research
More than a decade ago, Hubbard (Reference Hubbard2002), a pioneering theorist, researcher and practitioner in the field of Computer-Assisted Language Learning (CALL), described a gap in the research on language learning games, stating that “the majority of [previous] research focused on demonstrating the validity of the general [game-based] approach rather than specific elements of its implementation (op. cit.: 211).” With the exception of a handful of recent studies that modified constituents of game-based learning environments (e.g., deHaan, Reed, & Kuwada, Reference deHaan, Reed and Kuwada2010; Ranalli, Reference Ranalli2008), the need for careful attention to implementation still exists today. One “specific element of implementation” that deserves greater attention is feedback, which is widely recognized as crucial both for linguistic development and as a core feature of game mechanics.
The game-based learning (GBL) literature identifies feedback as an element that is both central to games and indispensable for learning (e.g., Aldrich, Reference Aldrich2005; Becker, Reference Becker2007; Prensky, Reference Prensky2001). In commercial games, feedback is considered to give players, who engage in a series of goal-directed activities, a measure of how well they are progressing towards goals. GBL theory proposes that it is essentially “from the feedback in a game that learning takes place” (Prensky, Reference Prensky2001: 121). While Prensky's statement suggests that feedback applies to both games that are intended to entertain as well as to educate, it is obvious that for the latter, developmentally useful feedback is all the more essential.
This paper will first conceptualize feedback in CALL games by interweaving theory in the second-language acquisition (SLA) and GBL literatures. It will then argue that individual difference factors, in particular learners’ self-perceptions, need to be taken into account in the design of CALL gaming environments. Finally, it will present results from an empirical study that aimed to chart the relation between learners’ perceptions of feedback and their self-perception in an immersive game for English pragmatics.
1.1 Conceptualisation of feedback in GBL and SLA
In the literature on SLA, the kind of feedback that is directed towards learning is generally known as “corrective feedback” (CF) or “negative feedback” (Long, Reference Long2007). CF refers to all responses to learners’ erroneous L2 utterances, and it may include an indication that an error has been made, can present the correct form or metalinguistic information about the nature of the error, or it may combine these various forms of information (Ellis, Loewen, & Erlam, Reference Ellis, Loewen and Erlam2006).
In the GBL literature there appears to be almost unanimous agreement on the beneficial role of feedback for learning. However, the field of SLA has long been divided over the topic, and the type and timing of CF remain debated issues (Long, Reference Long2007). Depending on the theoretical assumptions concerning how an L2 is acquired, CF is attributed a more or less favourable role, is thought to be more or less effective, or even to have harmful effects, such as to increase anxiety and to foster less favourable attitudes towards learning (Truscott, Reference Truscott1996).
The first assumption concerns the interface between explicit and implicit knowledge (Ellis, Reference Ellis1997). Theories that are largely constructed on the importance of implicit learning mechanisms for acquisition, such as generativist theories of language learning (Schwartz, Reference Schwartz1993) and Krashen's Monitor Theory (Reference Krashen1981), presume that explicit knowledge is disconnected from implicit knowledge, that CF inculcates explicit knowledge, and that therefore CF can only contribute to learned knowledge and not to acquisition. Conversely, if an interface between explicit and implicit knowledge does exist, CF is seen to foster acquisition in the longer term. For instance, Skill Acquisition Theory (DeKeyser, Reference DeKeyser2008) emphasizes the possibility of explicit knowledge becoming implicit over time. A related issue is the role of awareness and noticing in SLA (Schmidt, Reference Schmidt1990). From this perspective, CF, especially in more explicit forms, is generally considered to stimulate noticing and conscious processing, both of which are presumed to promote SLA. However, some kinds of implicit CF, which signal in less overt ways that an error has been committed (e.g., recasts), have received theoretical attention precisely because they are more implicit, and are considered beneficial for acquisition because they jointly focus on form and meaning, leading to strong form-function mappings in the flow of communicative interaction (Long, Reference Long2007).
Thus far, empirical research on learning outcomes suggests that some form of CF is beneficial. The effects of CF on various aspects of L2 development have been demonstrated in a number of studies, both in (quasi-)experimental instructed L2 settings (e.g., Carroll & Swain, Reference Carroll and Swain1993; Long, Inagaki, & Ortega, Reference Long, Inagaki and Ortega1998; Takimoto, Reference Takimoto2006), in more naturalistic classroom settings (e.g., Havranek, Reference Havranek2002; Loewen & Philp, Reference Loewen and Philp2006; Lyster & Ranta, Reference Lyster and Ranta1997), and in CALL (e.g., Brandl, Reference Brandl1995; Heift, Reference Heift2004; Nagata, Reference Nagata1993; Pujolà, Reference Pujolà2001). Also, reviews tentatively suggest that CF types that include metalinguistic information (such as grammar rules) and/or which function as prompts (signalling the error without providing the correct response), aid language development more than CF types which are generally subsumed under the header ‘implicit’, such as recasts (Ellis et al., Reference Ellis, Loewen and Erlam2006; Lyster & Saito, Reference Lyster and Saito2010). Although these findings are still tentative due in part to methodological difficulties in CF research (Ellis et al., Reference Ellis, Loewen and Erlam2006; Long, Reference Long2007; Lyster & Saito, Reference Lyster and Saito2010; Mackey & Goo, Reference Mackey and Goo2007; Russell & Spada, Reference Russell and Spada2006), Norris and Ortega's (Reference Norris and Ortega2000) quantitative meta-analysis of the effectiveness of explicit vs. implicit instruction showed larger effect sizes for instruction that included rule explanation. In summary, the preponderance of current research suggests that CF which stimulates conscious processing of L2 input through rule explanation or explicit prompting is likely to be more effective.
The GBL literature is less well articulated with respect to what kinds of feedback best support learning in games, and if the purpose is to educate (rather than purely entertain), feedback mechanisms as they relate to learning remain underexplored. A first observation is that, in contrast with non-GBL environments, games seldom give away answers to players (Becker, Reference Becker2007). Games usually stimulate explorative behaviour (Kiili, Reference Kiili2005), and aim to motivate players to find ‘correct’ answers through trial and error.
Secondly, it can be observed that games rarely articulate in a direct way the domain knowledge or rules which underlie the in-game content, and which might serve as an immediate support mechanism that leads players to solve problems successfully. Although the primary purpose of commercial games is to entertain rather than to educate, and the learning is only a side-effect of gaming, a comparison may be drawn with experiential learning models: problem-solving in games follows a fixed pattern of being exposed to a concrete experience and to data, making reflective observations, construing mental generalizations and hypotheses about the experience, and testing these hypotheses through active experimentation (Kiili, Reference Kiili2005). The discovery of patterns and construal of generalizations by players themselves is an essential feature of game experience. Koster (Reference Koster2005) points out that part of the attraction of games is the process of pattern seeking, which is a fundamental aspect of human experience and aligns with how our brains work. Koster argues that our brains seek patterns “so much we don't even realize we're doing it”, i.e., without our conscious attention and without explicit teaching, much like the process of first language acquisition (op. cit.: 16). In this view, gaming seems to cater to implicit learning.
Third, feedback in video games is different because it is connected to the representation of the game's world or theme. For Prensky (Reference Prensky2001), feedback comes about “as action” (op. cit.: 159), e.g. a player's character may die because enemies are quicker. Or, as in the simulation game The Sims, the player's character is sacked for unproductiveness at work as the result of continuous nights without sleep. Feedback in games, in other words, is largely dependent on the content or theme of the game. In SLA terms, this implies a focus on meaning.
Thus, the conceptualisations of feedback in GBL and SLA differ quite clearly, as feedback in games lacks the provision of correct responses and rule explanation, and is adapted to the game's theme. This warrants empirical research on the effectiveness of more ‘game-like’ types of feedback in comparison with more ‘traditional’ CF in language learning contexts.
1.2 The role of the learners’ self-perceptions in game-based feedback for SLA
It is evident that feedback in games also has purposes other than to deal with mistakes and failure (“negative feedback”), and that it can also serve to reinforce, to reward or to maintain motivation (“positive feedback”) (Becker, Reference Becker2007: 25). Although the latter is somewhat outside of the scope of this paper, it is difficult to separate these two types of feedback in games. Professional game designers spend considerable time on the design of so-called ‘failure states’ (Purushotma, Thorne & Wheatley, Reference Purushotma, Thorne and Wheatley2008). Failure states are phases in the game in which it is made clear to the player that something has gone wrong, or that the player has not adequately performed an activity. For game designers, it is critical that such failure states are interesting and enjoyable, and that the player can repeatedly fail without compromising the motivation necessary for successfully completing an action or task. So, feedback design for GBL must not ignore individual difference factors, more particularly learners’ perceptions of themselves, i.e., as successful or failing learners and players.
In SLA research on CF, claims have been made that individual differences have been underestimated, as they may mediate the effectiveness of CF (e.g., DeKeyser, Reference DeKeyser1993). This reflects the central tenet of the Cognitive Mediational Paradigm, which posits that the effects of instruction are mediated by learners’ cognitions such as their conceptions and perceptions of the learning environment, their prior knowledge and aptitude, and their attitudes and self-perceptions (Winne, Reference Winne1987). As to learners’ self-perceptions, some SLA scholars have argued that CF is harmful because it can reduce motivation (e.g., Truscott, Reference Truscott1996) and impede interactional processes. Teachers and pedagogies struggle with a “balancing act of two necessary but seemingly contradictory roles”, i.e., to “establish positive affect among students yet also engage in the interactive confrontational activity of error correction” (Magilow, Reference Magilow1999: 125). Even in tutorial CALL, where CF may be less face-threatening, learners may experience it as a form of “mild social punishment” (Schulze, Reference Schulze2003) or they may intensify “salient cues [feedback] which are evaluated as negative” (Robinson, Reference Robinson1991).
The research on individual differences and CF provides ample evidence that students favour CF and that they find it helpful (Cathcart & Olsen, Reference Cathcart and Olsen1976; Chenoweth, Day, Chun, & Luppescu, Reference Chenoweth, Day, Chun and Luppescu1983; Hedgcock & Lefkowitz, Reference Hedgcock and Lefkowitz1994; Radecki & Swales, Reference Radecki and Swales1988; Saito, Reference Saito1994; Schulz, Reference Schulz2001). There are also findings suggesting that students prefer detailed metalinguistic feedback more than a ‘right/wrong’ type of feedback (Nagata, Reference Nagata1993) and more than implicit feedback in the form of recasts (Kim & Mathes, Reference Kim and Mathes2001). Moreover, there is evidence that students’ attitudes towards CF (Havranek & Cesnik, Reference Havranek and Cesnik2001), their anxiety (DeKeyser, Reference DeKeyser1993; Havranek & Cesnik, Reference Havranek and Cesnik2001; Sheen, Reference Sheen2008) and their prior motivation (DeKeyser, Reference DeKeyser1993) explain differences in learning gains. In these studies, positive attitudes and low anxiety resulted in higher levels of L2 development. In addition, DeKeyser (Reference DeKeyser1993) found that learners with high extrinsic motivation did better without systematic and explicit error correction. Students with low extrinsic motivation, on the other hand, excelled when they did receive systematic CF. To our knowledge there is no empirical research on the effects of CF on motivation.
L2 pedagogy and communicative approaches to language teaching, in particular, tend to take into account the learner when advising on when and how to use CF, with the specific recommendation to not correct too frequently during communicative interaction. It is unclear whether the intention is to safeguard learner motivation, but one purpose certainly is to favour communicative fluency over linguistic accuracy, at least while tasks are being carried out. In a communicative approach, a focus on language as meaningful communication precedes a focus on isolated language forms (Ellis, Reference Ellis2003). As a consequence, CF plays a subservient yet crucial role during communicative tasks, as it can draw attention to linguistic form implicitly and/or after the completion of a task. High discrepancy between teachers’ beliefs on CF and learners’ attitudes towards CF, especially for speaking activities (Magilow, Reference Magilow1999; Schulz, Reference Schulz2001), shows that teachers take into account in daily practice the pedagogical reflex to use CF with care. CF is typically delayed in task-based language teaching (Willis & Willis, Reference Willis and Willis2007) and classroom simulation/gaming (Bullard, Reference Bullard1990), and occurs mainly during the post-task/debriefing phase. In communicative language teaching, teachers make wide use of (more or less) implicit recasts (Lyster & Ranta, Reference Lyster and Ranta1997), as they are relatively undisruptive to communicative flow.
1.3 Need for a balance between instruction and play?
A parallel can be drawn between the role of CF in SLA and how CALL games could embrace instructional feedback. A recurring theme in the game-based learning literature is the opposition between learning and playing. Such a dichotomy is often articulated by the claim that an “appropriate balance” needs to be found between learning and gaming (Kickmeier-Rust & Albert, Reference Kickmeier-Rust and Albert2010; Kiili, Reference Kiili2005). The underlying assumption here seems to be that learning is by default laborious or unpleasant, or that learners are demotivated, and that gaming is the panacea that will raise learners’ motivation. The ultimate goal for educational game designers, then, is to protect “flow”, that is, the feeling of being fully engaged in an activity (Csikszentmihalyi, Reference Csikszentmihalyi1990), the interaction, and the feeling of immersion in experience. Evidently, debate on the utility of feedback in educational games is influenced by such thinking. Although not necessarily an advocate of the learning vs. playing argument, Prensky (Reference Prensky2001) writes that “the art of providing feedback in a game is extremely important and complex because either too little or too much can lead quickly to frustration for the player” (op. cit.: 122).
Thus, the GBL literature suggests applying (corrective) feedback with moderation, or at least in playful forms, so as to keep the player/learner engaged. In communicative language teaching approaches, with which game-based language learning has been associated because of its emphasis on language as a resource to complete meaningful tasks (Baltra, Reference Baltra1990; Purushotma et al., Reference Purushotma, Thorne and Wheatley2008), the rationale seems different. Here, the primary purpose is to ensure communicative fluency rather than to safeguard motivation or prevent frustration.
Still, the parallel seems worth investigating for two reasons. First, as was indicated above, some SLA scholars have suggested that corrective feedback may reduce the motivation of language learners (Truscott, Reference Truscott1996). To date, however, empirical evidence for this claim is lacking. Secondly, there seems to be no theoretical or empirical reason to posit an opposition between ‘learning’ and ‘playing’. Possibly, the same intrinsic motivational processes are at play in learning as in gaming.
A theory that might help explain this is self-determination theory (SDT) (Ryan & Deci, Reference Ryan and Deci2000). SDT, as a comprehensive theory of how human beings are motivated to perform various activities in various contexts, might back up the claim that there is no tension between play and learning, or between intrinsic motivation and CF. According to SDT, people require a certain amount of feedback on their actions in order to feel competent. In games, specifically, a player's need for competence could be satisfied by the provision of “meaningful informational feedback”, that is, feedback which is useful, non-judgemental and immediate, and which thus allows a player to improve his or her performance (Rigby & Ryan, Reference Rigby and Ryan2011: 19). Our hypothesis is that in educational games, CF can satisfy learners’ need for confirmation of competence if the CF is found useful (i.e., if learners have the impression that they are learning) and if the CF is actually used to complete activities in the game.
1.4 Research questions
The convergence between the conceptualization of CF in the SLA literature and language pedagogy on the one hand and the hypothesized “appropriate balance” in GBL raises issues for the design of effective CALL games. Questions may be asked with respect to the utility and desirability of explicit CF and more implicit, “game-like” kinds of feedback in relation to the motivation of language learners, their perceptions and use of learning support in educational games, and the effects of different configurations of feedback in regard to L2 development.
The current study aimed to probe learners’ perceptions of CF in an immersive game for English pragmatics, and to explore the relation of these perceptions about the learning environment with learners’ perceptions of themselves as learners and as players. First, it may be argued that students will not learn from CF if they do not find it useful within the context of the goal-directed actions that comprise the game mechanics. Secondly, there may be differences between learners’ perceptions of explicit and implicit CF, as implicit CF may be more aligned with the feel of an immersive game. Third, if learners perceived themselves as competent, possibly as the result of interacting with useful, clear and informational CF in the game, perceived competence might explain perceptions of CF: players will find CF useful and will prefer conditions which provide it. Fourth, learners’ intrinsic goal orientation for learning English can predict their perceptions of CF. If they are intrinsically interested in learning English, students will probably find CF useful and will prefer it. Finally, learners’ game experience might predict CF perceptions. If they feel immersed in a game, are interested in its story (which implies a focus on meaning), and feel good as a result of playing the game, they might find CF, and especially explicit CF formats, disturbing or less useful.
Therefore, we pose the following research questions:
1. Do learners find CF useful in an immersive game for English language learning?
2. Do learners have different perceptions (perceived usefulness and preferences) of explicit CF than of implicit CF in such a game?
3. Do learners’ intrinsic goal orientation, their perception of competence as a result of playing the game, and/or game experience explain their perception of CF?
2 Method
2.1 Participants
The participants included 83 first-year university students and learners in their final two years of high school in Belgium. The university students, which represented the majority of participants, were enrolled in various programmes, a minority of which were language programmes. The large majority (82%) of these students did not have English as a compulsory study subject, but did have to read texts in English for other courses. Their level of English was around B1 (intermediate) of the Common European Framework of Reference for Languages (Council of Europe, 2011), the required level at the end of high school education. Sixty-one students were female, twenty-two male. The age range was between 16 and 24 except for two 33-year-olds (Mdn = 19).
The game and associated instructional materials were developed through a public-private partnership as a proof-of-concept of a language learning game, and were hence not integrated in the participants’ curricula. The university students participated either on a voluntary basis, or as part of a methodology course taught in the educational sciences. The high-school learners were invited to participate in the study through the researchers’ personal contacts with teachers. The learners were told that they would take part in an experiment, and played the game only once and in one session. The game sessions lasted between thirty and sixty minutes.
2.2 Description of the game/learning environment
In order to maximize the potential tension between CF and a game environment, we chose to create a fully immersive 3D avatar-based game, using a game development kit which had been provided by a renowned Flemish professional role-playing game (RPG) developer. In the commercial standalone RPG, the player is a dragon slayer who carries out various quests in a medieval-looking fantasy setting. The RPG relies heavily on narrative and point-and-click dialogues. The dialogues contain written transcripts, and feature voice actors and detailed character animations. As is the case with most commercial RPGs, written language is used in object descriptions, the player's inventory, and a logbook of completed and current quests.
On the basis of this RPG engine and the available character and world visuals, the first author co-developed a game customized for the training of English pragmatics. The learning goal was to equip high-intermediate learners with the typical constructions and speech acts necessary for making formal conversation in two domains: social introductions and professional network development. The first author developed the content for the game on the basis of materials provided by a language training company external to the university. Content included dialogues, a document with supportive information (text outlining conversational structures, model utterances of speech acts, and explanations on the situations in which to use these utterances; see figure 1), and the elaborate feedback messages that would be shown upon mistakes (cf. infra). Finally, a professional trainer in Business English proofread all instructional materials.
Fig. 1 “in-game document with supportive information”
The learners’ overall objective in the customized game was to advance from quest to quest by choosing in the conversations with non-player characters (NPCs) the most pragmatically appropriate response from a list of options that was presented on the screen. The voices of the NPCs had been pre-recorded with a team of English language teachers; the recordings were post-produced in order to make them sound ‘real’ and to improve the audio quality. In order to make the NPCs further ‘come alive’, professional game development artists animated them by applying gestures and lip-synchronization. Although the setting was medieval, the projected world was highly detailed in visuals (3D, fine-grained textures) as well as in aural detail (human voices and sound effects). So, it provided to learners a simulation environment that was high in perceptual fidelity – it looked and sounded ‘real’ – and moderate in functional fidelity (de Jong, Reference de Jong2005); the quests were real-world tasks but the point-and-click dialogues were impoverished in comparison with natural conversation. As a consequence of high fidelity, learners could practice real-world tasks in a safe environment, and were immersed in the 3D world, which resulted in a mean score on the intrinsic motivation inventory (Plant & Ryan, Reference Plant and Ryan1985) (cf. infra) of 66 percent (SD = .12).
In order to maximize learning opportunities, instruction was designed according to the Four Component Instructional Design (4C/ID) model (van Merriënboer & Kirschner, Reference van Merriënboer and Kirschner2007). 4C/ID is suitable for complex learning tasks and is composed of tasks (grouped in ‘task classes’ or sets of tasks that are similar in content and complexity), supportive information (general models and schemas that help to teach nonrecurrent aspects of tasks), just-in-time information (aimed at encoding recurrent aspects of tasks into rules), and part-task exercises (designed to help learners gain automaticity of recurrent aspects of tasks), which were not included because of the brief play period associated with this research. In the game, the tasks coincided with interactive dialogues. Learners had to complete two task classes (one for introducing people and one for networking), each of which contained three tasks and supportive information including a ‘theory’ document (cf. supra, figure 1) and a dialogue model (a non-interactive dialogue between NPCs which demonstrated the use of speech acts). In accordance with the chosen instructional design model, the dialogues within one task class were very similar in complexity and in content, but support within the dialogues of a task class, more specifically just-in-time information, was gradually withdrawn. Just-in-time information was operationalized as CF, and was different in each of the three tasks/dialogues (Table 1).
Table 1 CF types
In the first task, there was a high level of CF (type A): when a response was clicked, the dialogue paused, and learners were shown visual feedback as to whether their selection was correct or incorrect. When the response was incorrect, they were shown the written correct response and a written metapragmatic explanation. They also had access to explanations for alternative responses. When the learners had finished reading the feedback, they clicked a button, upon which the dialogue continued. In this phase, if an incorrect response is selected, the NPC replies with an implicit spoken comment which expresses surprise or a lack of understanding. This response is accompanied by the NPC's gestures (e.g., frowning, showing surprise, waving arms). In the second task (CF type B), metapragmatic explanation was no longer shown by the system, but learners could still request it for each possible answer by clicking on the answers. The third task (CF type C) contained the least support: the system no longer paused, but immediately moved on to the character's response, which is the default interaction in commercial RPGs, and the correct response was hidden. The system did show, as in the previous tasks/support levels, whether the chosen response was correct or incorrect, and the NPC responded accordingly. A written metapragmatic explanation could be requested for the response if it was inappropriate.
As this study was part of another experiment (Vandercruysse, Vandewaetere, Cornillie, & Clarebout, Reference Vandercruysse, Vandewaetere, Cornillie and Clarebout2012) which intended to investigate the effects of the game element ‘competition’, the positive/negative feedback was visualized to half of the students as a green checkmark and a red exclamation mark, respectively, and to the other half of the students as a golden coin and a silver coin, respectively. Students were in each case told before the experiment what the meanings of the icons were.
In total, three types of CF were included in the game ranging from explicit to implicit. Type A was most elaborate and showed a metapragmatic explanation immediately upon a mistake (see figure 2); type B was thought to stimulate self-discovery of rules as learners could compare appropriate with inappropriate responses and could see the metapragmatic explanation on request; type C was more aligned to formats common to recreational game environments as it relied principally on the reactions of NPCs. Students would typically have been familiar with CF types A and B on the basis of their experience in a ‘traditional’ learning environment, but familiar with CF type C only if they had experience playing commercial video games.
Fig. 2 “elaborate CF (type A)”
For the metapragmatic explanations, we avoided terminology with which the learners would not normally be familiar at the targeted proficiency level. The metapragmatic explanations were between two and four sentences in length. Below is a sample of the metapragmatic CF:
Remember to always ask open-ended questions in networking conversations, these will stimulate your partner to say more than a simple “yes” or “no”. Open-endedness may be realized by using questions in a past tense, using the progressive, or chunks like ‘In what ways …?’, ‘How have you been …?’, ‘What kind of …?’, ‘What … like?’, etc.
Apart from the CF immediately following a response, students could rely on other support mechanisms. First, at the top of the screen a student could see the transcript of the currently active dialogue as he or she had played it, and in which the CF was available so that past errors could be reviewed. Secondly, a pedagogical agent in the form of a language trainer appeared after each dialogue in which the student made mistakes. This character offered help, which learners could reject or accept (see Figure 3). In the latter case, learners could access the supportive information (cf. supra): a document containing speech acts related to the past dialogue (see figure 1) and the dialogue model. Students could also access the supportive information in the absence of the trainer by pressing a key, and they were continually reminded of these materials through the game interface.
Fig. 3 “Language trainer character offering help”
2.3 Data collection
The data were collected in December 2010 and January 2011 through questionnaires, interviews and game logs. Before the game, students filled in the motivated strategies for learning questionnaire (Pintrich, Smith, Garcia, & Mckeachie, Reference Pintrich, Smith, Garcia and Mckeachie1993), of which we retained the intrinsic goal orientation subscale. This scale consisted of four items (Cronbach's α = 0.69), and focused on mastery, learning and challenge (e.g., “I prefer tasks which I can learn from, even when this does not result in good grades”). After the experiment, we measured perceived competence using the intrinsic motivation inventory (Plant & Ryan, Reference Plant and Ryan1985). The subscale of perceived competence contained six items (α = 0.89) (e.g., “I think I am pretty good at this activity”). Students also filled in a game experience questionnaire (De Grove, Van Looy, & Courtouis, Reference De Grove, Van Looy and Courtouis2010), of which we retained the immersion (e.g., “I felt totally absorbed”), vividness (e.g., “I was captivated by the story of the game”) and positive affect dimensions (e.g., “I felt satisfied”) (6 items; α = 0.63), as these dimensions seemed most crucial to determine a positive experience of playing such a 3D RPG. Finally, students also filled in a 7-point Likert questionnaire on CF (see Appendix), which we developed for this study. All 83 students filled in these questionnaires before and after playing the game.
Additionally, the first author conducted semi-structured interviews with twelve students on the basis of a convenience sample. The students were interviewed in their mother tongue (Dutch). Nine of these students worked in the ‘competition’ condition, which implies that they also saw their in-game score compared with the score of a virtual opponent, which was dynamically adapted to the student's score to create a feeling of competition. The other three students did not see any scores. During the interviews, the researcher first probed students’ general conceptions about CF, and then asked what they felt about the in-game CF in terms of usefulness and preferences. In the fashion of a stimulated recall measure (Loewen & Reinders, Reference Loewen and Reinders2011: 160), students were shown screenshots of different types of CF in the game. Interview data were coded and analyzed in two cycles, first deductively on the basis of an initial set of constructs (e.g., a number of general feedback characteristics, perceived usefulness, perceived competence, preference, immersion), followed by more fine-grained inductive coding on the basis of a second reading.
The game logs measured learners’ behaviour in the game, such as the chosen responses in the dialogues, how long they worked on the tasks, and whether they made use of the supportive materials.
3 Findings and discussion
3.1 Preliminary analyses
Using the game logs, we first determined whether students were actually ‘exposed’ to the CF during the game. By taking the ratio of correct responses as a proportion to the total number of responses (three students responded a few times less than the expected 38 times), we computed the students’ performance in the game, which ranged between accuracy scores percentages of 31 and 77 (M = .58, SD = .1). This implies that all students had been presented with CF. This was confirmed in the interviews, as none of the students showed surprise when seeing the screenshots containing CF, and because they could clearly explain the purpose of the CF.
In what follows, findings will be presented and discussed with respect to the perceived usefulness of CF in the game, the perceptions of explicit and implicit CF, and the relation of these perceptions with variables associated with self-perception.
3.2 Usefulness of CF
Quantitative analysis showed that, generally, students found the CF quite useful, with average scores for items 2–5 of over 5 on a 7-point scale with 7 being perceived of as most useful (see appendix). The first item, which targeted inductive/discovery learning had a lower mean and a higher standard deviation.
The relatively high score for perceived usefulness was confirmed in the interviews. None of the students responded that the CF was not useful generally. When questioned about the reasons, students said they found CF useful because it helped them to learn, and to remember the content. Additionally, one particular student seemed to claim that CF helped to realize transfer to contexts outside of the game.
Interviewer: Let's summarize. How did you experience correction on mistakes in general?
Student 2: Err, well, I just remembered it. Also in order to use it further on in the game, but also for now. Well yeah, I think it's good that it [the game] contained feedback, because, as I said, you learn from your mistakes. [emphasis added]
3.3 Perceptions of explicit and implicit CF
We defined explicit CF as CF which contains a rule (according to the definition of explicit instruction, Hulstijn, Reference Hulstijn2005) and explicit information about the correctness of learners’ responses (positive/negative feedback), which is immediately given when mistakes are made, and which offers students the opportunity to reconsider the options after they have responded (7 items, α = .69). Implicit CF was defined as CF that is adapted to the game environment (i.e., the characters’ reactions), which stimulates autonomous inquiry by learners, and in which errors and rules are only shown after the task (6 items, α = .59). In order to create the subscales, we took the means of the corresponding items. ‘Perception of CF’ was defined as the combination of perceived usefulness of and preference for a particular type of CF. The data from the Likert responses (see Appendix) show a higher mean score for perception of explicit CF (M = 5.2, SD = .8) than for implicit CF (M = 3.8, SD = .9).
Analysis of the interviews revealed similar results. Students specifically found the most elaborate feedback (type A) most useful. They also preferred it to appear throughout all the dialogues, instead of just in the first one.
Here the answers were judged on what was best in the situation, and also the second best … that I found good. Also the blue text that appeared, explaining the situation in which the chosen response was better, I found really useful. I had preferred it to appear all of the time, but sometimes it didn't. And also that the answers stayed. Because sometimes I still had to look in the history of the text, whether I had given the correct response or not, and there the correct response wasn't shown. (Student 4)
Additionally, students indicated that they found the rule-based feedback most useful because they could easily memorize it and apply it to similar situations in the game.
Interviewer: Had you preferred the blue text to appear all of the time?
Student 4: Yes.
Interviewer: Why?
Student 4: To be able to learn from your mistakes, to know why you made a mistake, and as such to be able to apply it to future problems.
Interviewer. So you did not find this [the CF without rule] sufficiently informative?
Student 4: No, not really. Well, of course it's good that you can still, well, see what the correct answers are, but … not really why. In this way you cannot extrapolate it to other problems.
Student 9: Yes, well that the response was incorrect. And the feedback you get. [Explicit explanations of] Why the response was incorrect, I found really good. Because then you are immediately taken on your point, and then you know that you will maybe not make the same mistake in the future. Or hopefully not. And then you also know what the correct response was. So that was positive, yes.
The implicit feedback was found less useful for learning:
Student 2: In fact he is responding to the incorrect response that was uttered before.
Interviewer: And is that useful? Is that an example of feedback for you?
Student 2: No. Not that.
Interviewer: Why not?
Student 2: (laughs) Well in fact what that man is doing is to answer beside the point, so in fact you don't learn anything from that. In fact he says: ‘what are you saying?’, for example. So that is not really feedback to me.
Student 7: But wasn't that so with these characters? If you indicated the incorrect response, then it was like ‘hmm yes, ok, well’. Incorrect.
Interviewer: And what does that do to you?
Student 7: I actually preferred the feedback so that I knew I was wrong.
Interviewer: Why?
Student 7: Because then I could learn something. But now I just thought ‘they are reacting so foolishly, come on!’.
Implicit CF was preferred by two students, one of whom thought she was very competent in English, while the other was reading for a degree in languages. Interestingly, they argued that they preferred it because it was most fun, not because it helped them to learn.
Student 6: The reactions. Yes, really.
Interviewer: Why?
Student 6: Ah, because it was more human. In real life that is the most evident reaction in a conversation. And that is what you'll have to do with. With the reactions, in real life. So you need to attend to that, in reality.
Interviewer: So that was sufficient for you?
Student 6: Yes, I also found that most fun. Most challenging.
Interviewer: OK. And if you had to choose one of these three, for yourself? In your case it would be to learn, right?
Student 8: Well in that case the implicit one. It was fun, when they responded so weirdly.
Further, some students preferred a combination of explicit and implicit CF, as they considered the rule-based CF most useful for learning, and the implicit CF to be most fun, challenging and attention-grabbing.
Student 9: I think a combination of both would be best. If you didn't give the blue feedback [explicit explanations], then the person who played the game, or used the educational program, would just start the conversation over. He would just pick something else until it was correct, but he wouldn't know why. So the blue feedback is certainly necessary. On the other hand, when the NPC responds ‘incorrectly’, as here, that shakes you up a bit. When he is agitated.
Interviewer: The fact that you have an impact on the world?
Student 9: Yes, yes.
Further, others claimed that the CF type is best adapted to prior knowledge, and that attention and noticing played a significant role.
It depends. It depends on the age. If it's more for secondary education, then I think it's better to have some explanation at the bottom. And the more you advance, the less explanation you need each time. Or maybe work with levels. When you reach a certain level, it's going better for you, and you don't need the explanation each time. (Student 5)
Then I think that, if you make a full game, and it's all implicit, that it's a bit dangerous, whether it would always be understood. Or maybe in a first phase explicit, and when you have a better command, then implicit. Because then most of the times you know that it's a comment, because you've been pointed at it a few times. (Student 8)
3.4 Effects of intrinsic goal orientation, perceived competence and game experience
Pre-experimental intrinsic goal orientation, and perceived competence and game experience (measured afterwards) can be considered possible predictors of how students perceive feedback. As the three predictors had low inter-correlations (Table 2), they could be jointly used in the regression model to explain different aspects of the data.
Table 2 Pearson's correlation coefficients
The three predictors jointly explained more than 20 percent of the variance of how students perceive explicit CF (R2 = .21, F(3,79) = 6.97, p < .001). Perception of explicit CF was positively affected by perceived competence (β = .04, p < .01) and game experience (β = .09, p < .05). The difference in perception was not significant for intrinsic goal orientation (β = .05, p < .1).
Upon removal of two outliers (Cook's distance higher than 2.5), the three predictors jointly explained 25 percent of the variance of how students perceive explicit CF (R2 = .25, F(3,77) = 8.50, p < .001). Perception of explicit CF was positively and significantly affected by all three predictors: perceived competence (β = .03, p < .05), intrinsic goal orientation (β = .06, p < .01), and game experience (β = .25, p < .05).
Implicit CF was regressed onto the same three predictors but none of the effects described above for explicit CF were found to be significant.
3.5 Discussion
For research questions 1 and 2, respectively, we found that students found the CF useful in general, and that they found immediate and explicit CF (containing metalinguistic explanation) more useful than and preferable to implicit CF (delivered through the characters’ responses and designed to stimulate autonomous inquiry). However, these findings do not imply that implicit and more playful feedback are not relevant. In the interviews, several respondents replied that the implicit CF was fun and made them feel immersed. What seemed optimal for them was a combination of elaborate and immediate CF (type A) with feedback that is adapted to the game (type C). Such feedback can elevate learners’ sense of immersion, which might increase their commitment to work through the CF in order to advance in the game. Thus, playful and creative feedback loops can complement explicit feedback mechanisms deemed effective for learning.
For research question 3, it was found that positive perceptions of explicit CF could be partly explained by three factors related to self-perception: intrinsic goal orientation, perceived competence, and game experience. This indicates that learners who consider themselves intrinsically interested in learning English as a foreign language, who felt competent while playing the game, and who had a positive experience playing this particular game had more positive perceptions of explicit CF. This finding is in line with our hypotheses that intrinsic goal orientation and perceived competence would explain positive perceptions of CF, but runs counter to the idea that learners who had a more positive experience of the game would have less positive perceptions of CF, especially of explicit CF. This tentatively suggests that in educational games, explicit CF could be most helpful for learners who are a priori intrinsically motivated, and that it might also contribute to the motivation of individual learners as the result of playing.
In the warm-up phase of the interviews, we probed students’ general conceptions of feedback. They associated it with testing, exams, assignments, and scores (i.e., summative evaluation), rather than with meaningful information that would support their learning (i.e., formative evaluation). The finding that students generally did not conceptualize feedback as formative could partly explain their positive perception of explicit CF in the game, as it was elaborate, immediate, non-judgemental and helped them to do better on subsequent tasks. It is thus likely that, as a result of getting such CF and by using this information in the similar tasks that followed, students felt supported and competent in the game, and formed positive perceptions of such CF after playing. However, this finding is somewhat surprising; since students’ average performance was quite low, they would have received a large volume of CF that might have them feeling less competent. So, either students’ perceived competence was unaffected by the CF, or they improved as a result of such feedback. The relation between learners' self-perceptions and learners’ in-game behaviour requires deeper investigation which is outside of the scope of this paper, but an additional regression analysis revealed that higher performance partially predicted positive perception of explicit CF (β = 1.8, p < .05) (R2 = .05, F(1,78) = .075, p < .05).
4 Conclusion, limitations of the study, and suggestions for research
Our findings can be summarized as follows. First, language learners generally found CF useful in an immersive educational game, and found implicit CF that lacks correct responses or metalinguistic explanation too weak for L2 learning. Secondly, individual difference factors related to learners’ self-perception determined the perceived usefulness of and preferences for explicit CF in the immersive game (not so for implicit CF): learners who were intrinsically interested in learning English, who perceived themselves as competent during the game, and who had an enjoyable game experience had more positive perceptions of explicit CF (i.e., they found it useful and preferred it). Third, learners reported ‘fun’ and a sense of immersion when being confronted with CF that was implicit and adapted to the game (the characters’ comments).
These findings have two implications. First, if we define ‘intrinsic motivation in games’ as an individual's subjective experience that is the combined result of enjoyable immersive gameplay with his or her positive perception of competence as the result of play, then instruction, including non-judgemental CF, need not necessarily get in the way of intrinsic motivation. This study thus provides evidence that the “dichotomy between overt instruction/guidance, on the one hand, and agentful immersion in experience is a false one” (Gee, Reference Gee2007: 156). This has positive consequences for educational game design: instruction in games does not necessarily sacrifice ‘fun’, and designers should not shy away from including CF and other forms of instructional support as “overt verbal information […] ‘just in time’ (when it is needed and can be used) or ‘on demand’ (when the player is ready for it and knows why it is needed)” (ibid).
A second and potentially more crucial implication of our findings is that the effectiveness of feedback in game-based language learning might depend on how useful learners think it is, and on whether it stimulates intrinsic motivation. This is in line with the Cognitive Mediational Paradigm (Winne, Reference Winne1987), which posits that the effectiveness of instruction is determined by a host of individual differences such as learners’ intrinsic motivation and their perceptions of the (instructional) environment and its constituents. In this study, learners found elaborate and explicit CF with explanations most useful, especially if they were highly motivated, and reported to have learnt from it most. Further research should thus also study the impact of feedback on learning outcomes.
This study was limited in a number of respects. First, the target audience included mainly highly educated learners, whose intrinsic interest in and prior knowledge of English was relatively high. The motivation, learning strategies, and actual usage of CF can be quite different for less advanced learners (see Brandl, Reference Brandl1995; Heift, Reference Heift2002).
Furthermore, research on (corrective) feedback in CALL games is quite novel, so a more explorative method seemed appropriate. Consequently, a second limitation of the study is that all learners received the same kinds of feedback, which makes it difficult to say anything about whether discrete elements of feedback (such as error indication, metalinguistic information, or feedback that is adapted to the theme of the game) could actually affect learners’ perceptions of instruction, their intrinsic motivation or their sense of immersion.
Therefore, future studies should first of all recognize that ‘feedback in CALL games’ is a multidimensional construct, which needs to be taken apart in order to experimentally examine the effects of its constituents on learners’ perceptions, motivation and learning outcomes. We propose that further research should distinguish by and large between, on the one hand, corrective feedback (and its different subcomponents) aimed at increasing a learner's understanding and, on the other hand, more ‘game-like’ feedback elements that can contribute to intrinsic motivation, namely positive feedback (designed to increase a learner's sense of competence) and situational feedback adapted to the game's theme (which can increase a sense of immersion). Various configurations of the constituents of feedback in a game-based language learning environment need to be implemented in different experimental conditions, so that the effects of feedback can be investigated directly. A key aspect that seems worthy of future research concerns the composite question (a) whether learners do actually process metalinguistic CF in games, which requires temporary time-outs from the flow of play; (b) whether this processing leads to the acquisition of explicit and/or (automated) implicit knowledge (through continued practice); and (c) what the complementary motivational role of positive and situational feedback might be in this respect.
Acknowledgement
We are grateful to Dr. Wilfried Cools, who gladly offered his expertise in statistics, and prepared the game logs for subsequent analysis.
CF questionnaire with means and standard deviations
