2 Open learner models for learners and teachers

There is now much interest in visualisation of learning data (Klerkx, Verbert & Duval, Reference Klerkx, Verbert and Duval2014), and learning analytics dashboards are being developed to help make sense of large amounts of data (see Verbert, Duval, Klerkx, Govaerts & Santos, Reference Verbert, Duval, Klerkx, Govaerts and Santos2013). These are often performance, activity, interaction, navigation, or behaviour-focussed. In contrast to this, in this section we introduce an approach that visualises current competence levels, based on learner modelling comprising data from a range of activities and platforms. We describe the typical role of a learner model; the extended approach of opening and visualising the learner model to the learner; and a further extension where the system does not perform any teaching, but leaves control and responsibility for learning with the user.

2.2 Open learner models and independent open learner models

As described above, learner models are typically used by a teaching system to identify the most appropriate next system action or intervention for the learner. Open learner models (OLM) are learner models that are ‘open’ to the user. This means that, instead of the learner model being used solely by the system to personalise the interaction to the learner’s needs, it can in addition be accessed directly by the learner or other users (Bull & Kay, Reference Bull and Kay2007). Access to their learner model must be in a form that the user can interpret, and will not necessarily match the format of the underlying model. For example, the learner model may be presented as a concept map (Mabbott & Bull, Reference Mabbott and Bull2004; Perez-Marin, Alfonseca, Rodriguez & Pascal Neito, Reference Perez-Marin, Alfonseca, Rodriguez and Pascual-Neito2007), or simpler skill meters (Bull, Jackson & Lancaster, Reference Bull, Jackson and Lancaster2010; Mitrovic & Martin, Reference Mitrovic and Martin2007; Weber & Brusilovsky, Reference Weber and Brusilovsky2001) or, more recently, treemaps (Hsiao, Bakalov, Brusilovsky & Koenig-Ries, Reference Hsiao, Bakalov, Brusilovsky and Koenig-Ries2011; Kump, Seifert, Beham, Lindstaedt & Ley, Reference Kump, Seifert, Beham, Lindstaedt and Ley2012). Further detail is given in Section 2.3. OLM visualisations differ from many of the currently popular learning analytics dashboards (see Verbert et al., Reference Verbert, Duval, Klerkx, Govaerts and Santos2013), in that they are based on an underlying inferred model of the learner’s current competences or understanding, rather than behaviour or performance data logged.

Figures 1–5 show some of the visualisations of the Next-OLM (Johnson, Cierniak, Hansen, Bull, Wasson, Biel & Debus, Reference Johnson, Cierniak, Hansen, Bull, Wasson, Biel and Debus2013), to illustrate different visualisations and complexity or structure of information that can be shown in an OLM. The example relates to learning English which, in this instance, can be practised in the language class (competences shown at the foot of Figure 1) or in a setting of (face-to-face or online) meeting facilitation and participation – a 21st-century skill where English is also used (top of Figure 1).

Fig. 1 Skill meters

Fig. 2 Network

Fig. 3 Radar plot

Fig. 4 Word cloud

Fig. 5 Treemap

Figure 1 shows skill meters for each competence and sub-competence: the “fuller” a skill meter, the higher the corresponding competence. Skill meters are quite common, as they are easy to interpret. Figure 2 shows the network visualisation. The larger and brighter the node, the stronger the competence indicated by that node. The network enables the structure of the competence set to be displayed in a smaller area, while the skill meters accumulate in a longer list. However, the underlying structure of the visualisation matches that for the skill meters. The radar plot in Figure 3 enables easy comparison of strengths and weaknesses across the range of competences, though another visualisation will typically be more useful to follow up on specific areas if there is a large number of competences displayed. The same is true of the network view. The word clouds in Figure 4 show stronger competences in larger (blue) text on the left, and weaker competences in larger (black) text on the right. This is especially useful for a quick overview of strengths and difficulties, but less useful for a detailed examination since competences at the borderlines of strong/weak are harder to see. In the latter case, users can follow up their exploration using a different visualisation. Figure 5 shows a treemap, which, through a layered approach where clicking on an area reveals the corresponding sub-competences, allows complex competence structures to be examined within a restricted screen space – especially applicable for MALL. It is, however, more difficult to compare different areas of the curriculum or competency set.

2.4 The Next-TELL open learner odel

The Next-TELL OLM visualisation examples in Figures 1–5 are from an independent OLM. Unlike most (independent) OLMs, the Next-TELL OLM can take data from a variety of online activities and tools, for example, as described in Sections 1.1–1.3. It has an API through which such learning data can be transferred, or self-, peer and teacher assessments can be given in cases in which data is not automatically contributed. The data from the various sources is combined using a weighted algorithm with more recent entries having greater influence as these are likely to more accurately represent current competence levels (see Johnson et al., Reference Johnson, Cierniak, Hansen, Bull, Wasson, Biel and Debus2013). The weightings may, however, be edited by the teacher. This is because the quality of the data from any source depends on the quality of the various activities and assessment methods that generated the data, and this can be best determined by the teacher. Students and teachers can use the visualisations according to their own preferences for how to view the information, in line with previous findings with several OLMs (Bull, Gakhal, Grundy, Johnson, Mabbott & Xu, Reference Bull, Gakhal, Grundy, Johnson, Mabbott and Xu2010). Figures 1–5 give the teacher’s overview of the competences of all students (“test students”, in this example), but this can also be displayed for individual learners; and learners can also view their own learner model in the same way, as shown in Figures 1–5. A restricted interface is available for smartphones, as shown in Figures 6a and 6b (using the word cloud and smiley visualisations in the Norwegian interface – see Section 4).

Fig. 6 Smartphone interface

OLMs and independent OLMs have been developed for a variety of language contexts. Examples include: overviews of the extent of understanding English grammar items (Zapata-Rivera, Hansen, Shute, Underwood & Bauer, Reference Zapata-Rivera, Hansen, Shute, Underwood and Bauer2007); pronunciation (Demmans Epp & McCalla, Reference Demmans Epp and McCalla2011); learning technical terminology (Dimitrova, Reference Dimitrova2003); historical text comprehension (Grigoriadou, Tsaganou & Cavoura, Reference Grigoriadou, Tsaganou and Cavoura2003); prompting noticing (Shahrour & Bull, Reference Shahrour and Bull2009) and awareness (Xu & Bull, Reference Xu and Bull2010) in language learning; and as a means to bring together information on student competences drawn from a variety of classroom activities (Bull, Wasson, Kickmeier-Rust, Johnson, Moe, Hansen, Meissl-Egghart & Hammermueller, Reference Bull, Wasson, Kickmeier-Rust, Johnson, Moe, Hansen, Meissl-Egghart and Hammermueller2012). Similar approaches in the foreign or second language domain are reflected in performance reports, for example, as described by Heift (Reference Heift2005). However, we here take a broader view and argue for the potential for OLMs and independent OLMs as a central focus for supporting users who are using a range of 21st-century technologies in the language curriculum. Such OLMs can, for example, combine competence data from Facebook posts, Second Life interactions and MALL, as well as data from other sources. We illustrate this in the next section.

3 Learning languages in the European context

The CEFR gives competence-based common reference levels for language learning, according to language use and abilities (what learners can do with language, rather than a knowledge-based perspective). The CEFR describes scales with reference to reception, production, interaction and mediation. Self-assessment is also important, and learners can identify their skills based on “can do” statements according to three areas: speaking (spoken interaction and spoken production); understanding (listening and reading); and writing. Thus, skills are identified as integrated, rather than separate. While the CEFR is not sufficiently detailed to define task difficulty or to design diagnostic testing items, it serves as a useful starting point (Huhta & Figueras, Reference Huhta and Figueras2004).

While our approach to integrating learning data from a range of sources is a general one, as an illustration we use the context of learning English in Norway. Using an adaptation of the CEFR, Norway has a set of national learning goals and competences which must be incorporated into the teaching and learning of English in schools (Utdanningsdirektoratet, n.d.), the details of how this should be achieved being determined at local level. The approach of the Next-TELL OLM can easily be adopted in situations like these; in more rigid teaching frameworks where the curriculum is tightly defined; and also adapted for more flexible contexts.

For our example of learning English in Norway, there are three main general areas: communication (Kommunikasjon); culture, society and literature (Kultur, samfunn og litteratur); and language learning (Språklæring). These are shown in the Norwegian interface to an individual’s OLM in Figure 7, under the list of competences (Kompetanser). These areas are allocated target competences for specific timespans in school education, after two, four, seven and ten years (Utdanningsdirektoratet, n.d.). For example, after two years of English “language learning” students should be able to give examples of: situations where it may be useful to know some English; common words and phrases of native English speakers; English words and phrases related to the student’s own interests. By the tenth year the focus has shifted to: using a range of situations and strategies for language learning; identification of similarities and differences between the native and target language, and transfer of use to the target language; use of appropriate basic grammatical and structural terminology; independent and critical use of a variety of aids; and an ability to describe and assess one’s own attempts at learning English. The “communication” target sub-competences become similarly more complex: after year two, one should be able to understand and use common language with reference to the immediate environment, understand simple instructions, and ask and answer simple questions; after year ten one should be able to use a broader range of vocabulary, understand spoken language and written texts in a range of subjects, adapt language to suit the genre and situation, write appropriately structured and argued texts, and so on. Likewise, the “culture, society and literature” competence targets become more focussed over the learning period, progressing from discussion of aspects of children’s daily lives and participation in culture and literature through words, pictures, music and movement to richer discussions of lifestyles, beliefs and cultures, explanations of aspects of history and geography in the UK and USA, discussion of literature from various genres, and creation of their own oral or written language drawing on inspiration from literature and art.

Fig. 7 Competences for Norwegians learning English, and activities contributing to the competences

Given the type of targets outlined above, and using the intermediate (four- and seven-year) target competences plus some additional ones, schools and teachers plan activities to facilitate the development of these competences. They also have to assess the extent to which the competences have been acquired. Each teacher may do this in their own way, and there may be similarities or differences across the approaches of any group of teachers. Thus, a flexible way of supporting this activity creation and assessment task, and extending the utility of the information gathered, is required. Our Next-TELL OLM offers such benefits by flexibly bringing together assessment data from different assessment/data sources, and by different teachers with potentially different approaches to assessment, and presenting this to learners and teachers.

The top half of Figure 7 shows the combined data for a sub-set of the competences. “Language learning” (språklæring) is here broken down into sub-competences (to illustrate, the first three are “describe and assess one’s progress in learning English”; “use a wide range of digital and other aids, including monolingual dictionaries, in an independent manner”; “use relevant and precise terminology to describe language form and structure”. The bottom half of Figure 7 breaks the competence levels down by activity, or data source. Thus, for example, the CLIL test has yielded lower levels of evidence for competences than the other activities, especially mobile activities.

In this illustration we see online tests following the Content and Language Integrated Learning (CLIL) approach (Bentley, Reference Bentley2010); brief e-postcards sent to learners of English in another country; more formal writing (marked by the teacher); face-to-face discussions; MALL activities; online grammar practice using OLMlets (Bull et al., Reference Bull, Jackson and Lancaster2010); online discussion; activity in Chatterdale (3DLES, 2011), a virtual world based on Second Life. As stated previously, each of these activities can provide information relevant to the competence levels, either automatically (through the API), or manually (through self-, peer or teacher assessments). While many of the activities may contribute data to many of the competences, this is not usually a straightforward mapping. For example, “use relevant and precise terminology to describe language form and structure” may more likely be observed in online interaction about language form (e.g. discussion forum, online discussion in Facebook), than in an e-postcard. It may not normally apply in a virtual world, but if such a task formed part of a specific in-world activity, this then would be expected.

To specify which competences are linked to which activities/data sources, teachers can create or select the relevant competences when they add a new activity in the OLM (see Johnson et al., Reference Johnson, Cierniak, Hansen, Bull, Wasson, Biel and Debus2013). This allows any competence (be it from CEFR; the Norwegian competence set that was adapted from CEFR; or similar; or a competence set defined by the teacher) to be linked with any activity that the teacher chooses to use. So, for example, a virtual world activity, say the context of a party, may involve competences related to language about introducing oneself and describing one’s professions and hobbies (for adult learners at early stages of learning), or may involve general communication, including understanding the language of others; a Facebook post may use informal language to describe an ongoing event; a MALL activity may use competences of using language appropriate to the current location. The latter could, of course, also be included in the virtual world and Facebook activities. Alternatively, as indicated by the arrows in Figure 7, a language-focussed online discussion in Facebook may tell us about students’ competence in describing and assessing their progress in learning English (beskrive og vurdere egen framgang i arbeidet med å laere engelsk), as well as their use and assessment of various situations, methods and strategies for learning English (utnytte og vurdere ulike situasjoner, arbeidsmåter og strategier for å laere seg engelsk). The virtual world activity may also contribute to students’ assessment of situations and strategies for learning English, as they might identify different activities or avatars with which to interact. Mobile activities enable learning to take place beyond scheduled activities, or they may be used alongside them – for example, students might look up vocabulary during a virtual world interaction (bruke et bredt utvalg digitale og andre hjelpmidler, inkludert ettspråklige ordbøker, på en selvstendig måte – use a wide variety of digital and other aids, including monolingual dictionaries independently). Thus, multiple activities and data sources can contribute to single competences, and individual activities can contribute data to multiple competences.

As stated above, the Next-TELL OLM has an API through which external environments can pass it data, for integration into the competence visualisations for learners and teachers. This may occur directly, or data (e.g. chat, discussion or other log data) may be pre-processed, for example by the ProNIFA tool (Kickmeier-Rust & Albert, Reference Kickmeier-Rust and Albert2013), which can analyse text based on teacher-constructed rules. Our Norwegian students used the OLMlets tool (Bull et al., Reference Bull, Jackson and Lancaster2010) to answer teacher-defined grammar questions (shown in the activities in Figure 7), which automatically sent learning data to the Next-TELL OLM. They also took part in a quest in a virtual world (Chatterdale – 3DLES (2011)) using text chat – also included in the activities in Figure 7. In this instance, the chat was analysed by ProNIFA, and the competences then automatically updated in the OLM. However, where such software was not available, teachers could review the chat logs to identify competence levels of the participating students, and then enter a teacher assessment using radio buttons (see Johnson et al., Reference Johnson, Cierniak, Hansen, Bull, Wasson, Biel and Debus2013), on a numbered scale or on a scale of strong–weak. The same two approaches could be applied to Facebook posts where, for example, students may have been following an event, which they discussed in a Facebook group created by the teacher. MALL may help students to continue their learning after classes, for example, using Facebook or exercises designed specifically for the MALL context. MALL activities that produce text could also be automatically analysed in the same way. While our Norwegian teachers did not provide manual assessments, their students performed self-assessments, providing an additional source of data for the learner model.