1 Introduction
Information is generally presented to students in a linear format without providing adequate cues to help them recognize the logical structure of the text, which results in cognitive load due to memory limitations to handle the abundance of information (van Gelder, Reference Van Gelder2000). According to Sweller (Reference Sweller2010) and van Gelder (Reference Van Gelder2003), cognitive load is concerned with demands placed on working memory during information processing that may prevent memory, comprehension, and critical thinking in the process of reading. It can be lessened by rendering text-based information into hierarchically visual representations (van Gelder, Reference Van Gelder2003). Similarly, findings from cognitive science have demonstrated that hierarchical representation of information improves learning (Vekiri, Reference Vekiri2002) through visual as well as propositional encoding of information in memory by processing it via more than one modality (Paivio, Reference Paivio1983). A variety of strategies for organizing information have been suggested to facilitate comprehension and recall (Farrand, Hussain & Hennessy, Reference Farrand, Hussain and Hennessy2002). One such strategy is argument mapping (Butchart et al., Reference Butchart, Forster, Gold, Bigelow, Korb, Oppy and Serrenti2009; van Gelder, Reference Van Gelder2000), which is used to diagrammatically represent the text-based structure of arguments (Oliver, Reference Oliver2009). Armbruster, Anderson and Ostertag (Reference Armbruster, Anderson and Ostertag1987) assert that by recognizing and distinguishing various structural elements of a text, students will comprehend and recall more information because they will more easily and successfully be able to identify main ideas and top-level propositions as well as central claims within a text.
However, among various graphic organizers such as semantic or concept maps, research on argument mapping as a pedagogical tool has captured less attention. Despite its significance in enhancing memory and comprehension (Dwyer, Hogan & Stewart, Reference Dwyer, Hogan and Stewart2010), the application of argument maps and recently computer-assisted argument mapping (CAAM) on learners’ recall and comprehension has rarely been explored, even in English as a foreign language (EFL) settings. Therefore, this study sought to investigate whether CAAM instruction would promote EFL majors’ comprehension, recall, and retention of argumentative texts.
Moreover, in the EFL context of Iran, a variety of teaching methods such as the grammar translation method, communicative language teaching, and audio-lingual method have been applied by state-run schools and universities. Despite a new emphasis on communicative skills, most English teachers are still keeping with their outdated methods, focusing on repetition and translation based on their pedagogical expertise and experiences. As Safari and Pourhashemi (Reference Safari and Pourhashemi2012) claim, students tend to receive English as bits and pieces of knowledge transmitted by the sole authority in the class – that is, the teacher. Most teachers even avoid using information and communication technology tools in their classes due to their lack of positive attitudes towards the application of such technologies as smart boards, videos, and computers in their classrooms, as well as the constraints of implementing the program (Safari & Sahragard, Reference Safari and Sahragard2015). In universities, English courses aim at improving learners’ reading abilities via general English or English for specific purposes (ESP). As Farhady, Hezaveh and Hedayati (Reference Farhady, Hezaveh and Hedayati2010) point out, translation methods are widely used to enable students to read and understand prose-based professional materials in English. Similarly, EFL majors are mostly concerned with summarizing, paraphrasing, and memorizing prose-based materials for comprehension and later recall. This further highlights the significance of investigating the effect of diagrammatical representation of materials (i.e. argument mapping), either by computer software or pen and paper, on EFL majors’ comprehension, recall, and retention.
2 Literature review
2.1 Applying organizational strategies to compensate for the inadequacy of prose
Information is usually presented linearly via streams of words and sentences. Similarly, arguments are generally presented as “prose”, which is considered as “the medium of philosophical argumentation” (van Gelder, Reference Van Gelder2002: 85). However, students are likely to have difficulty cognitively processing the prose for the purpose of learning and later recall (Dwyer, Reference Dwyer2011). That is, to make the central point of the main argument explicit requires serious thinking and some reading (Davies, Reference Davies2009). Harrell (Reference Harrell2005) considers one reason to be the inability of students to recognize the argument presented in the text and therefore misunderstanding it as a story instead. Furthermore, argumentative prose “contains many more sentences than just the propositions that are part of the argument, but also … proceeding necessarily linearly, the prose obscures the inferential structure of the argument” (Harrell, Reference Harrell2004: 2). Accordingly, assimilating relevant information in argumentative prose (e.g. propositions supporting a claim in an argument that are often located in different paragraphs) is problematic (van Gelder, Reference Van Gelder2003), impeding successful comprehension and recall of materials. In terms of complex arguments, students are even engaged in “circular verbal dispute or maze-like structure of forbidding volumes of prose” (Monk, Reference Monk2001: 8). Likewise, Davies (Reference Davies2012) asserts that prose is not an efficient means of transmitting complex information as a result of memory limitations.
As Meyer, Brandt and Bluth (Reference Meyer, Brandt and Bluth1980) point out, analyzing the structure of the text by identifying its organizational structure facilitates memorization of the most important points. Moreover, “additional processing of superordinate propositions and their interrelationships increases the depth with which they are processed” (Craik & Lockhart, 1972, cited in Meyer et al., Reference Meyer, Brandt and Bluth1980: 78) because it is the structure of the text that dictates how the superordinate and subordinate propositions are logically linked. Therefore, knowledge of text structure simplifies text comprehension and assists later recall (Watson, Gable, Gear & Hughes, Reference Watson, Gable, Gear and Hughes2012).
Among organizational strategies necessary for exploring the text structure and thereby promoting recall and retention of textual materials such as hierarchical summarization, other evidence-based methods including graphic organizers have been also recommended (Watson et al., Reference Watson, Gable, Gear and Hughes2012). Graphical organization of textual information converts text into visual arrays that help learners by portraying the logical relationship between key concepts of the text to provide a mental plan for comprehending (Jiang & Grabe, Reference Jiang and Grabe2007). Graphic organizers, therefore, help students develop a schema (Wittrock, Reference Wittrock1992), connect prior information to the textual information they are reading (Mayer, Reference Mayer1984), and successfully recall and remember information (Griffin, Malone & Kameenui, Reference Griffin, Malone and Kameenui1995). However, they are believed to act more effectively in the post-reading than the pre-reading stage (Griffin & Tulbert, Reference Griffin and Tulbert1995). In the post-reading stage, they are used for the purpose of evaluating students’ level of comprehension in addition to improving their recall, retention, as well as the ability to summarize key ideas (Manoli & Papadopoulou, Reference Manoli and Papadopoulou2012).
Successful recall of information requires such essential memory processes as encoding, storage, and recall of information in memory (Bernstein, Reference Bernstein2016). In the process of recall, stored information such as stimulus and event is said to be recollected by responding to an external stimulus. Recall and retention are among the main processes of memory and are related to each other as well as factors such as learning and capacity limit of memory (Amin & Malik, Reference Amin and Malik2013) based on cognitive research results. They refer to remembering the same information just at different times; that is, immediate recall and delayed retention in the present study. According to Paivio’s (Reference Paivio1983) dual coding theory, verbal stimuli, which are stored in the form of propositions in the verbal code, and visual stimuli, stored in visual code, provide two representations that increase the probability of recalling information. These verbal and visual codes together promote retention of information (Schreiber & Verdi, Reference Schreiber and Verdi2003) that is often assessed by a cued recall test at a fixed interval (Lindsey, Mozer, Cepeda & Pashler, Reference Lindsey, Mozer, Cepeda and Pashler2009).
Although the length of the retention interval is usually taken as a robust factor in retention success (e.g. Bankó & Vidnyánszky, Reference Bankó and Vidnyánszky2010), it can be affected by factors other than time per se, including situational factors as well as the task itself (Naylor & Briggs, 1961, cited in Arthur, Bennett, Stanush & McNelly, Reference Arthur, Bennett, Stanush and McNelly1998). Moreover, increasing the length of retention intervals makes it more educationally realistic in comparison with short laboratory intervals of hours or a few days (Richland, Linn & Bjork, Reference Richland, Linn and Bjork2007). As Cepeda et al. (2006) recognized in their meta-analysis, most studies have examined retention intervals in laboratory learning environments and failed to assess performance on tests of recall after delays of weeks, months, and years; that is, “across educationally realistic retention intervals” (Richland et al., Reference Richland, Linn and Bjork2007: 264).
In addition to applying organizational strategies for rendering text to diagrams, Dwyer et al. (Reference Dwyer, Hogan and Stewart2010) suggest that if propositions within a complex argument are represented in reasonably small chunks, the subsequent encoding and recall of those arguments will turn out to be more successful. In essence, chunking is central to the concept of short-term memory capacity (Gobet & Clarkson, Reference Gobet and Clarkson2004). Anderson (Reference Anderson1983) claimed that working memory can sometimes include over 20 active chunks at one time. However, further research indicated that people cannot process and remember more than seven plus or minus two (a “magic number”) pieces of information, which is known as the chunking limit (Miller, Reference Miller1956). Revising Miller’s magical number, Cowan (Reference Cowan2001: 87) asserted that it “was meant more as a rough estimate and a rhetorical device than as a real capacity limit”. He estimated that the chunk size would probably include three to four items. Duration rate for the items in working memory was regarded as another factor affecting its content irretrievability capacity (Peterson & Peterson, 1959). Cognitive scientists, therefore, considered chunks (i.e. β=a number from two to six), items (i.e. γ=from three to four), and their retrievable duration (δ=measured by seconds) as three necessary quantities in calculating the working memory irretrievability capacity and arranged them in a formula called alpha value (i.e. α=(β * γ) * δ). Accordingly, in their research, Gobet and Clarkson (Reference Gobet and Clarkson2004) asserted that short-term memory capacity, as estimated by the number of recalled chunks, was between three and 15 chunks at all skill levels. However, for those with higher capacities, the number of chunks is less than three; the number of items per chunk can increase to 15. Thus they concluded that our memory capacity may consist of around 30 items based on the alpha-value formula (i.e. two chunks, each containing 15 items).
Notably, research on the effects of converting text-based information into diagrams and maps shows that diagramming not only reduces the cognitive load (Pollock, Chandler & Sweller, Reference Pollock, Chandler and Sweller2002) but also positively affects recall and comprehension (Berkowitz, Reference Berkowitz1986; Oliver, Reference Oliver2009). In a six-week study, Berkowitz (Reference Berkowitz1986) explored the effect of mapping instruction on reading comprehension of six-grade students. He assigned the participants (n=99) into four experimental groups (i.e. map construction, map study, question-answering, text rereading). A free recall test as well as a short-answer test were then administered immediately after the study and again two weeks later. The results showed that the map construction group gained higher scores compared to other groups, suggesting that map construction facilitates recall when students construct the maps themselves. In another study conducted by Suzuki, Sato and Awazu (Reference Suzuki, Sato and Awazu2008), the effectiveness of spatial graphical depiction of a sentence compared to linear sentential presentation was explored that suggested that graphical representation of textual information enhances ESL readers’ comprehension of sentences.
2.2 Argument mapping with pen and paper as well as computer
Due to difficulties learners have in making the highly abstract relationships in an argumentative prose explicit, argument mapping was introduced. Davies (Reference Davies2011) refers to the justification and purpose of using mapping tools in educational settings, suggesting that mapping increases the chance that meaningful learning (as opposed to rote learning) will occur. New information can be built on prior knowledge, thereby becoming more useable. Furthermore, mapping greatly improves recall and processing of information as well as meaningful engagement for learners. When students manage to represent complex textual information in a map, they are likely to have understood the relationship between ideas, as well as recalling and analyzing various propositions. Moreover, by asking students to construct maps, teachers are provided with helpful insights as to the students’ mental model of the given arguments (Butchart et al., Reference Butchart, Forster, Gold, Bigelow, Korb, Oppy and Serrenti2009).
As graphical organizers, argument maps provide a manageable representation of the structure of the sophisticated and ill-structured arguments that many people may find otherwise difficult to understand. Argument maps draw upon theories of visual and diagrammatic reasoning that explicitly depict the inferential structure of an argument (van Gelder, Reference Van Gelder2005). Arguments are suggested to be handled through argument mapping as they are demonstrated in “streams of words, whether written or spoken” (van Gelder, Reference Van Gelder2005: 44). Boxes and arrows are normally applied in diagramming arguments in which propositions such as reasons and objections are displayed within the boxes and the evidential relations among them are indicated via arrows (van Gelder, Reference Van Gelder2002; see Figure 1).
Figure 1 Example of an argument map created by Rationale (adapted from https://www.rationaleonline.com/)
Moreover, as a result of advances in computer technology, a number of software packages (e.g., Reason!Able, Araucaria, Athena, and recently Rationale) have recently been introduced in educational settings (van Gelder, Reference Van Gelder2007) in addition to traditional pen-and-paper methods of argument construction. CAAM is concerned with the use of a computerized program specifically designed to assist in the design of a structured argument through using concept structures and their relationships for the reasons of usability, complementation, and semi-formality (van Gelder, Reference Van Gelder2007). One such software is Rationale, developed by Austhink Company, which applies color coding to differentiate between various propositions of an argument (e.g. green for reasons and red for objections – the boxes labeled “because” and “but”, respectively, in Figure 1) as well as visual and spatial arrangement of textual information.
Applied research on the impact of CAAM on enhancing memory processes is limited. The only example at the time of writing this article includes Dwyer et al.’s (2010) experiment that compared the effect of prose reading versus argument mapping reading on psychology students’ memory and comprehension abilities. They assigned the psychology students (n=400) into six experimental groups including (1) a 30-proposition text, (2) a 30-proposition color argument map, (3) a 30-proposition monochrome argument map, (4) a 50-proposition text, (5) a 50-proposition color argument map, and (6) a 50-proposition monochrome argument map. The study took place in just three sessions, and the participants received no instruction on constructing maps of arguments but were just given a short lecture on argument mapping. Their comprehension was tested by asking whether a subset of propositions support or deny the main contention of a given claim. Moreover, they were asked to complete a fill-in-the blank test of recall. The results revealed that all experimental groups were similar in terms of comprehension; however, performance on the recall test was better for the 30-proposition group compared to the 50-proposition group. Students in both black-and-white and color map conditions had better memory performance than the text-only condition, although color map groups outperformed the monochrome groups significantly. The researchers therefore concluded that argument mapping enhances memory for arguments significantly when compared to pen-and-paper methods of argument presentation. However, their research took place within three weeks with no longitudinal CAAM training.
In such a context, several studies have explored the effects of applying graphic organizers such as semantic mapping and concept mapping on enhancing students’ reading comprehension (Asadollahfam & Shiri, Reference Asadollahfam and Shiri2013; Sadeghi & Taghavi, Reference Sadeghi and Taghavi2014). The results are promising, suggesting a positive correlation between training in mapping skills and the development of recall and comprehension. For example, Asadollahfam and Shiri (Reference Asadollahfam and Shiri2013) investigated the impact of training semantic mapping on EFL learners’ reading comprehension performance. Using a pre-/post-test control group design, their results revealed that semantic mapping instruction enhanced EFL learners’ reading comprehension of expository texts compared to the conventional reading group. However, at the time of writing, very few published studies concerning the impact of argument mapping instruction and reading in general and CAAM in particular on the development of EFL learners’ comprehension, recall, and retention has been reported. Therefore, this study contributes to the literature by addressing the impact of pen-and-paper versus computer-assisted argument mapping on EFL learners’ recall, retention, and comprehension skills after receiving a semester-long instruction on argument mapping.
This study investigated the following research questions:
1. Does type of treatment (computer vs. pen and paper) have any significant effect on the performance of EFL majors on tests of comprehension, recall, and retention while controlling for possible effects of spatial and verbal intelligences?
2. Does argument size (i.e. maps with differing numbers of propositions) have any significant effect on the performance of the participants on tests of comprehension, recall, and retention while controlling for possible effects of spatial and verbal intelligences?
3. Do proficiency levels have any significant effect on the performance of the participants on tests of comprehension, recall, and retention while controlling for possible effects of spatial and verbal intelligences?
3 Materials and methods
3.1 Participants
Participants included a sample of 120 EFL undergraduates, aged 18–27, who studied English translation at Payame Nur University in Aran and Bidgol (a city in the province of Isfahan), Iran. Almost one month before the commencement of the study, posters were distributed around the university campus, announcing and recruiting EFL majors for two extracurriculum reading classes. Initially, 139 students enrolled and took part in the classes, but 19 students were excluded because they failed to attend some of the training sessions (n=4), missed one of the tests (n=5), or were at an intermediate level of proficiency (n=10). They were randomly assigned to two experimental groups: the CAAM group were taught argument mapping via the Rationale software, and the conventional group practiced it via pen and paper. All 120 students were aware of the purpose of conducting the study and their contribution to the development of the project. Moreover, they were categorized into high and low levels of reading proficiency on the basis of their scores on a sample First Certificate in English (FCE) reading test.
3.2 Instruments
Several quantitative instruments were applied in the present study. First, a sample FCE reading test was used to assess the participants’ level of reading proficiency. The test includes three parts (multiple-choice, jumble-ordered, multiple-matching questions) comprising 30 questions and takes 60 minutes to complete. Since all the participants were to take part in a reading course infused with argument mapping or a pure reading course for the second experimental group, only the reading section of FCE was administered. As the distribution of FCE scores was not significantly different from the normal distribution, classifying the participants into high and low levels of proficiency was done based on the mean and standard deviation (SD) of their scores. The high proficiency group (n=60) included those scoring +1 SD above the mean, and the low proficiency group (n=60) consisted of those falling –1 SD below the mean; the intermediate group (n=10) was excluded from the study.
A software package for diagramming reasoning and mapping arguments, called Rationale (van Gelder Reference Van Gelder2007), developed by Austhink, was also used in this study. As van Gelder (Reference Van Gelder2007) suggests, it promotes reasoning because it is more useable for reasoning activities compared to prose. Furthermore, it is designed to enhance the brain’s ability to comprehend reasoning by complementing what it could already do imperfectly. It also aims to link the brain’s natural informality with the semi-formality of structured maps (Davies, Reference Davies2009). In this software package, information is displayed in color-coded boxes to distinguish among the central claim, reasons, objections, and rebuttals. Moreover, the strength of arguments can also be judged using the Evaluation section in the toolbar of the Rationale editor page. Some indicator phrases (e.g., because, however) are employed to determine the reasoning structure.
Another instrument included a sample of verbal and spatial subtests from the Differential Aptitude Tests (DAT; Bennett, Seashore & Wesman, Reference Bennett, Seashore and Wesman1986); the results entered the analysis as covariates. The reason for this was to see whether participants’ verbal and spatial reasoning abilities accounted for the differences in their comprehension, recall, and retention performances, as the reading and construction of argument maps is highly interrelated with the ability to reason both verbally and spatially (Dwyer et al., Reference Dwyer, Hogan and Stewart2010). Therefore, the chances are that students with various verbal and spatial reasoning ability levels approach argument map reading tasks in different ways. The verbal reasoning subtest of the DAT contains 40 tests that measure analogical reasoning by providing sentences with missing words completed through selecting from five word-pairs. The spatial reasoning subtest consists of 50 items that ask participants to visualize a three-dimensional object from a two-dimensional pattern. To make the object, they need to mentally fold cut-outs, and then choose the correct one among the four choices provided (Dwyer et al., Reference Dwyer, Hogan and Stewart2010).
The reading skill measures included tests of comprehension, recall, and retention. The cued recall test consisted of fill-in-the-blank questions aiming to assess participants’ recall for reasons and objections confirming or refuting the central claim of the arguments. It consisted of a set of 10 statements (regardless of the size of the argument selected) derived from the original study materials. The cue in each statement was specified by such conjunctions as because or but, followed by a set of blank lines. The comprehension test demanded understanding of the relationships between propositions in the argument structure. Participants were required to indicate if a subset of 10 of the propositions extracted from argument maps either supports (i.e. reason) or refutes (i.e. objection) the central claim. The same recall test developed by the researcher was applied as the retention test within a two-week interval after the first administration. This time interval was selected as Mackey and Gass (Reference Mackey and Gass2005: 149) recommend: “Often this is 1 week following the first posttest and then 2 weeks later and even 2 or 3 months later”.
3.3 Procedure
The study took place over 12 sessions, each lasting almost two hours. In session 1, 120 EFL majors took the reading section of a sample FCE test to be divided into low and high levels of reading proficiency based on their scores. They were also randomly assigned to the first experimental group (i.e. software group) or the second experimental group (i.e. pen-and-paper group), each having 60 students (i.e. 30 from the low and 30 from the high reading proficiency groups). In the second session, the spatial and verbal subtests of the DAT were administered. The course actually began in the third session, and lectures as well as classroom activities were presented to the two experimental groups. The content of the training materials was the same for the two groups except that in-class handouts and exercises differed based on the conditions of Rationale or pen-and-paper presentation. In this session, technical argument mapping terms were introduced, and the two groups were instructed how to use the software or pen and paper for making maps of the argument. A rudimentary familiarity with computers was adequate for working with the software. From sessions 4 to 10, mapping skills and the way to apply such skills through the examples and exercises when reading and interpreting arguments of various topics were explained. They were also shown how to extract the structure of the arguments, judge their logical strength, relevance, and credibility, and recognize the sources of arguments. In the 11th week, the study materials, which included two maps with differing numbers of propositions from an argument called “Advertising: Good or bad?”, were randomly presented to high and low proficiency students via computer for the software group and via paper for the pen-and-paper group. The small version of the article containing 30 propositions was developed based on Gobet and Clarkson’s (Reference Gobet and Clarkson2004) fresh working memory capacity estimate. The large version included the total number of constituent propositions of the study material (i.e. 45). As students in both experimental groups had already been classified into high (n=60) and low (n=60) reading proficiency levels, 30 students at high proficiency levels in both groups (n=15 in each group) received large argument maps, whereas 30 students at low proficiency levels in both groups were randomly given small argument maps. Students were allotted almost 15 minutes to read the maps and were told that they would be tested. Then, the argument maps were collected and relevant cued-recall tests were administered with 15 minutes given for completion. Both groups were then provided with the comprehension tests, which were administered similar to the recall test, again with 15 minutes allotted to the test completion. To check retention, the same cued-recall test was administered to the students in both experimental groups 14 days after the immediate post-test.
4 Results
Using an alpha level of .05, a three-way between-subjects MANCOVA (which enjoyed normality of distribution) was used to investigate the impact of types of treatment, argument size, and level of language proficiency on the participants’ comprehension, recall, and retention of argument. Before using the MANCOVA, preliminary analysis of data revealed that all MANCOVA assumptions were met. Table 1 lists means and standard deviations for comprehension, recall, and retention.
Table 1 Descriptive statistics by groups controlling for visual and verbal tests
a Covariates appearing in the model are evaluated at the following values: verbal=21.99, spatial=18.95.
The results suggest that the software group (M=27.33) significantly outperformed the pen-and-paper (M=22.24) group on the recall test, F(1, 110)=175.86, p<.05, partial η2=.61, retention test, F(1, 110)=64.09, p<.05, partial η2=.36, and comprehension test, F(1, 110)=171.33, p<.05, partial η2=.60. Moreover, students gained higher mean scores on comprehension as opposed to recall and retention tests.
In the case of argument size, the results revealed that there was no significant difference between the small argument size group (M=24.63) and the large argument size group (M=24.94) on the recall test, F(1, 110)=.63, p<.05, partial η2=.006, the retention test, F(1, 110)=.98, p<.05, partial η2=.009, or the comprehension test, F(1, 110)=.52, p<.05, partial η2=.005; see Table 2. The results further suggested a lack of significant difference between low (M=24.72) and high (M=24.86) proficiency groups on the recall test, F(1, 110)=.13, p>.05, partial η2=.0010, retention test, F(1, 110)=1.009, p>.05, partial η2=.009, and comprehension test, F(1, 110)=.81, p>.05, partial η2=.007. Moreover, no significant interaction effect was found. To sum up, only type of treatment had a significant effect on recall, retention, and comprehension after controlling for spatial and verbal covariates; however, proficiency level and argument size did not show any significant effect.
Table 2 Tests of between-subjects effects, analysis by group * argument size * level
5 Discussion and conclusion
Findings from this experiment suggest that presenting arguments via software (in this case, Rationale) helps students in better comprehension, recall, and retention of materials compared to using pen and paper. In fact, in the presence of significant correlation between the covariates (i.e. verbal and spatial reasoning) with the dependent variables (i.e. tests of comprehension, recall, and retention), argument map reading via computer significantly supported subsequent comprehension, recall, and retention of study materials despite similar presentation of arguments via pen and paper. Notably, the significant correlation between variables and the covariates may be attributed to the fact that participants with higher levels of verbal reasoning ability have recalled and apprehended study materials for a particular period of time so as to respond appropriately to the recall and comprehension questions by assimilating the relevant propositions within arguments (Colom, Abad, Rebollo & Shih, Reference Colom, Abad, Rebollo and Shih2005). Apparently, students have successfully applied their spatial reasoning ability in assimilating study materials irrespective of their interest or familiarity with certain topics (Dwyer et al., Reference Dwyer, Hogan and Stewart2010).
Notably, the advent of modern argument mapping software packages such as Rationale, which applies color coding to differentiate between various propositions of an argument (e.g. green for reasons and red for objections – the boxes labeled “because” and “but”, respectively, in Figure 1) as well as visual and spatial arrangement of textual information, is compatible with Gestalt grouping principles. It refers to the application of Gestalt grouping laws of similarity and proximity proposed by Wertheimer (1950, cited in Peterson & Berryhill, Reference Peterson and Berryhill2013) to the way learning materials are grouped in working memory. Similarity refers to grouping of items with similar features, such as color and shape, whereas proximity deals with the grouping of nearby categories. Research shows that general configuration of information in terms of proximity and similarity affects storage of information in working memory (Jiang, Oslon & Chun, Reference Jiang, Olson and Chun2000). That is, items grouped together either due to similarity or proximity appear to be stored in visual working memory (Woodman, Vecera & Luck, Reference Woodman, Vecera and Luck2003) and are thereby easier to recall later. As Dwyer (Reference Dwyer2011) puts it, Rationale not only facilitates grouping of prose-based materials but also increases the capacity of visual working memory.
The better performance of the software group is therefore due to the usability of the software package (van Gelder, Reference Van Gelder2007), which facilitates the construction and arrangement of propositions more rapidly than the pen-and-paper method. Thus, it leaves more time for assimilating and comprehending the structure of the arguments and makes the subsequent recall of information easier. Furthermore, the skillfulness in the reading of argument maps via computer may also facilitate the application of chunking strategies by the students, thereby resulting in better recall and comprehension of propositions (the aim of using the retention test was to check whether this advantage would remain over time). It definitely highlights the role of technology in enhancing EFL learners’ reading comprehension skills (Stearns, Reference Stearns2012).
There are several possible explanations why, in this study, students in the software group gained higher mean scores on tests of comprehension than of recall and retention. Empirical research indicates that acquiring a good mental representation of the spatial layout of a text, which is facilitated via computer working space, supports comprehension of the text (Cataldo & Oakhill, Reference Cataldo and Oakhill2000). That is, the better performance of the software group could be due to such factors as the ability to zoom the text in/out, document navigation via scrolling, rapidly searching and locating the keywords of the test items in the document, and better accessing and retrieving the essential pieces of information for in-depth comprehension, hence detecting the relation between the spatial reconstruction of the text and reading comprehension. In fact, it seems that the fixity of the argument map on paper hampered the construction of the physical representation of the document for map comprehension. This is in contrast with Mangen, Walgermo and Brønnick’s (Reference Mangen, Walgermo and Brønnick2013) claim that fixity of the printed text and its fixed spatial cues support the readers’ mental representation of the text and improve comprehension compared with computer-mediated text. Moreover, when the reading time is accounted for (as in the present study), students seem to read the maps more rapidly and therefore are less efficient at recalling the information they have read from the computer, leading to lower recall mean scores.
Retention also demonstrated the smallest mean score difference between the two groups, as research findings indicate that visual information when followed by relevant verbal information can improve retention (Schreiber & Verdi, Reference Schreiber and Verdi2003). That is, both software and pen-and-paper groups received visual information (i.e. argument maps) along with verbal information (i.e. propositions within the boxes) resulting in their similar performance on test of retention. This supports Paivio’s (Reference Paivio1983) dual coding theory that states that these cognitive joints promote retention of information.
As another finding, the size of the arguments had no significant effect on students’ performance on tests of comprehension, recall, and retention. In other words, when receiving small and large argument sizes that both supported and refuted some claims, participants tended to perform in a similar fashion, although the large argument size group gained slightly higher mean scores on tests of recall (M=24.94) and retention (M=19.97) compared to the small argument size group (recall, M=24.63; retention, M=19.55). This finding is surprising, as prior research highlights the limited capacity of working memory and the fact that overburdening memory by asking students to recall too many propositions would result in cognitive load and poor recall (e.g. Pollock et al., Reference Pollock, Chandler and Sweller2002). Accordingly, those who received small arguments should have outperformed the large argument size groups on tests of recall and retention. However, the findings of this study demonstrated that recalling 10 target propositions from a small argument map (30 propositions) was not much easier than remembering them out of a large (i.e. 45 propositions) argument map.
One possible reason might be the small sample size, which was reduced to 15 students in each group after randomly assigning the participants to the final subgroups. Another reason might be the time constraint for those receiving large argument sizes who had to assimilate more propositions than the group who received an argument map with fewer propositions within the same time interval. This becomes even worse when the topic of the study material (i.e. advertising) is not of interest to the students, which might then require more time to assimilate. In a similar study conducted by Dwyer (Reference Dwyer2011), smaller argument groups outperformed larger argument groups on tests of recall. As suggested by Dwyer et al. (Reference Dwyer, Hogan and Stewart2013: 21) the findings demonstrate that “there is a threshold in terms of the number of propositions that can be reasonably assimilated in a short space of time”.
This finding also partly lends support to the psychological methods of reading comprehension that encourage students to read large instead of small passages. Such methods, in fact, reject the intuitive belief that small passages are easier to comprehend than large ones (Mehrpour & Riazi, Reference Mehrpour and Riazi2004). One related implication is that longer texts provide the reader with more text-based information that facilitates their comprehension of the content. This position was supported by Feldmann and Stammer (1987: 255) who claimed that “the more clues the learners are able to pick up, because of the natural tendency of a text, the more developed is their foreign language competence and the better they will accomplish the task”. Accordingly, large argument maps constructed from larger argumentative texts can be easier to assimilate and recall as well.
The present results further revealed the non-significant effect of level of reading proficiency on tests of comprehension, recall, and retention. Whereas high proficiency groups scored slightly higher (though not significant) on tests of recall and retention, the low proficiency group outperformed the high group on tests of comprehension in the overall analysis. One reason may be that the high proficiency group focused more on grasping information via memorizing propositions, hence gaining higher recall and retention scores. Their better performance on the retention test also shows how well the study materials were encoded into their long-term memory (Tulving, 1984). Moreover, they might have guessed the answer to the comprehension questions that asked whether the given proposition supported or refuted the central claim (Dwyer, Reference Dwyer2011) due to taking their time for memorization. However, the low proficiency group might have focused on comprehending and assimilating the relational associations among propositions and thereby gained higher mean comprehension scores. As Dwyer (Reference Dwyer2011) suggests, remembering a certain amount of information may occur despite lacking a true comprehension of the logical associations between propositions in an argument. This may even be enhanced by argument mapping since it isolates each proposition in distinct boxes, which in turn enhances subsequent recall and comprehension.
Despite the lack of significant difference between high and low reading proficiency level groups, it can be concluded that on average the high proficiency groups benefited more from reading CAAM when answering recall and retention questions (gaining higher mean scores), implying that argument mapping might be a better learning strategy for students at higher levels of proficiency. This idea can be explained since knowledge of text structure enhanced through the map “facilitates understanding of text and, consequently, a student’s recall of what has been read improves” (Watson et al., Reference Watson, Gable, Gear and Hughes2012: 82). This finding supports the results of previous research that highlights better performance of students at higher levels on tests of recall (e.g. Keshavarz, Atai & Ahmadi, Reference Keshavarz, Atai and Ahmadi2007; Manoli & Papadopoulou, Reference Manoli and Papadopoulou2012). Moreover, research findings suggest that optimal performance on tests of comprehension and memory requires initial reading to understand the text and then rereading to memorize it (Pollock et al., Reference Pollock, Chandler and Sweller2002). Thus it is possible that the time devoted to reading argument maps before administering relevant recall and comprehension questions (15 to 20 minutes) might not be adequate, leaving less time for the students at low levels to reread the maps and consequently better recall the information. Similarly, it is likely that high-level students’ lack of interest in, or familiarity with, the topic of the study material (i.e. advertising) led to their lack of motivation to read and answer the comprehension questions carefully.
Unfortunately, the literature fails to acknowledge the impact of the level of proficiency on comprehension, recall, and retention of argument maps. However, this finding contradicts the results of previous research that highlight the positive impact of the level of reading proficiency on learners’ general recall and comprehension ability with better performance of students at higher levels (e.g. Manoli & Papadopoulou, Reference Manoli and Papadopoulou2012).
In summary, the results demonstrated that training in argument map construction and reading via software significantly augments students’ comprehension, recall, and retention ability within a limited space of time, confirming that it is a beneficial teaching methodology. The results, therefore, were congruent with similar prior studies that highlighted the potential of argument mapping as a helpful strategy for enhancing memory and comprehension (e.g. Berkowitz, Reference Berkowitz1986; Oliver, Reference Oliver2009). However, its superiority seems not to hold across the level of reading proficiency or the size of argument maps. Further research is required to investigate the impact of argument mapping on other important variables such as setting, time interval, and other tests of recall (e.g. free recall and serial recall).
Given the findings of the present study, teachers can enhance EFL majors’ ability to comprehend and recall argumentative texts by presenting them hierarchically with the help of computer technology since it helps students gain a deeper understanding of the materials (Stearns, Reference Stearns2012). For instance, EFL teachers can use it as a teaching aid, which supports text-based (prose) presentation of arguments. This can provide students with more opportunities to deeply understand the structure of the reasoning behind the argument by grasping the associations among propositions and judging the credibility as well as logical strength among them, which in turn will lead to better analysis and evaluation of the arguments and subsequent success in their comprehension and recall. In addition, by reducing the cognitive load imposed on memory as well as constructing stronger memory links than text alone, the software can motivate students to become more engaged in deliberate practice (van Gelder, Reference Van Gelder2005). Furthermore, curriculum developers and course planners should be invited to consider educational techniques that focus more on applying computer-assisted technology in EFL settings, which in turn makes language teaching more progressive (Dudeney, Reference Dudeney2007) and amenable to students’ learning requirements.
The findings of the present study might serve as the basis for future research on the application of argument mapping software in various educational settings. Although the impact of verbal and spatial reasoning as factors affecting recall and comprehension was controlled in the present study, future research can address other factors such as students’ motivation or familiarity with topic or novelty of instructional method (i.e. the software, Dwyer, 2011), which could affect students’ performance on tests of recall and comprehension.
About the authors
Maryam Eftekhari holds a PhD in TEFL from Alzahra University. She is an assistant professor in the English Department of Islamic Azad University, Kashan Branch, where she teaches various English courses. Some of her current interests include CALL, psycholinguistics, and teaching skills.
Elaheh Sotoudehnama holds a PhD in TEFL and is a full professor in the English Department of Alzahra University in Iran, where she has been teaching for more than 20 years. Her area of interest is teaching skills in general and language learning strategies, culture, and motivation specifically.
