For many faculty members, the move by universities toward greater reliance on online and hybrid courses raises questions of course content and quality. Many faculty, socialized to deliver information to students in a manner described as “chalk and talk,” wonder whether moving courses out into the online environment somehow compromises the education students receive. In fact, there is a small but growing body of information in political science and other cognate disciplines that addresses these questions and offers evidence that online courses, while different from more traditional face-to-face classes, can offer rich learning opportunities in their own right. This article seeks to contribute to that body of knowledge by reporting the results of a quasi-experiment comparing student success and satisfaction in online and face-to-face courses.
During the 2005–2006 academic year, supported by a grant from my university's Center for Instructional and Professional Development (CIPD), I had the unique opportunity to simultaneously teach an online and a face-to-face section of Introduction to American Government and Politics in each semester. In an effort to extend this teaching opportunity into a related research opportunity, I structured the course with an eye toward collecting data that might allow me to examine whether and how the mode of instruction influenced student performance and satisfaction with the class.
As with any new technology, incorporating online course components into university classes offers faculty a dizzying array of options. Textbook web sites, web-based quizzes and data-gathering exercises, hybrid or reduced seat courses, and courses offered fully online bring to the classroom expanded opportunities for the facilitation of student learning. However, the course that is substantially or completely online differs in some fundamental ways from courses that incorporate online components but maintain the traditional face-to-face contact and dialogue between student and instructor. This lack of physical, face-to-face contact and interaction raises questions about whether education in cyberspace can be as effective. To this point, much of the evidence suggests that the answer to this question is “yes.” Of course, as with any classroom situation, “success” is influenced by many variables—the abilities, interests, and motivations of students and instructor; the interaction of the unique mix of people brought together in the course; related events going on in the world outside the classroom; etc. And, it must be acknowledged that technological innovations are a means to an end and how well they are used depends on how they are integrated into a course.
Taking all of these things into account, the evidence suggests that the well-designed course with online components can be just as successful and satisfying as the more traditional face-to-face offerings. For example, Clawson, Deen, and Oxley (2002) found success in using online discussions that linked students at three universities across the country, noting that students ranked online discussions as one of the most valuable aspects of the course for developing communications skills and learning to apply course knowledge to real-world situations. Pollock and Wilson (2002) report on a comparison between traditional face-to-face courses and a reduced-seat time (RST) section in which the course met both face-to-face and online. They find that students in the RST section had higher levels of satisfaction with the course and scored better on a political knowledge index than students in the face-to face-course. While, from a pedagogical standpoint, student satisfaction isn't necessarily the be all and end all, Pollock and Wilson suggest that the RST students expressed a greater willingness to pursue related course work in the future. Interestingly, in an extension of this project, Wilson, Pollock, and Hamann (2006) found that women in particular benefited from RST courses, gaining computer literacy skills on par with their male peers and far outstripping men on measures of attention to politics. This and other evidence of a lack of gender difference in online activity is important, given the still fairly common, but increasingly inaccurate, assumption that the online environment is dominated by men and works to disadvantage women (Ono and Zavodny 2003; Weiser 2000). Another study comparing online and face-to-face political science courses found that students in the online courses did the same or slightly better than students in face-to-face courses on several indicators of success (Botsch and Botsch 2001). Finally, a recent study of online introductory economics courses at the University of Akron found that the online students performed as well as face-to-face students in complex tasks and outperformed them at lower levels of learning (Myers and Nelson 2004). So, at this point in the evolution of our ability to evaluate online courses, most of the available evidence points to these courses being at least as successful as face-to-face courses.
Data and Methods
Introduction to American Government and Politics is a 100-level lecture course with between 100 and 250 students in a section and covers the traditional structural, institutional, and behavioral aspects of the American political system. I created the online section of this course for the first time in fall of 2005 for 15 students and taught it again in spring 2006 with 25 students. Since I was teaching it simultaneously with the face-to-face section, I wanted to keep as many elements of the two courses as comparable as possible but still take advantage of the different opportunities that the online environment provided. I taught a face-to-face and online section in each of the fall and spring semesters, so the data reported here are based on these four sections of the course.
One important element of this examination is that I had no control over who registered for the face-to-face lecture course and who registered for the online section. The choice of which section to take was completely up to the students. My university places no restrictions or qualifications on who can take an online course, so that section was available to all students. It is generally the assumption that online courses are most popular with nontraditional students who might be working full-time or live a great distance from a university campus. However, here at UWM we have seen a strong interest in online class among the traditional student population. These students may take several face-to-face classes on campus and supplement their registration with one or more online classes to attain the benefits of flexible scheduling, which may facilitate work commitments, child care arrangements, or commuting time/costs. I had to evaluate the two different sections of the course with the knowledge of this self-selection, since a particular type of student may be more likely to take online courses. For example, Botsch and Botsch (2001) found that students registered for their online courses were older, more knowledgeable about politics, and from a different array of majors than those who registered for face-to-face lecture courses. On the other hand, Pollock and Wilson (2002) found that the students in their RST sections had less academic experience, lower levels of interest in the class, and less experience with online courses than those who enrolled in the face-to-face lecture course. So, while conventional wisdom assumes that online classes serve more non-traditional students, it may be that any differences between the populations of the two types of courses are dependent, in part, on the culture of a particular university and/or the vagaries of any given semester's registration process.
Students in the face-to-face section of the course were required to attend a lecture twice a week, take three exams and three quizzes over the course of the semester, and do a web-based writing assignment that was tied to the course textbook (We The People by Ginsberg, Lowi, and Weir 2005). Students in the online section were responsible for the same content, although it was delivered online in the form of a written “script” of the lecture they would have heard if they were present in the face-to-face course. The online students accessed lecture materials each week and were responsible for posting an answer to a weekly discussion question that I posed on the course web site. They also took the same three exams and did the same web-based writing assignment as the students in the face-to-face section.
The similarities between the two sections were the textbook, the lectures, the instructor, the time period, the exams, and the writing assignment. Also, whenever I would make reference to some occurrence in the “real world” in the lecture course, I would post the same information to the web site for online course so that each section would be exposed to these course/real world linkages. There were two major differences between the sections, both probably important. First, the face-to-face students were responsible for the quality of their lecture notes, based on their presence at lecture and their own note-taking ability. In class, I provided an outline of the lecture as I spoke and then made that outline available to the face-to-face students on the course web site for the their section. The online students, on the other hand, had a more complete set of “notes” in the form of the script of my lecture. I would imagine that this could have provided the online students with an advantage when studying for exams. Second, instead of taking the same written quizzes that the face to face students did, and in order to take advantage of the ability to more easily facilitate (or require, depending on your perspective) student discussion and interaction, the online students were required to post answers to weekly discussion questions and to respond several times over the course of the semester to other students' postings. This difference between the courses could have a significant effect on student learning. As anyone who has taught a course with more than 50 students can attest, stimulating or even requiring ongoing discussion is very difficult. Each of the face-to-face sections I taught during the year had over 100 students. Because of this, discussion was limited to the questions I would ask the students and the responses I would get from the same 10–12 students all semester. One of the frustrations of teaching large enrollment courses is the limited ability faculty have to engage students in ongoing discussion. So, to the degree that taking part in this sort of back and forth is helpful to student learning, the students in the face-to-face section were at a bit of a disadvantage, since the vast majority of them were unwilling or unable to take part in these attempts. But, as some would argue, one of the advantages of the online environment is the ability to lead (force?) students into taking part in discussions and responding to teacher and student questions on an ongoing basis. Taking part in these discussions by formulating their own answers and considering and responding to other students' answers could offer the online students a way to deepen and reinforce their understanding of key concepts in the course.
The other obvious difference, class size, can be related to both of my dependent variables—student performance and satisfaction—raising the question of whether any differences between the two sections are related more to mode of instruction or to class size. Since I did not have any online sections with large enrollments, I cannot control for this in the analysis. However, online courses may allow instructors more flexibility to create discussion and smaller learning communities than face-to-face courses. For example, with something comparable to the Desire 2 Learn software used at UWM, an instructor with an online course with 100 students could very easily create five groups of 20 students (or any number of groups) to encourage interaction between the students and the instructor. This is more difficult to achieve in a larger face-to-face class. So, depending on the way an online course is structured, the mode may allow instructors to more easily overcome the challenges of large student enrollments.
My primary research question was whether the mode of instruction would have a significant impact on the success students had in mastering the material and on their levels of satisfaction with the class. However, there are many factors that can have an impact on how students perform in a course. In an effort to obtain relevant information that would be useful in the analysis, I gave students in each section a brief survey on the first day of the semester. The survey asked students to provide their year in school, grade point average, reason for taking the course, sex, and age.1
For a variety of reasons, I did not include substantive questions about American government on this survey. In hindsight, doing so might have allowed me to control for the students' knowledge of previous information in the model on performance. However, given the demographic similarities between the students in the two sections, there is no clear reason to expect that students in one section would necessarily know more about the subject matter before taking this class than students in the other.
Analysis
The first thing to consider before analyzing whether the mode of instruction had a significant impact on student performance is whether there were significant differences between the students in each type of class from the start. In the research reported here, there were no differences between the students in the two sections with regard to sex, grade point average, or reason for taking the course. The average grade point for students in both sections was a C+ and the majority of them were in the course to fulfill a major (education) or college (general education) requirement. There were two significant differences in the student populations with regard to age and year in school. Students enrolled in the online sections were a bit older and a bit farther along in school than their counterparts in face to face sections. Those in the lecture sections were overwhelmingly first-years and of a traditional age.
Graded Performance
The first step in the analysis is to compare the students' grades for various course components in each section. Table 1 presents a difference of means analysis for the three exams, quizzes or discussion questions (DQ), and final course grade by the mode of instruction.2
Because the writing assignment was a fairly low-risk/low-reward task, there was very little variance in the grades on this component. For that reason, this variable is not included in the analysis.
Bivariate Analysis of Graded Course Components by Mode of Instruction Difference of Means Analysis
Of course, there could be several factors that could explain these differences that are not taken into account in the bivariate analysis. The next step, then, is to conduct a regression analysis to see whether these bivariate differences remain after employing appropriate control variables. Table 2 presents a model that predicts each of the course grade components as a function of the survey variables. With regard to the control variables beyond the mode of instruction, the only consistent influence on the grade was GPA. It is, of course, not surprising that students with higher GPAs were more likely to get higher grades throughout the semester. In one or two instances, age, sex, or semester was significantly related to course grade, but there is no clear or consistent pattern to these influences. With regard to the variable measuring the mode of instruction, we see that the significant differences between the two sections revealed in the bivariate analysis are maintained even in the presence of the controls. The mode of instruction is not significantly related to the grade on the first exam, but is related to each of the other graded components. In each case, the students in the online section scored higher than the students in the face-to-face section. This finding is consistent with the recent work in economics that demonstrated higher achievement in students in online courses than in face-to-face lectures (Myers and Nelson 2004).
Determinants of Graded Components of the Course Regression Analysis
While on their face we could interpret these results to mean that, at worst online instruction is no worse than face-to-face and, at best, could actually be superior, I would suggest that these findings be placed in context. First, the students in the online sections were a bit older and farther along in school than those in the face-to-face sections, which could bring some intangibles related to experience and maturity that my analysis is not capturing. (Although, the variable measuring year in school is never significant in the regression analysis and the age variable is only significantly related to performance in two of the five models.) Second, I don't have information on other personality intangibles like organizational ability and time-management skills that could be the key to success in an online setting, so don't know if the students in the different sections were randomly distributed on these important variables. Granted, while these skills should dictate success in any course setting, online courses probably require more self-motivation since they don't have structured hours (Garson 1998). Finally, while I did try to make the course as comparable as possible across sections, there are aspects of the online section that may have worked to improve student performance. As I suggested earlier, the more complete lecture notes and the weekly practice of formulating and responding to discussion questions may have given those students an advantage.
Satisfaction with the Course
One other way to measure whether mode of instruction had an impact on education is to examine students' evaluations of the course. While most of the items used on the form asked students to specifically evaluate different aspects of the instructor's performance in the class, the responses can serve as a rough measure of whether students believed the course was a successful experience. Although, I should note that the course evaluation form I used was written with traditional face-to-face courses in mind and may not capture the experience of being a student in an online class as well. Beyond the usual vagaries of teaching evaluations, there is nothing about the student populations in each section that should cause us to expect significant divergence in evaluation beyond the different experiences shaped by the mode of instruction.
As Table 3 indicates, students in each of the sections were largely positive about the course. While the face-to-face students gave higher ratings on some items and the online students rate the course higher on other items, there are very few real differences between their evaluations. However, it is interesting to note that several of the items that the online students rated lower than the face-to-face students did involved “communications” in one way or another: whether “the instructor's class presentations were clear,” whether “the instructor's presentations added to my understanding of the material,” whether “the instructor seemed enthusiastic about the material,” and whether “the instructor clearly communicated what was expected of me in this class.” Again, the differences between the sections on these items are small and not necessarily meaningful. But it may be the case that communicating effectively in the online courses takes more or different types of interactions, since students can't rely on your physical expressions, body language, and follow-up comments.
Teaching Evaluation Scores by Mode of Instruction
Discussion
In the end, I am comfortable saying that the data from the courses I taught contribute to a growing body of work that finds online courses to be at least as successful at providing university-level education as more traditional face-to-face courses. While most students in each section were successful in the course, the students in the online section did seem to outperform their face-to-face colleagues, at least as measured by course grades. Indeed, given the organizational and pedagogical differences between the two sections, the online course could be considered superior to the face-to-face format for achieving some goals. As online courses become more numerous at universities around the country, findings such as these can contribute to conversations about how to use the advantages and flexibility of the online environment to maximum effect.
For example, the hybrid RST course is a fast-growing format for university courses. Here, what is best about the face-to-face and online formats can be combined to create a course in which the students meet face-to-face with the instructor and students for some part of the week and operate online at other times. This may open up new options for courses, particularly those with large enrollments. In situations where large classes do not have teaching assistants holding discussion sections, creating online discussion sections and a RST course could facilitate the instructor's ability to engage the students in greater discussion and analysis than the face-to-face environment may allow. Such an approach could have the additional benefit of increasing a sense of community and student satisfaction with the larger course. At the same time, even in smaller enrollment courses, online components can create rich opportunities for students to engage in research, discussion, group projects, and cooperative learning. In the same way, RST courses could work to improve upon the limitations of an exclusively online class by providing students physical interaction with faculty and peers and opportunities to ask questions in a real-time environment.
In the end, online courses are not a panacea for what ails university teaching and education. There are vibrant contemporary debates about the role and appropriate use of online courses as part of a university education. For instance, are online courses better suited for particular subject matters or course levels? Should the university discourage beginning students from taking online courses for fear that they feel too isolated from faculty and other students? Do online courses create barriers to some students who may have limited access to the necessary technologies or do they increase access among those who do not live proximate to a university? By relying on online courses are we substituting convenience and cost-savings for the “real” learning that takes place in the physical classroom? While there are no clear answers to these questions that can be generalized across the diversity of U.S. colleges and universities, it is clear that these questions will not slow the pace of online course integration. Instead, online educational formats can be seen as another tool available to faculty to broaden the opportunities they offer students—tools with the same potential advantages and limitations of any of the other methods of education we currently employ.
Author Bio
Kathleen Dolan is a professor of political science at the University of Wisconsin–Milwaukee and the co-editor of the journal Politics & Gender. Her research focuses on elections, voting behavior, and gender politics.
