A common question that often serves as starting point to discussions of online education asks if there is any research that indicates whether online learning is as effective as classroom-based learning. Its a complex question that might be answered with another question: “effective at what?” As a quick reference I compiled a brief list of reputable research review abstracts that reflect our current understanding of “traditional” online education compared to classroom-based education in mostly college settings (not MOOCs). Such reviews are more indicative of the state of research than any individual studies that might become the buzz of the day. I also recently updated this post with reference to more recent research of online learning at the community college level and blended learning.
The emerging answer is that there is no significant difference in general, but that design matters and as the years go by, online learning appears to be improving relative to classroom-based instruction. You will note that the more recent comprehensive reviews of research (Means et. al. 2009 e.g.) favor online learning over classroom instruction (with caveats of course).
I will add to this as time allows, but I wanted to be able to point people to a single page that might help begin to answer these questions. Again, these articles were written by reputable scholars/researchers, have been the subject of review, and have been published in credible venues. The text that is bold face indicates the main finding (usually no significant difference).
Bernard RM, Abrami PC, Lou Y, Borokhovski E, Wade A, Wozney L, et al. (2004). How does distance education compare to classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research, 74, 379–439.
“A meta-analysis of the comparative distance education (DE) literature between 1985 and 2002 was conducted. In total, 232 studies containing 688 independent achievement, attitude, and retention outcomes were analyzed. Overall results indicated effect sizes of essentially zero on all three measures and wide variability. This suggests that many applications of DE outperform their classroom counterparts and that many perform more poorly. Dividing achievement outcomes into synchronous and asynchronous forms of DE produced a somewhat different impression. In general, mean achievement effect sizes for synchronous applications favored classroom instruction, while effect sizes for asynchronous applications favored DE. However, significant heterogeneity remained in each subset.”
Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation, and Policy Development.
“A systematic search of the research literature from 1996 through July 2008 identified more than a thousand empirical studies of online learning. Analysts screened these studies to find those that(a) contrasted an online to a face-to-face condition, (b) measured student learning outcomes, (c)used a rigorous research design, and (d) provided adequate information to calculate an effect size. As a result of this screening, 50 independent effects were identified that could be subjected to meta-analysis. The meta-analysis found that, on average, students in online learning conditions performed modestly better than those receiving face-to-face instruction. The difference between student outcomes for online and face-to-face classes—measured as the difference between treatment and control means, divided by the pooled standard deviation—was larger in those studies contrasting conditions that blended elements of online and face-to-face instruction with conditions taught entirely face-to-face. Analysts noted that these blended conditions often included additional learning time and instructional elements not received by students in control conditions. This finding suggests that the positive effects associated with blended learning should not be attributed to the media, per se.”
Tallent-Runnels MK, Thomas JA, Lan WY, Cooper S, Ahern TC, Shaw SM, et al. (2006). Teaching courses online: A review of the research. Review of Educational Research, 76, 93–135.
“This literature review summarizes research on online teaching and learning. It is organized into four topics: course environment, learners’ outcomes, learners’ characteristics, and institutional and administrative factors. The authors found little consistency of terminology, discovered some conclusive guidelines, and identified developing lines of inquiry. The conclusions overall suggest that most of the studies reviewed were descriptive and exploratory, that most online students are nontraditional and Anglo American, and that few universities have written policies, guidelines, or technical support for faculty members or students. Asynchronous communication seemed to facilitate in-depth communication (but not more than in traditional classes), students liked to move at their own pace, learning outcomes appeared to be the same as in traditional courses, and students with prior training in computers were more satisfied with online courses. Continued research is needed to inform learner outcomes, learner characteristics, course environment, and institutional factors related to delivery system variables in order to test learning theories and teaching models inherent in course design.”
Zhao, Y., Lei, J., Yan, B., Lai, C., & Tan, H. S. (2005). What makes the difference? A practical analysis of research on the effectiveness of distance education. Teachers College Record, 107(8), 1836–1884.
“This article reports findings of a meta-analytical study of research on distance education. The purpose of this study was to identify factors that affect the effectiveness of distance education. The results show that although the aggregated data of available studies show no significant difference in outcomes between distance education and face-to-face education as previous research reviews suggest, there is remarkable difference across the studies. Further examination of the difference reveals that distance education programs, just like traditional education programs, vary a great deal in their outcomes, and the outcome of distance education is associated with a number of pedagogical and technological factors. This study led to some important data-driven suggestions for and about distance education.”
Recent Community College Research
While there is general consensus that overall online and distance education do not differ significantly from classroom education when defined by various course-level learning outcomes (Bernard et. al, 2004; Mean’s et. al. 2009; Tallent-Runnels et. al, 2006; Zhao et. al 2005); relatively little quality research exists examining outcomes such as persistence, attrition, and attainment of college credentials, especially for less prepared students at the community college level. The few stronger studies addressing these outcomes have not been generally positive. For example, previous researchers who have studied large samples of online learners in state community college systems (e.g. Smith Jaggars & Xu, 2010; Xu & Smith Jaggars, 2011) have reported several discouraging findings. Analyzing data on nearly 24,000 students in 23 institutions in the Virginia Community College system Smith Jaggars and Wu (2010) concluded that learners had a greater likelihood of failing or withdrawing from online courses than from face-to-face courses. The authors also found that students who took online coursework in early semesters were somewhat less likely to return to school in following semesters, and students who took a higher proportion of credits online were slightly less likely to attain a credential or transfer to a four-year institution.
These same authors (Xu & Smith Jaggars, 2011) also studied the Washington State Community College Systems and came to similar conclusions. Analyzing data from more than 51,000 students in 34 community and technical colleges Xu & Smith Jaggars found that although students with better educational preparation were more likely to enroll in online courses, these students were also significantly more likely to fail or withdraw from these courses than students who took traditional face-to-face classes. Washington State Community College students who took more online courses were also slightly less likely to complete a degree or transfer to a four-year college than those who took fewer online courses.
The more recent study in this area (Johnson & Cuellar Mejilla, 2014) focused on the California Community Colleges, the nation’s largest two-year system enrolling nearly 1 million students in online courses. Methods employed and results were similar to the Washington and Virginia studies; community college online learners were less likely to complete an online course than a traditional course, and were less likely to complete an online course with a passing grade.
While it is clear that the some conclusions drawn from the community college students in California, Virginia, and Washington don’t hold great promise with regard to online learning, both the California Community College study and recent national study (Shea & Bidjerano, 2014) yielded some positive results. Shea & Bidjerano (2014) utilized federal data from the Beginning Post-Secondary Survey (04/09) to determine if national trends were consistent with negative conclusions drawn from analysis of large-scale State level data. The study found that when controlling for relevant background characteristics; community college students who take some of their early courses online or at a distance have a significantly better chance of attaining a college credential than do their classroom only counterparts. Evidence suggests that early participation in online learning and distance education predicts higher rates of degree attainment even when self-selection bias is controlled for. These more positive results are replicated and extended within the California Community College study, where the authors found that student who take at least some online courses are more likely than those who take only traditional courses to earn an associate’s degree or to transfer to a four-year institution (Johnson & Cuellar Mejilla, 2014, p. 2).
Given the mixed results we can tentatively conclude that online learning appears to represent at least a potential boost to community college degree completion and college transfer. However, additional study is needed to understand the conflicting findings and to understand and amplify the potential benefits of online learning as they relate to course level outcomes, transfer and degree completion.
Most Recent Research
Bernard, R.M., Borokhovski, E., Schmid, R.F. et al. A meta-analysis of blended learning and technology use in higher education: from the general to the applied. Journal of Computing in High Education 26, 87–122 (2014). https://doi.org/10.1007/s12528-013-9077-3
Some of the most recent systematic investigations of online learning have been conducted by Bernard and his colleagues (Bernard et. al. 2014). These authors reference sixteen (16) meta-analyses and come to the following conclusions regarding distance/online learning (DE/OL) compared to classroom instruction (CI).
“Literally thousands of comparative primary studies, (…), have pitted DE/OL against CI and since 2000, sixteen major meta-analyses have been mounted to assess the differences between CI and DE/OL.1These are among the important things that we have learned from all of this primary research and synthesis activity:
1. there is general consensus of the effectiveness of all forms of DE (including OL) compared with CI (i.e., the average effect sizes range from d +≈ 0.00 for conventional DE and correspondence education to d + ≈ 0.05–0.15 for OL)—in other words there is little difference in these two instructional patterns;
2. there is wide variability among studies, from those strongly favoring DE to those favoring CI, thereby bringing into question the value of point one
3. there is a tendency for researchers to describe the DE/OL condition in great detail while characterizing the CI condition as “traditional classroom instruction,” thereby diminishing the opportunity to describe and compare salient study features;
4. comparative primary research is plagued with a variety of methodological problems and confounds that make them very hard to interpret (Is it the distance, the media, the instructional strategies, etc., or combinations of these?)
By Peter Shea