SAMPLE

RESULTS

More than 500 abstracts were reviewed for this meta-analysis, leading to the retrieval of 325 full-text articles and ultimately 90 primary research articles for consideration, which were further refined to the inclusion of 51 studies that yielded 51 effect sizes, consisting of 4,630 participants.

•Reported research methods, including participants, instructional interventions, instruments, research design and results.•Employed a quasi-experimental or experimental research method.•Study established pretest equivalence and employed a statistical control for pretest differences.•The experimental group utilized some form of online digital literacy practice involving a social media platform or Web 2.0 tool.•Control groups received traditional instructional delivery methods and practices. •Examined the impact of online digital literacy practices on learning outcomes.•Included higher education participants. •The research presented in one study did not overlap research from another study.•The article was written in English.

Introduction

ABSTRACT

This paper reports the findings of a Stage I meta-analysis exploring the effectiveness of online digital literacy practices performed through social media tools in higher education classrooms as measured by learning outcomes. An extensive literature search culled more than 500 potential articles and resulted in a sample of 51 representative quasi-experimental studies, consisting of 4,630 total participants. Fifty-one effect sizes were extracted and yielded a moderately positive statistically significant weighted average effect size of g+ = 0.315, k=51, p <. 01. This overall effect size suggests that integrating online digital literacy practices into the higher education classroom benefits students on measures of academic achievement and offers significant learning support.

In addition, several moderator variables were tested to determine what factors and literacy practices impact learning outcomes; namely, field of study (STEM, ARTS), conceptualization of tool (social learning theory, delivery tool), peer interaction (yes, no), modality (blogs, collaborative communication text-based technologies, i.e. forums and wikis; podcasts, and virtual worlds), practice (consuming, prosuming behaviors), and the learning outcome measured (course project, knowledge-based exam); moderator effect sizes were statistically significant for the following variables: conceptualization of the tool, practice, and learning outcomes.

RESEARCH QUESTIONS

• What is the effect of digital literacy practices on learning outcomes in formal educational settings?

• How do pedagogical variables; such as field of study, peer interaction, form of online modality, practice, and learning outcome moderate this effect?

The Impact of Digital Literacy Practices on Learning Outcomes in The Impact of Digital Literacy Practices on Learning Outcomes in Higher Education: A Meta-AnalysisHigher Education: A Meta-Analysis

Jeannette Novakovich, Graduate Student in PhD Program, Educational Technology, Concordia University, Montreal, QuebecJeannette Novakovich, Graduate Student in PhD Program, Educational Technology, Concordia University, Montreal, QuebecNicole Fournier-Sylvester, Graduate Student in PhD Program, Educational Studies, Concorida University, Montreal, QuebecNicole Fournier-Sylvester, Graduate Student in PhD Program, Educational Studies, Concorida University, Montreal, Quebec

Jeannettenovakovich@gmail.com, 514-755-3987Jeannettenovakovich@gmail.com, 514-755-3987

Introduction

DISCUSSION

The overall effect of digital literacy practices on learning outcomes The overall results of this Stage 1 meta-analysis (g+ = 0.315, k=51, p < .01) indicate that on average, learners who engage in digital literacy practices in the form of a social media technology in higher education achieve higher positive learning outcomes than students who engage in traditional literacy practices. Our results are fairly close to those reported by Richard Schmid et al (2014) in a meta-analysis on the overall effect of the impact of technology on learning outcomes in higher education (g+ 0.27, k=879, p < .05) and a meta-analysis conducted by Robert Bernard et al (2014) on the impact of blended learning or computer-supported online learning combined with classical instruction, (g+ = 0.334, k = 117, p< .001). Although these results cannot answer Tess’ (2013) question about whether or not digital technologies represent a “fundamental shift in the way students learn,” (p. A65), they do support Bennett’s (2012) contention that the content creation, collaboration and sharing that is facilitated by digital literacy practices through online social media in higher education can have important learning benefits.  The impact of conceptualization, practice and measured learning outcome  First, the significance of the moderator variable Conceptualization suggests that the approach taken to the integration of the practice will impact to some extent learning outcomes (g+ = 0.325, k=51, p < .002; Social learning theory g+= .390, and Delivery tool, g+= -.013). For example, using social media as a delivery tool (ie. posting a podcast of a lecture) had no impact on learning outcomes. Effective uses of social media in the classroom are thus grounded in a socially situated and constructivist approach to teaching and learning.  To further support the finding of the variable conceptualization, the significance of the moderator variable Practice, suggests that learning outcomes appear to be considerably higher when students are called upon to produce knowledge rather than consume it: g+ = 0.326, k=51, p < .022, Prosuming g+= .504, and Consuming g+=.042. Consuming practices have basically no impact on learning outcomes and register with an effect size close to zero or nil. This moderator variable was chosen based on Lin et al (2010) theoretical framework for developing digital literacy practices by distinguishing between function and critical “consuming” (reading/viewing) and “prosuming” (producing/creating) of the works of others.  And lastly, the significance of the moderator variable Learning Outcome suggests that the content produced through digital literacy practices exceeds content produced through traditional practice: g+ = 0.235, k=51, p < .02, Exam g+= .168, and Project g+= .719. Our findings support Bennett’s (2012) findings that the main benefits that come from the “effective use of Web 2.0” is “particularly through student content creation and sharing” (p. 533). The effectiveness of the integration of social media in the classroom is dependent on how a teacher utilizes its capacity for interaction and the development and sharing of projects. Effective uses of social media in the classroom are thus grounded in a student-centered and constructivist approach to teaching and learning. Greenhow (2011) summarizes that integrating social media tools in learning promotes a more student-centered course. Students are better able to create their own understanding of content when creating with these tools.  Thus, the integration of social media also facilitates active engagement with the world. Learners create meaning, by experiencing the world and engaging in meaningful ways.   References Bennett, S., Bishop, A, Dalgarno, B., Waycott, J., & Kennedy, G. (2012). Implementing Web 2.0 technologies in higher education: A collective case study. Computers & Education. 59: 524-534.Bernard, R., Borokhovski, E, Schmid, R., Tamim, R. & Abrami, P. (2014) A meta-analysis of blended learning and tcchnology use in higher education: from the general to the applied. Journal Computers in Higher Education. 26: 87-122.Greenhow, C., Robelia, B., & Hughes, J. E. (2009). Learning, teaching, and scholarship in a digital age. Educational Researcher, 38(4), 246–259. doi:10.3102/0013189X09336671.Lin, T., Li, J., Deng, F. & Lee, L. (2010). Understanding New Media Literacy: An Explorative Theoretical Framework. Educational Technology & Society, 16(4), 160-170. Schmid, R, Bernard, R, Borokhovski, E. Tamim, R., Abrami, P., Surkes, M, Wade, C., & Woods, J. (2014) The effects of technology use in postsecondary education: A meta-analysis of classroom applications. Computers & Education. 72: 271-291

RESEARCH QUESTIONS

• What is the effect of digital literacy practices on learning outcomes in formal educational settings?

• How do pedagogical variables; such as field of study, peer interaction, form of online modality, practice, and learning outcome moderate this effect?

DEFINITION OF TERMS

Jeannettenovakovich@gmail.com, 514-755-3987Jeannettenovakovich@gmail.com, 514-755-3987

The following terms are essential for understanding the underpinnings of the research questions:

Literacy event. Defined by Bhatt, (2012) “In analysis, researchers begin with a literacy event as an empirical occasion involving interaction around a written text, from which they then infer the literacy practices” (p. 291).

Literacy practices. This type of activity involves the creation of knowledge, values, attitudes, beliefs and feelings correlated with the reading and writing of texts within a social context (Street 1984, Baynham 1995, Barton & Hamilton 1998, Barton et al. 2000).

Digital literacy practices. As established for the purposes of this meta-analysis, digital literacy practices are comprised of literacy activities that involve interaction around digital content, including, texts, images, sound, and video.

Prosuming. Digital literacy practices where knowledge is created or generated (Lin et al, 2010).

Consuming. Digital literacy practices where knowledge is consumed.

Social media. “ Internet based applications that build on the ideological and technological foundations of Web 2.0, and that allow the creation and exchange of user generated content” (Kaplan & Haenlein, 2010, p. 61), and encompass “(a) social networking sites, such as Facebook, Twitter, and LinkedIn, (b) media sharing sites, such as YouTube and Flickr, (c) creation and publishing tools, such as wikis and blogs, (d) aggregation and republishing through RSS feeds, and (e) remixing of content and republishing tools,” (Greenhow, 2011, p. 140).

Blogs. Individual digital spaces where individual learners can consume and /or prosume knowledge and other learners can interact through a commenting function.

Podcasts. Video and audio online digital content that learners can consume and/or prosume.

Collaborative communication technology. Early and current online text-based technologies that allow learners to communicate collaboratively, including wikis, forums, and other online communication tools.

Virtual worlds. A digital literacy practice where learning takes place through online simulated worlds designed for formal learning activities, such as Second life.

Learning outcome. A measured gain of knowledge or skills by the learner as measured by a valid instrument.

Project. Learning outcomes measured through the creation of course content by a literacy event.

Exam. Learning outcomes measured through an examination of course material.