The Effect of Digital Stories on Academic Achievement: A Meta-Analysis

The purpose of this study is to determine the effect of digital stories on academic achievement. In order to achieve this purpose, meta analysis method was used in the study. Within the scope of the study, ERIC, Google Academic, YÖK Thesis Center, ProQuest, Science Direct and ULAKB İ M databases were scanned and 23 studies (10 theses, 13 articles) were included in the meta-analysis using the criteria determined by the researchers. Cochran’s X 2 (Q = 285,155, p < .05) test was conducted to test whether the studies included in the study were heterogeneous. Random effects model was used to calculate effect size since heterogeneity was determined between studies. At the end of the study, it was seen that the general effect size (Hedge’s g = 1.081) regarding the effect of digital stories on academic achievement was strong, that there was a positive effect in all areas according to the lessons which was higher for Science. It was also observed that the effect was positive in all dimensions according to the education level, there was a difference between the education levels and the highest effect occurred at university and middle school levels.


Introduction
Our age is known as the age of information; countries all over the world make an intense effort to become an information society, aiming to raise individuals who have the ability to learn how to learn (Keser, 2011). In the 21st century, which is defined as the information age, the rapid developments in technology take all societies to a world where information is intense; accordingly, education programs, investments in education and education policies take a form in which more information is stored. In this context, educators have to develop new educational programs and learning/teaching models (İşman, 2011). Prensky (2001) describes today's students as digital natives and states that e-mail, internet, mobile phone, computer game and instant messaging are integral parts of their lives. In other words, children growing up with digital technology see technology as a tool rather than a barrier to learning (France & Wakefield, 2011).
While the word 'story' is defined as "the event described in detail" by the Turkish Language Association [TDK] (2020), according to Işıtan and Turan (2014), storytelling is one of the oldest arts and people have used this method to express themselves since the day they existed. With the advancing technology, classical methods have ceased to be the focus of attention of students and students' expectations from education have changed. Therefore, it can be said that using new and different methods and materials in parallel with the developing technology will increase the efficiency in education (Uslupehlivan, Kurtoğlu Erden, & Cebesoy, 2017).
Digital storytelling consists of creating a narrative using personal digital media (e.g., photos, video, etc.) and detailing it with personal experience to show the narrative. The digital story offers the opportunity to write and record a script, edit digital photos and videos, and combine these media to create a coherent personal story (Landry & Guzdial, 2006). When the definitions related to digital stories are examined in general, it is seen as combining the art of storytelling with various digital multimedia such as image, sound and video. Almost all digital stories consist of a mix of digital graphics, text, recorded voice narration, video and music to present information on a particular subject. Stories usually take only a few minutes to tell and are used to tell personal tales, historical events or to inform or instruct on a particular topic (Robin, 2006). Digital story is the process of integrating traditional literature (listening, reading, speaking, writing) with digital images, video, music and computer graphics (Bandi-Rao & Sepp, 2014). According to Inceelli (2005), "digital storytelling is the process of presenting the purposeful narrative that enables the users in an interactive digital environment by giving the control of this environment; that enables them to discover their own knowledge and to apply this knowledge in real life situations; that enables the users to construct information in his mind in a meaningful way and to reinforce it with exercises and applications while moving in a story flow; and that uses multimedia applications such as sound, picture, graphics, motion graphics, images, music, text, photography, animation in addition to the narrator's own voice for information, education, entertainment purposes". As a teaching tool, teachers have the option to show their students preformed digital stories to grab students' attention while presenting content and new ideas (Robin, 2009). Digital stories are an important educational tool that develops students' creativity skills and prepares them a collaborative and social learning environment (Robin, 2016).

Purpose of the Study
The aim of this study is to determine the effect of digital stories on academic achievement using meta-analysis. In this context, following research questions were asked: What are the effects of digital stories on academic achievement? Do the effects of digital stories on academic achievement vary by learning area?
Do the effects of digital stories on academic achievement vary by education level?

Method
In this study, meta-analysis method was used to determine the effect of digital stories on academic achievement. Dinçer (2014) defined meta-analysis as "grouping similar studies on a subject, theme or field of study under certain criteria and combining quantitative findings of these studies and interpreting them". Meta-analysis is a meaningful synthesis of the analysis results collected systematically (Borenstein et al., 2011). It adds a new dimension to the criticism by examining the findings of different studies altogether (Akgöz et al., 2004). Meta-analysis is the synthesis of the findings obtained from many experimental studies and can be used in many fields such as social sciences and medical sciences (Wolf, 1986). It creates a meaningful summary of many studies with effect size values (Hedges et al., 1999). The statistical meta-analysis method can be used to combine two or more individual study results. More specifically, an overall effect is estimated by calculating the mean of the weighted estimates in individual studies. There are several meta-analysis methods that differ mainly in the weighting scheme used. Meta-analysis provides a statistical method for evaluating the direction and size of the effect, and includes the question of whether the effect is consistent in different studies (Schwarzer, 2007). Meta-analysis is an assessment list to evaluate different studies that have quantitative data (Hedges & Olkin, 2014).

Data Collection
After the research topic has been determined, a criterion has been determined for the studies to be included in the meta-analysis. These criteria can be listed as follows: • The studies should be articles published in refereed journals or master/doctoral theses • The studies should be in Turkish or in English • The studies should be experimental studies and should have experimental and control groups • The required values for calculations of effect size (X, ss, F, p…) should be given in the study.
Between 15.04.2020−15.08.2020, the keywords of "dijital öykü AND başarı" (Turkish for digital narrative AND achievement), "dijital hikaye AND başarı" (Turkish for digital story AND achievement), "digital story AND achievement", "digital storytelling AND achievement", "digital story AND performance" were searched in the databases of ERIC, Google Scholar, YÖK Thesis Center, ProQuest, Science Direct, ULAKBİM, and initially 35 studies were downloaded. Later, it was determined that 5 studies were articles produced from theses and some of the other studies did not provide sufficient information for analysis or were not suitable for meta-analysis. As a result, 23 studies were included in the meta-analysis.

Coding of the Studies
The studies included in the study were processed in an Excel table containing the name of the author(s), publication year, field, participants' grade level, statistical methods and data required for meta-analysis (X, ss, N, p, F, etc.). In order to calculate the reliability coefficient of the study, a comparison was made after the studies were coded separately by the researchers. According to the formula suggested by Miles and Huberman (1994) (Consensus/[Consensus + Disagreement]), the coefficient of congruence of the codes was found to be 95%. Yıldırım and Şimşek (2006) state that in order to ensure reliability in such studies, at least 70% compliance is required. Therefore, it can be said that the reliability of the study is high. Information about the studies included in meta-analysis is presented in Table 1. When Table 1 is examined, it is seen that the studies included in the meta-analysis consist of 8 master theses, 2 doctoral theses, and 13 articles published in refereed journals. It is also seen that 1 of the studies were carried out with preschool students, 5 with elementary school students, 11 with secondary school, 3 with high school and 3 with university students.

Data Analysis
For data analysis, firstly the data required for meta-analysis were processed into a form in Microsoft Excel format. Comprehensive Meta-Analysis (CMA) program was used for the analyses. Data entry into the coding form was made by two researchers separately and then two entries were compared. In this way, errors related to data entry were prevented.
In this study, funnel plot, Rosenthal's Safe N Theory and Correlation Rank methods of Begg and Mazumdar (1994) were used for calculating publication bias. The funnel plot is based on the idea that as the sample size of the studies increases, the precision in estimating the effect will increase, and studies with a small sample will be widely distributed at the bottom of the graph and the distribution among larger studies will narrow. The situation where the drawing resembles a symmetrical inverted funnel indicates that there is no bias (Egger et al., 1997).
The most common way of expressing the extent of heterogeneity is using the statistical test known as Cochran's X 2 test or Q test. A p value is frequently quoted as an indicator of the extent of variability between studies (Higgins & Thompson, 2002). In this study, the p value and Q values for the heterogeneity test were examined and the calculated Q value was compared with the corresponding value in the X 2

Results
In this sect

Findin
First, it wa in Figure 1 org cked whether t was used at 9 ion determined al. (2007)  In order to test publication bias, most of the publications in the graph are expected to gather symmetrically within the funnel lines and around the general effect line (Dinçer, 2014;Borenstein et al., 2011). When the graphic is examined, it is seen that most of the studies take place inside the funnel. However, there are points in the graph that are not symmetrical. There are studies with a small sample at the bottom of the funnel plot and large studies at the top (Dinçer, 2014). Accordingly, it can be said that the studies examined in this study are studies with small and large samples. From this graph, it can be said that 23 studies included in the study were close to show publication bias. There are also statistical procedures to detect publication bias one of which is Rosenthal's Protected N theory. Rosenthal (1991; cited in Bakioğlu & Göktaş, 2018) stated that when calculating the safe N number for the study obtained from k studies, if N > 5k + 10, the meta-analysis findings would be quite strong. That is, according to this theory, it can be said that the farther the number of studies examined from the safe N number is, the less the study shows publication bias. In this case, it can be said that there is no publication bias in this study since it is 2409 > 5*23 + 10.
Another statistic for publication bias is the Begg and Mazumdar Rank Correlation statistics. According to this statistic, it is accepted that there is no publication bias if the Tau coefficient is close to 1.00 and the two-tailed p value is not significantly different (Begg & Mazumdar, 1994). When Table 3 is examined, it can be said that there is no publication bias in this study since the Tau coefficient (0.83000) is close to 1.00 and the p-value (two-tailed) is > 0.05.

Findings Related to Heterogeneity Test
Before calculating the effect size, it was investigated whether the studies included in meta-analysis had a heterogeneous structure and the analysis method to be used was decided according to the heterogeneity test result. According to the heterogeneity test results, it was observed that the studies were different from each other, that there was a heterogeneous structure between the studies (Q = 285.155, p < .05) and the heterogeneity level (I 2 ) between the studies was determined as 92%. An I 2 value above 75% indicates a high level of heterogeneity (Cooper, Hedges, & Valentin, 2009). According to Higgins and Thompson (2002), there is a positive correlation between the addition of different studies to the analysis and heterogeneity. According to this result, random effects model was preferred in the calculation of effect size in this study.

Findings Related to the Effect of Digital Stories on Academic Achievement
Information about the general effect size obtained by using random effects model due to the heterogeneous nature of the studies included in the meta-analysis is given in Table 5. According to the information in Table 5, the general effect size of the effect of digital stories on academic success was determined as Hedge's g = 1.081 according to the random effects model, and this value was determined as a strong effect size according to the classification made by Cohen et al. (2007). It can also be said that the effect size value shows a significant difference (Z = 5.609, p = 0.000). According to this result, it can be concluded that digital stories have a strong positive effect on academic achievement in favor of the experimental group. Table 6 contains meta-analysis results in terms of subject. When Table 6 was examined, it was seen that there was a difference between the fields according to the statistical results regarding the effect of digital stories on academic achievement in terms of subject, and the largest effect was in the field of Science (g = 1.562). The calculation could not be made due to the inadequacy of studies in the fields of Physics, Chemistry, Biology, Information and Communication Technologies and Morality.

The Effect of Digital Stories on Academic Achievement Based on Sub-Dimensions
In the heterogeneity test conducted to determine whether the effect sizes differ between courses, the value of Q between (200,963) was above the critical value (16.919) specified for the degree of freedom in the X 2 table 9 at the 95% significance level and the p value was less than 0.05 (p = 0.000). For this reason, it could be said that there was a heterogeneous structure among studies (Q between = 200,963; p = 0,000). In other words, it could be said that the field type was a variable that changed the effect size. When Table 7 was examined, it was seen that according to the statistical results regarding the effect of digital stories on academic achievement in terms of education level, there was a significant difference according to education levels and the largest effect coefficient (g = 1.294) was found in university students. Since the Q value (81.390) found in order to determine whether the effect sizes differ according to the education level was above the corresponding degree of freedom (4) in the X 2 table and the critical value of 9.488 determined for the 95% significance level and the p value was less than 0.05, it could be said that the distribution had a heterogeneous structure (Q between = 81.390; p = 0.000). In this case, it could be said that the education level changed the effect jel.ccsenet.org Journal of Education and Learning Vol. 9, No. 6; size.

Discussion
In this study, it was aimed to determine the effect of digital stories on academic achievement using meta-analysis method. In this context, 23 studies including 10 theses and 13 articles published in refereed journals were examined. In a study conducted by Talan (2019), it was also seen that the studies on digital stories were generally in the form of articles. While the studies examined in the current study mostly conducted at the middle school level (11), there were also studies at the elementary school (5), high school (3), university (3) and preschool (1) levels. According to Talan (2019), studies on digital stories were mostly conducted at middle school and university level. In the study conducted by Şimşek et al. (2018), it was found that digital stories were mostly used at university and middle school levels. It was observed that the studies included in the current meta-analysis were mostly conducted in fields such as Science, Turkish Education, Mathematics and Social Studies. In the study conducted by Talan (2019), Science, Turkish and Foreign Language were also the most common subjects.
In the study, funnel plot, Rosenthal's safe N theory and correlation tests of Begg and Mazumdar were applied to test publication bias. According to Rosenthal's safe N theory, it can be said that the farther the number of studies examined from the safe N number is, the less the study shows publication bias. In this case, it could be said that there was no publication bias in this study since 2409 > 5*23 + 10. Another statistic for publication bias is the Begg and Mazumdar Rank Correlation statistic. According to this statistic, it is accepted that there is no publication bias if the Tau coefficient is close to 1.00 and the two-tailed p value is not significantly different (Begg & Mazumdar, 1994). Since the Tau coefficient (0.83000) in this study was close to 1.00 and the p-value (two-tailed) was > 0.05, it could be said that there was no publication bias in this study.
According to the results obtained from the research, the effect of digital stories on academic achievement was found to be g = 1.081 and this value is a high-level effect according to Cohen's (2007) classification. This result coincides with the meta-analysis studies conducted by Camnalbur (2008), Dinçer (2015), Dikmen and Tuncer (2018), Grgurovic et al. (2013), and Li and Ma (2010) in order to determine the effect of computer-aided education on academic achievement. Likewise, Cheung and Slav (2012), Schmid et al. (2009), Schmid et al. (2014, Tamim et al. (2014), and Cheung and Slav (2012) found a strong effect in their meta-analysis studies on the effect of technology use in education on academic achievement. In the meta-analysis study conducted by Lee (1999) on the effect of computer-aided simulations on academic achievement, it was found that there was a positive result in favor of the experimental group. In the meta-analysis study conducted by Orhan and Durak Men (2018) on the effect of web-based teaching on science achievement, it was seen that there was a high effect. A meta-analysis study by Ayaz et al. (2016) aimed at determining the effect of using instructional technology on the academic success of primary school students, and it was found that there was a moderate but close to high positive effect. Today, developments in the field of internet and technology have been reflected in educational environments and brought a new perspective. Information-media-technology literacy (Kotluk & Kocakaya, 2015), which is a common feature in the studies of all organizations and which is defined as 21st century skills by by organizations such as ISTE, P21 and as "survival skills" Wagner (2008), is also supported by digital stories (Robin, 2009).
In the current study, it was seen that the effect of digital stories on academic achievement according to the fields led to positive results in all dimensions and these values were evaluated as high effect according to Cohen's (2007) classification, and that there was a significant difference between the fields and the highest effect size value was in the field of Science. When the literature is examined, Ayaz et al. (2016) found that the highest effect occurred in the Social Studies course and there was no significant difference between the fields. In the meta-analysis study conducted by Camnalbur (2008) on the effectiveness of computer aided education, it was observed that there was a difference between the fields. In the study conducted by Kablan et al. (2013) on the use of materials in classroom teaching, the use of materials in the classroom had a positive effect in all areas, there was no difference between the fields and the highest effect value was in Science. This result may be due to the fact that the studies are mostly conducted in the field of Science. In the study conducted by Dinçer (2015), it was seen that the studies on computer-aided education were mostly conducted in the field of Science.
Another result obtained from the study showed that the effect of digital stories on academic achievement according to the education level was positive in all dimensions, there was a difference between education levels and the highest effect occurred at the university and middle school levels. In the study conducted by Cheung and Slav (2012), it was observed that there was a significant difference in favor of middle school students. On the other hand, in the study conducted by Wenglinsky (1998), it was observed that computer-aided education was more effective on elementary school students and a significant difference occurred.

Suggestions
In line with the findings of this research, the following recommendations can be developed: • As a result of the meta-analysis, it was seen that the effect of digital stories on academic achievement was strong. Therefore, teachers can use digital stories to increase student success in the educational environment.
• It has been determined that studies on the effect of digital stories on academic achievement are mostly conducted at the middle school level. By doing more studies in other education levels, healthier results regarding the effect size can be obtained in future meta-analysis studies compared to the education levels.
• It has been determined that studies on the effect of digital stories on academic achievement are mostly carried out in the field of Science. For some courses, the calculation could not be made because there was not enough research. By conducting studies in different fields, the effect size can be calculated for other courses in future meta-analysis studies.
• Meta-analysis studies can be conducted on the effects of digital stories on motivation, self-efficacy and collaborative learning, etc.
• More studies can be done to include more studies in future meta-analysis studies.