A Meta-Analysis Study on the Effectiveness of Creativity Approaches in Technology and Engineering Education

There are numerous approaches aimed at developing creative capacities which have been proposed. These have been implemented in various fields including science, business, nursing, technology as well as engineering. This study used a quantitative method of meta-analysis to synthesize the effect of several approaches to creativity, specifically in the field of technology and engineering across educational levels. Several databases such as EBSCOhost, ProQuest, Taylor & Francis, SpringerLink, IEEE Electronic Library, Eric, and Google Scholar were searched for articles on studies about the effectiveness of approaches to creativity or issues related to the topic within a 2000-2012 timeframe. Based on 16 articles adopted, it was found that the grand mean effect size for creativity approaches is 1.02 with a standard deviation of 0.72. This study also identified seven approaches and twelve different instruments used to enhance and measure creativity in terms of product and individual creativity. The findings showed that the implemented creativity approaches have a positive effect on students’ creativity development. As such, it is recommended that further empirical research on the effect of creativity approaches be conducted, specifically in technology and engineering fields, to support these findings.


Introduction
The evolution of civilization depends on innovation, while innovation itself highly depends on the creative ability possessed by every individual in these groups (Ma, 2006).In the new economy environment and well-developed technology, more innovation is needed to drive the growth and development of a country.Since creativity is the basis of innovation, there is no doubt that creativity is important to be fostered among individuals, especially students who are liable to be a catalyst in modernization and development of a country.
What is creativity?Creativity has been defined through numerous perspectives.However, it is generally defined as people's ability to produce products that are not only high in quality but are also novel (Sternberg, 2001).According to Lubart (2000Lubart ( -2001)), creativity involves the production of original, potentially usable as well as novel solutions to problems of relatively high level of difficulty.The realization of one's creative ability depends not only on knowledge and skills, but also on the use of quick and different kinds of information found in tasks requiring this ability (Karkockiene, 2005).
According to Investment Theory proposed by Sternberg & Lubart (1993), creativity requires a confluence of six distinct but interrelated resources: intellectual abilities, knowledge, styles of thinking, personality, motivation, and environment.These resources combine interactively to yield creative performance.Three aspects of intellectual abilities are key for creativity: (a) the synthetic ability which refers to the ability to see problems in new ways and to escape the bounds of conventional thinking, (b) the analytical ability which refers to the ability to identify whether an idea can be applied or not, and (c) the practical-contextual ability which refers to the ability to translate abstractions and theories into realistic application.It is a skill to sell or communicate one's ideas to others and also to convince others that the ideas, work or products are valuable and should be considered (Aqda et al., 2011;Sternberg, 2006).Creative people use these three types of thinking to consider the implications and possible responses to a problem (Aqda et al., 2011).
In addition, based on the Investment Theory (Sternberg & Lubart, 1993), creative individuals are also claimed to be those who buy low and sell high in aspects related to the idea.They invest their thinking in ideas that are considered less clear and less well known, and sell those ideas high as a result of creative input.They will continue with the same routine until their idea is generally accepted and then start to generate more new ideas which are to be invested to meet the needs and desires of the community.
The importance of creativity in our life can be seen from many aspects.For instance, creativity has been considered as a predictor of one's outstanding achievement in arts and sciences, it has been linked to the development of new social institutions and the leadership of extant institutions, it has been instrumental in entrepreneurial activities and long-term economic growth and most obviously, it has been associated with well-being and successful adaptation to the demands of daily life (Scott et al., 2004).Realizing the importance of creativity in daily life, it begs the question of how do we stimulate people's creative effort?Several approaches have been used to encourage creativity, including providing effective incentive or rewards (Eisenberger & Rhoades, 2001), optimizing climate and culture (Isaken & Akkerman, 2011), and providing training or programs to enhance creativity (Ma, 2006;Scott et al., 2004).Among these approaches, training programs and also intervention in teaching and learning methods have been the choices in enhancing creativity.Both organizations and educational institutions have invested a lot of resources, time and energy in developing and implementing a range of programs to encourage and enhance creativity.These training programs have been developed for almost every age group, from preschool to tertiary level institutions (Scott et al., 2004) and not forgetting those in the working world (Coleman, 2010).Thus, it clearly shows that the creativity of a person can be nurtured and enhanced through various trainings or appropriate approaches which are implemented continuously.This assertion is supported by Karkockiene (2005) who states that creativity can be enhanced because the human capacity can be met.He added that, in addition to creativity, the components associated with it such as cognitive, affective, psychomotor, and interpersonal components can be enhanced through the provision of an environment that fosters the generation of ideas and the creation of a solution to the problem.
Based on a literature review, numerous training programs have been developed with the aim to enhance creativity of students in various educational fields such as science, business, psychology, nursing, technology, and engineering.Therefore, this study aims to investigate the effects of various types of training programs as well as teaching and learning strategies on creativity at the preschool, school or university level with the use of quantitative meta-analysis.However, this study focuses only on the effectiveness of training programs implemented in the areas of technology and engineering as, to date, no such studies have focused on the two areas.The questions underlying the discussion of this study are as follows: 1) What types of approaches (training programs or teaching and learning strategies) are used to enhance creativity?
2) What kinds of instruments and criteria are used to assess creativity?
3) What is the effect size of each approach?4) Are the approaches used effective in enhancing creativity across educational levels?

Approaches to Creativity
In this study, the term 'approach' refers to training programs or teaching and learning methods that have the same goal i.e. to promote and enhance creativity.An approach that is often used is the Creative Problem Solving (CPS) technique which is integrated into the teaching and learning process (Chang, 2012;Hokanson, 2006;Mrymel, 2003).The CPS technique involves the process of identifying problems or challenges, generating ideas and implementing innovative solutions to produce a unique product (Mrymel, 2003).There are five steps involved in the CPS technique, namely fact finding, problem finding, idea finding, solution finding, and acceptance finding (Isaksen & Treffinger, 2004).Basically, students are trained to use divergent thinking at the beginning to generate as many ideas or solutions as possible and then they have to change the way of thinking to convergent thinking to select the most effective ideas or solutions (Vidal, 2003).Such activities are able to shape creative thinking and increase creativity.
The brainstorming technique is also one of the commonly used approaches in enhancing students' creativity.It involves creating ideas and sharing them with others without evaluation.This technique requires students to study the new information using all senses to tackle the problems and try to relate them to everyday life apart from sharing their innovations (Ma, 2006).Nemeth et al. (2004) stated that brainstorming instructions improve group creativity because they address issues of evaluation apprehension and social loafing.To lower these issues, individuals are encouraged to emphasize quantity of ideas and more importantly, they are instructed not to criticize their own and others' ideas.Rather, they are encouraged to build upon and elaborate others' ideas.In implementing this approach, the instructor should follow the following six rules (Lui & Schonwetter, 2004;Rossiter & Lilien, 1994): 1)Provide direction and emphasize quantity rather than the quality of the idea generation.
2)Set goals that are quite difficult for some ideas.
3)Asking individuals rather than groups to generate initial ideas.4)Using groups to integrate and refine each idea generated.5)Ask individuals to provide the final evaluation and choose the best idea.6)Ensure time is allocated for brainstorming.
Besides that, computer-aided teaching method is also preferred by instructor in the teaching and learning process in order to enhance creativity.Gokhale (1996) used computer simulation activities to identify their effect on students' problem solving abilities.These activities have been integrated into a traditional lecture-lab sequence.He found that students who use software simulation activities performed significantly better on the problems than students who were taught only using traditional lecture-lab instruction.According to Veenam et al. (1994), problem-oriented computer simulations help develop higher-order thinking and improve students' cognitive abilities which function in the process of recalling, generating creative ideas and solving problems.

Assessment of Creativity
Several methods have been applied in determining whether the product generated is creative or not.Creative products, which include ideas, solutions, performances or products generated by the creative process can be classified into two aspects: firstly, ideation without evaluation and secondly, ideation with evaluation (Ma, 2009).Each aspect has its own criteria that can be measured.One method used in assessing the creativity of a product depends on the judgement of others (Scope, 1998).This method is known as the Consensual Assessment Technique (CAT); it was first introduced by Teresa Amabile in 1982 (Baer & McKool, 2009).According to Kaufman et al. (2008), in making an assessment of creativity in the real world, it is common for panels of experts in a given domain to be asked to evaluate the creativity of a creative product or group of products.Baer & McKool (2009) also noted that the most valid assessment on the creativity of an idea or product in any field is based on the combined opinions of experts in the fields.
Another method of measuring creativity involves the use of tests.The answers to a given test are considered evaluated products (Scope, 1998).The Torrance Test of Creative Thinking (TTCT) is the most frequently used instrument in measuring creativity specifically related to divergent thinking (Ma, 2009) in both child and adult populations in studies.Students' responses are evaluated based on fluency (many relevant responses), flexibility (different categories of relevant responses), originality (novelty of the responses), and elaboration (the amount of detail in the responses) (Al-Ali Khaled et al., 2011).Other instruments such as the William Creative Thinking Test (WCTT), the Wallach and Kogan Creativity Test and the Creative Product Semantic Scale (CPSS) are less frequently used and less thoroughly researched.They do however continue to be used in experimental studies to measure creativity (Scope, 1998).

Previous Meta-analysis on the Effectiveness of Creativity Approaches
In this study, some evidence has pointed out the effectiveness of creativity approaches.For example, Torrance (1972) reviewed the findings from 142 studies designed to test the effectiveness of the approaches that are used to teach children to think creatively.From all of the reviewed studies, 103 studies used the Torrance Test of Creative Thinking (TTCT) as criteria for evaluation.The study showed that 72% of the training programs that have been done were successful; the most successful training program was the Osborn-Parnes Creative Problem Solving program and its modifications.According to Torrance (1972), the most successful approaches to teach children to think creatively seem to be those that involve both cognitive and emotional functioning, provide adequate structure and motivation and give opportunities for involvement, practice, and interaction with teachers and other children.Scope (1998) conducted a meta-analysis to study the effects of instructional variables on creativity.The instructional variables defined by Scope (1998) include the review of the taught content or materials, the structuring of new teaching materials which can be in the form of an overview, advance organizers, outlines, and review of objectives, as well as a few other variables.He identified 30 studies that produced 40 effect sizes with a mean value of 0.90 (SD = 1.19).Scott et al. (2004) also used the same method to evaluate the effectiveness of creativity training.They had two objectives in their study: firstly, to provide a reasonable assessment of the overall effectiveness of creativity training using quantitative analysis of previous program evaluations and secondly, to identify the main features in the training content and delivery methods that influenced the success of training programs.They classified the dependent variables into four categories: divergent thinking, problem solving, performance, and attitude and behaviour.The overall effect size obtained was 0.68 with a standard deviation of 0.65.The main difference between the study conducted by Scott et al. (2004) and previous meta-analysis studies is that Scott et al. (2004) had done a more detailed study on internal and external validity.In other words, they tried to identify the effect of moderators.
A recent study which assessed the effectiveness of creativity training was performed by Ma (2006).His research focused on the effectiveness of creativity training using a single component (such as brainstorming) and a package component (such as computer-aided training).According to him, creativity training will be more effective and the process of creativity will be better understood if the effectiveness of single component and package components of creativity training are identified.Overall, 268 effect sizes have been produced from 34 studies and the grand mean effect size of creativity training is 0.77.The effect of moderators such as dependent variables, training period, and the design of experiment was not significant.Hence, he suggested more experimental studies be conducted in order to evaluate the effectiveness of each approach in enhancing students' creativity.
Based on the aforementioned meta-analysis studies, none of them have attempted to synthesize the effect of creativity approaches in a specific area.Therefore, the aim of this study is to narrow down the meta-analysis into a specific area by focusing on the effectiveness of creativity approaches in the field of technology and engineering.The reason for choosing technology and engineering as a main focus is because of the emphasis on creativity development of individuals or groups so as to ensure that they are viable and competitive in the global economic world.For example, technology education offers a good potential environment to develop creative problem-solving abilities and creative behaviours among students (Buelin-Bieseker, 2012).This is in line with technology education objectives which state that the students should be able to identify and solve problems and make decisions using critical and creative thinking (Vandeleur et al., 2001).Creativity and innovation are clearly stated in the goals of technology education.The ability to create a dynamic learning environment helps to support creative aspirations (Buelin-Bieseker, 2012).
According to Charyton et al. (2011), greater emphasis has been given on increasing creativity in upcoming engineers due to the importance of creativity in engineering as profession.Cropley & Cropley (2000) raised a similar issue of fostering creativity in engineering education.Engineering students also realize that creativity is very important to learn.In a survey conducted by Zampetakis et al. (2007), 87% of engineering students agreed that creativity was a skill that is required for engineering and 77% of engineering students stated that they would like to take courses related to creativity and creative problem solving.Student acceptance of the importance of creativity can be considered as the first step towards the development of university programs to foster creativity and effective implementation of techniques such as mind mapping to enhance creativity (Zampetakis et al., 2007).

Research Design
Meta-analysis is a research methodology which, according to Gliner et al. (2003), is composed of synthesis methods that use a measure of effect size to indicate the strength of relationship between the treatment and dependent variables involved in the study.They added that by performing a meta-analysis, a researcher can have some advantages such as the computation of a summary statistic for a large number of studies.This summary statistic provides an overall estimate of the strength of relationship between independent and dependent variables.Another advantage of meta-analysis is that it provides evidence of the reliability of a research finding and also increased external validity.

Selection of Studies
The selection of studies used in this meta-analysis was started by searching through several databases such as EBSCOhost, ProQuest, Taylor & Francis, SpringerLink, IEEE Electronic Library, Eric, and Google Scholar.A variety of search terms or keywords were used in locating appropriate studies; for example, "creativity approach", "creativity training", "brainstorming activity", "divergent thinking", "technology education", "engineering education", and a few others.Systematic study was carried out on two journals, namely the Journal of Creative Behavior and the Creativity Research Journal.In addition, several studies were identified through the reference of selected articles.Only articles published from 2000 until 2012 were considered in this study.As a result, 51 studies were successfully indentified as potential samples.However, only 16 studies qualified for use in this meta-analysis after filtering the articles according to the following sample selection criteria: 1) It should focus on the effectiveness of the approach in nurturing and enhancing creativity, whether it is a training program or intervention in the teaching and learning method, specifically in the field of technology and engineering.
2) It is an empirical research where the study should use the quasi-experimental or experimental method that involves treatment and control groups.
3) It has made a clear discussion about creativity assessment procedures, the type of instrument used and how the scores are given.
4) It provides sufficient statistical data for the calculation of effect size.

Coding Procedure
This study adopted a sample coding procedure introduced by Ma (2006;2009).The data required to be coded were the following: articles, definition of the independent variables, definition of the dependent variables, instruments used, sample size of the treatment group (Ne), sample size of the control group (Nc), mean value of the pre-test on treatment group (Me1), mean value of the pre-test on control group (Mc1), mean value of the post-test on treatment group (Me2), mean value of the post-test on control group (Mc2), standard deviation of the control group in the pre-test (SDc1), standard deviation of the control group in the post-test (SDc2), t-value, F-value, and χ 2 values.All data were recorded in a Microsoft Excel spreadsheet for calculation purposes.

Calculation of Effect Size
Effect size is used to indicate the strength of relationship between independent variables and dependent variables in standard deviation (Gliner et al., 2003).In general, the effect size is obtained by calculating the difference between the mean of the treatment and control groups on dependent variable over the standard deviation of the control group (Glass et al., 1981).However, for studies which do not provide the mean and standard deviation values, effect size can be determined by using the value of the statistical tests such as t, F, and χ 2 .An appropriate equation which is used for calculating effect size depends on the source of empirical data available for each study.Ma (2006) summarized six effect size calculation formulas taken from Cooper & Hedges (1994).Four of them were used in this study and are stated as follows: In this study, a standard value and accuracy of the effect size follow Cohen's (1988) guidelines, as shown in Table 1: Table 1.Category of effect size Source: Jackson (2006)

Results
A total of 16 studies focusing on the effectiveness of approaches to creativity were selected as samples in this meta-analysis.Two studies are in the form of thesis and the rest are journal and proceeding articles.The results of this study should be viewed with an understanding of some methodological limitations.One of them is that only studies focusing on the effectiveness of approaches to creativity, specifically in the field of engineering and technology, were considered in this meta-analysis.This has caused a very limited number of studies to be used.

Effect Size (ES)
Category of Effect Size 0.20 Small 0.50 Medium

>0.80 Large
According to Becker & Park (2011), small numbers of samples could lead to an inflation of results and a tendency to over-reach the conclusions.However, it is impossible to get a large number of samples since many creativity approaches proposed specifically in the field of engineering and technology still remain in the form of opinion or review papers without empirical data.
Overall, 42 effect sizes were produced from the 16 studies.The effect sizes were obtained from various statistical data: 21 were from mean and standard deviation of post-tests, 10 were from values of F, six were from mean and standard deviation of pre-and post-tests and two were from t-value.The other three effect sizes were obtained directly from the articles reviewed.The overall mean of effect sizes is 1.02 with a standard deviation of 0.72.According to Cohen's guidelines (1988) (see Table 1), the mean value of effect sizes, 1.02 was greater than 0.8; thus, it is categorized as a large effect size.Further analysis of results was carried out according to the four research questions that were mentioned earlier:

Approaches to Creativity and Effect Sizes
The results in Table 2 specifically answer the first and third research questions in this meta-analysis.Seven approaches to enhance creativity were identified.The most frequently used are Creative Problem Solving (CPS)/problem solving and computer/technology applications which involve five studies, respectively.This is followed by inventive problem solving or TRIZ methods which involve two studies.Other approaches such as integration of creativity lectures, case studies and creativity counseling approach, group plays for pre-school students approach, operational mechanisms of group composition approach and group interactive brainstorming approach involved only one study each.In terms of effect sizes, all of the approaches used had effect sizes ranging between 0.32 (smallest) and 1.72 (largest).Five approaches had effect sizes that are categorized as large and the other two are medium.Group plays for preschool students show the highest value of effect size, whereas group interactive brainstorming gives the smallest value of effect size.The findings can only be generalized within the scope of this study due to the small number of samples that were considered.However, value of effect size (small or large) is meaningful in describing the degree of effectiveness of the approaches used in enhancing creativity in a particular study.

Instrument and Creativity Assessment Criteria
A detailed study of the selected articles found 12 different instruments used to evaluate the individual or product creativity.As shown in Table 3, eight instruments were used for evaluating individual creativity and the rest were used for evaluating product creativity.Individual creativity assessment mainly focused on divergent thinking aspects such as fluency, flexibility, originality, and elaboration, such as that used in the Technological Creativity Test (Chang 2012) and the Torrance Test of Creative Thinking (TTCT) (Al-Ali Khaled, 2011;Hokanson, 2006;Shawareb, 2011).Meanwhile, product creativity assessments such as the Creative Product Semantic Scales (CPSS) (Michael, 2001) and consensual assessment technique (Cropley & Cropley, 2000) involved effectiveness, novelty, originality, elegance, and usefulness as evaluation criteria.

Individual
Measures creativity in four subscales: intuitive awareness, external locus of control, self confidence, and originality.

The Effects of Creativity Approaches across Educational Levels
In Table 4, the effect sizes of creativity approaches for 16 studies by educational levels are distributed according to Cohen's guidelines (1988).Educational levels are broken down into three levels: pre-school, school, and university.The numbers of studies that were implemented at the university, school and preschool level are 8, 6, and 2, respectively.At the pre-school level, studies done by Rizi et al. (2011) and Shawareb (2011) showed large effect sizes of 1.72 and 1.33, respectively.At the school level, four studies (Barak, 2012;Chung & Ro, 2004;Eow et al., 2010;Myrmel, 2003) showed large effect sizes of between 0.80 and 2.28, while the other two studies (Chang, 2012;Michael, 2001) had medium (0.27) and small (0.19) effect sizes, respectively.In Michael (2001), the effect size produced was small due to the similarity between activities in computer simulations and hands-on tasks.In this case, product creativity may be concluded to have more dependence on individual cognitive and creative abilities rather than on the device or the way of producing a product.Thus, in assessing the impact of computer simulation activity on product creativity of students, researchers may be able to focus on the creative individual characteristics and cognitive processes used to create products than focusing on the tools or the way the product is produced.However, the effect of a number of approaches for development of creativity in schools is significant as the majority of the effect sizes are large.Finally, at the university level, five studies (Al-Ali Khaled, 2011;Eyadat & Eyaat, 2010;Hokanson, 2006;Seechaliao et al., 2011;Warr & O'Neill, 2005) showed large effect sizes of between 0.98 and 3.44.The other three studies (Cropley & Cropley, 2000;Hernandez et al., 2012;Park-Gates, 2001) presented medium effect sizes of between 0.32 and 0.65.The overall mean of effect sizes at different educational levels are 1.53, 1.08, 1.15 for pre-school, school, and university, respectively.

Discussion
The results of this study show that the overall mean effect size of creativity approaches (1.02) is large.The value is also larger than the overall mean effect size obtained by Scope (1998: 0.90), Scott et al. (2004: 0.68) and Ma (2006: 0.77).However, it should be bear in mind that the number of samples involved in this study is smaller than the other three studies.Therefore, researchers cannot over-reach conclusion.Seven approaches used in enhancing creativity were identified.Creative problem solving and application of computers/technology in teaching and learning were most frequently used.According to Chang (2012), problem-oriented learning activities which end with the appearance of a concrete solution is most effective in fostering creativity.Besides that, computer and other related modern technology also influenced creativity (Eyadat & Eyadat, 2010).In contrast to traditional teaching methods, the use of computers can help produce a creative technique by providing information in different ways.This prompts individuals to come up with creative ideas that ultimately help them master the skills of thinking (Al-Ali Khaled et al., 2011).
In addition, this study also identified 12 different instruments used to assess creativity.The difference between them can be seen in terms of scoring processes and the dimensions of creativity evaluated on individuals or products.Basically, individuals' creativity can be assessed using divergent thinking elements, while creativity of products are often assessed based on the judgement of experts.Finally, this study reported mean effect sizes of creativity approaches across educational level i.e. 1.53 for preschool, 1.08 for school, and 1.15 for university level, respectively.Based on effect sizes obtained, it can be concluded that approaches used in promoting and enhancing creativity skills across various levels of study were effective.
In fostering children's creativity, Rizi et al. (2011) believe that playing games has the most effective role in learning and can be considered as indirect learning opportunities for the development and enhancement of creativity.Therefore, teachers should consider the spirit of games in training activities.Moreover, according to Shawareb (2011), preschool is a critical stage for the development of creativity.Because, in the first four or five years, 60 percent of a child's intelligence matures and their basic developmental skills begin to grow.The application of technologies such as computers and the Internet that are appropriate to the needs and ages of preschool children are very effective in improving creativity.Computer, Internet and other technological devices act as scaffolding for the formation of concepts and cognitive development of children.This is in line with Vygotsky's theory that recognizes the importance of scaffolding in helping children enter the social environment and build new competencies.The findings of this study also confirm the results of Torrance's (1972) investigation which states that a child can actually be taught to think creatively.
In addition, Eyadat & Eyadat (2010) stated that creativity is important for students, especially at the tertiary level.
Students at this level must have the ability to see things from different angles, analyze problems, produce appropriate answers, and also have an open mind.These skills are essential to prepare the workforce in facing the future.Engineering students, for example, require knowledge, skill, experience, and creative thinking to analyze, design, and solve problems.This is because as an engineer, he or she will always face complicated problems (Seechaliao et al., 2011).Therefore, teaching and learning that emphasizes the development of creativity in technology and engineering students must be actively implemented to produce human resource that are capable of solving problems related to their work in order to shape the nation's future development.The results of this meta-analysis can be a guide for teachers and lecturers in choosing appropriate approaches as well as assessment methods to enhance creativity.

Conclusion
The findings of this study imply that there is a need for increasing further empirical research on the effectiveness of creativity approaches, especially among technology and engineering students.An effective and appropriate approach to cognitive development must be identified to instill creativity at an early stage.This is because students in technology and engineering fields rely heavily on creative thinking to excel in their chosen fields.Teachers or trainers should increase the use of learning approaches involving real-world problem solving to enhance students' creative ability and higher order thinking skills apart from preparing them for similar situations in their future career.Therefore, it is important for research such this to be implemented continuously in order to provide the latest information on the effectiveness of creativity approaches as it can be used as a guide for educators in selecting appropriate approaches to be applied in the teaching and learning process.

Table 2 .
Analysis based on approaches to creativity and effect sizes

Table 3 .
Instrument and creativity assessment criterion

Table 4 .
Effect sizes of creativity approaches across educational levels