A Delphi Study to Determine Innovation Orientation Survey Items for Undergraduates in Malaysian Public University

This research aims to produce innovation orientation survey items for quantitative research use in the context of undergraduates in Malaysia. The hypothesized model of innovation orientation consists of creativity, innovation motivation, entrepreneurial orientation and entrepreneurial self-efficacy constructs. These five constructs was established employing comprehensive literature reviews. 24 items was adapted and three rounds of Delphi study were conducted to validate them. Participants were 20 Malaysian higher education institutions’ academicians. The initial 24 items to measure the five constructs went through refinement and contextualization to conceive innovation orientation for undergraduates’ survey. Consensus achieved after three rounds of Delphi study. The qualitative comments written by the participants was collected, coded and changes made were translated into the 24 items until the quantitative result for all items received moderate and high level of agreement.


Introduction
There is a need to have a generic and appropriate innovation orientation items measure for undergraduates in Malaysia.However, items for innovation orientation in higher education institutions context is currently not present.Most of the innovation orientation item-measure available was designed for businessman and entrepreneur.To directly adapt innovation orientation that is readily available to be employed into undergraduates setting may leads to inaccurate findings.This is because the item measures for innovation orientation in business are conceived for application in company and profit-making.Therefore this knowledge gap is present.Based on literature, there are many researches on innovation in education, but inconsistent definition for innovation orientation makes it difficult for researcher to replicate and further the research.This research seeks to conceptualize the items for innovation orientation questionnaire in Malaysian higher education setting.Based on Skulmoski et al. (2007) Delphi method is a well suited research method when there is incomplete knowledge about a problem or phenomena.Conceptualization of items for undergraduates' innovation orientation is the central phenomenon of research.
In business management and organizational research, innovation orientation is defined as the organization's strategic direction towards being innovative (Manu & Sriram, 1996).Innovation orientation is depicted by exercising innovativeness, demonstration of their openness to innovation (Zaltman & Duncan, 1977;Berthon et al., 1999) and demonstration of the capacity to innovate (Burns & Stalker, 1977).Amabile (2012) and Athuahene-Gima and Ko (2001) defined innovation orientation as a strategic direction and positivity towards risk.Siguaw et al. (2006) endorse that innovation orientation involves understandings and beliefs about innovation, such as innovation is a continuous and radical change, innovation happens by adoption of new methods, thus new knowledge is requisite for growth, sustainability, to be ahead of competitors and markets.For Siguaw et al. (2006), innovation orientation is the real source of competitive advantage and it shapes, guides, orients and coordinates ones' competencies that yield innovations and subsequent performance.
The dissimilarities between innovation orientation terminologies in real-business and in education or school are evident.This is true especially in term of students' limitation on accessibility to innovation resources.For example, eligibility for loan, financial aids, experience limitation due to the level of maturity and those alike.These are the threats to measurement instrument internal validity.Therefore, innovation orientation item measures findings in this Delphi study are useful for researchers in higher education.This research produces a set of innovation orientation quantitative survey items.The survey item is useful for baseline study on nurturing innovation orientation among the undergraduate students in Malaysian university.
In order to understand undergraduates' innovation orientation, this research identified five factors that stimulate undergraduates' innovation unfolding.These five factors are undergraduates' perception on their: innovation orientation, entrepreneurial orientation, creativity, innovation motivation and entrepreneurial self-efficacy.These were derived from theories.These factors were further conceptualized into the context of undergraduates in Malaysian university by the researchers and validated through Delphi study.The overarching theoretical framework that binds these factors is based on the Theory of Componential Creativity and Innovation (Amabile, 2012), and Social Cognitive Theory of Self-Regulation (Bandura, 1999).

Theory of Componential Creativity and Innovation
The Theory of Componential Creativity and Innovation (Amabile, 2012) was derived from interview study of personal qualities to model creativity and innovation in organizations by Amabile and Gryskiewicz (1987), and developed from experimental research of the Model of Creativity and Innovation (Amabile, 1983).According to the theory, innovation is built on creative ideas as the basic elements.Individual's innovation is the successful implementation of creative ideas.The term implementations that encompass elements of developing ideas and putting them to use clearly depict that creativity is an important factor to unfold innovation behaviour.Creativity is a skill that constitutes the individual's raw materials for innovation productivity.Herein lays the factor of innovation performance.Creativity gives a sustainable competitive advantage to an individual since it is a strategic resource, flexible, rare and imperfectly substitutable (Im & Workman, 2004).
Opportunity identification is also a driving force in innovation process.It is also dependent on the individual's ability to recognize potential or hidden entrepreneurial opportunities.Therefore, entrepreneurial orientation is a psychological factor that influences innovation behavior improves individual innovation performance and is also a strategic resource.Entrepreneurial orientation is one's strategic direction and willingness to turn ideas into money-making potential, either economic or social value.
Assuming that an individual is creative and entrepreneurial, an individual will not innovate if motivation relevant to innovation is lacking.Relevant motivation includes types of favorable drives to pursue and explore creativity.Therefore, the component of creativity and innovation in individuals should also include innovation motivation.No amount of creative skill can compensate for the lack of appropriate motivation to perform an activity.Task motivation determines the extent creativity will be engaged in the innovation performance.
The third component of innovation and creativity performance is the individual's perception of his or her capability to meet the standards of the task.One's perceptions appear to be dependent on social and environmental factor, this is self-efficacy.Self-efficacy is a psychological factor that is extrinsic to innovation work, as it controls one's intention to engage or disengage in innovation behavior.Self-efficacy has demonstrated negative impact on rewards and recognition.Motivational factor that compels an individual toward competence and satisfaction is also influenced by feelings of self-efficacy.

Social Cognitive Theory of Self-Regulation
According to the Social Cognitive Theory, individual innovation behavior is regulated by their socio-structural influence.Socio-structural influences operate through cognitive construction and regulation of human behavior in the service of diverse purposes (Bussey & Bandura, 1999, p. 13).Self-efficacy is defined as an individual's cognitive estimate of his or her capabilities to mobilize motivation, cognitive resources, and courses of action needed to exercise control over events in their lives (Wood & Bandura, 1989).One important effect of self-efficacy is on the choice of behavior settings.Individuals tend to choose situations in which they anticipate high personal control and avoid situations in which they anticipate low control (Bandura, 1982).They assess their personal capabilities against the requirements of the innovation behavior and to prepare for and behave innovatively in situations they feel more competent and efficacious.
People with high self-efficacy have more interest in the task, are more willing to expend their effort, and show more persistence in the face of obstacles and setbacks.As a result, they perform more effectively.On the contrary, people with low self-efficacy show no interest in the task, are reluctant to expend their efforts, and easily give-up.Hence, the conceptual model of the research proposed that entrepreneurial self-efficacy has a regulator effect on the relationship between factors.Entrepreneurial self-efficacy is hypothesizing as mediators on innovation motivation, entrepreneurial orientation, and creativity exertion into innovation orientation in the case of undergraduates' innovation orientation model in Malaysia.

Method
The aspect of contextual validity of the survey instrument is crucial.In this research, measurement instrument for innovation orientation and its items were adapted from the business context.Therefore, contextualization of the instrument into undergraduates' context was validated through consensus.Delphi survey technique was employed to build consensus and thus confirm the instrument's content validity.

Participant and Procedure
Twenty lecturers who are experts in various fields in Malaysian higher education institutions were participated throughout the three rounds of the Delphi study.Fifteen to twenty 15 to 20 participants are usually the optimal number (Ludwig, 1997).There is monolithic reliability value of group responses when the number of participants increases with coefficient of reliability nearing 0.9 for groups of 13 persons (Dalkey et al., 1972).Table 1 shows the relationship between error reductions to size of participants.

Constructs and Measured Variables
The survey questionnaire consisted of five constructs or factors that measure undergraduates' innovation orientation, entrepreneurial orientation, entrepreneurial self-efficacy, creativity, and innovation motivation.These factors' operational definitions delimit its measured variables.The operational definition for each construct links the theoretical concept to the empirical part of this research.They are more than one measured item that represents each construct.The items for each construct are guided by their operational definitions as follows.

Innovation Orientation
Innovation orientation variable is measured by four items adapted from Kuratko, Montagno and Hornsby (1990).Innovation orientation is operationally defined as one's perceived tendency to implement ideas into practice and manifested by four items: (1) interest to initiate new ideas, (2) committed to learn and accumulate knowledge, (3) courage to pursue one's own unique interest, and (4) producing new things to benefit others.

Entrepreneurial Orientation
Entrepreneurial orientation variable is defined as a person's perceived willingness to turn ideas into money-making potential (either economic value or social value) (Antoncic & Hisrich, 2003;Lumpkin & Dess, 1996).Entrepreneurial orientation is measured by five items adapted from Bolton and Lane (2012): (1) ability to recognize new opportunity, (2) incline to think ahead of trend, (3) sensitive to the demand and needs of the community, (4) wanting to put up ideas into business, and (5) willing to put effort to gain knowledge and skill to run a business.

Creativity
Creativity variable is defined as ones' perceived ability to generate ideas.The construct is measured by four items adapted from creativity dimensions developed by Amabile (1996): (1) actively seek to learn new and interesting things, (2) interested in exploring new ideas, (3) Like to do things differently, and (4) Like to focus and dwell on the positives in other people idea.

Entrepreneurial Self-Efficacy
Entrepreneurial self-efficacy is defined as one's perceived self-confidence in their performance in entrepreneurship.Entrepreneurial self-efficacy construct is measured by six items adapted from Chen, Greene and Crick (1998): (1) coping with unexpected challenges, (2) ability to create opportunity, (3) ability to define core purpose, (4) ability to develop human resource, (5) ability to create innovative environment, and (6) ability to imitate investor relationship.

Innovation Motivation
Innovation motivation is defined as one's perceived drive towards innovation.Innovation motivation construct is measured by five items adapted from by Sauermann and Cohen (2007).Perceived innovation motivation are measured by (1) the drive to commercialize my own product, (2) the drive to contribute to the society, (3) the drive to increased my financial independence, (4) the drive to heighten the level of responsibility, and (5) to benefit human kind.
In total there are 24 measured items were adapted and composed into a Delphi survey questionnaire.Each items used a five-point Likert scale: (1) Strongly disagree, (2) Disagree, (3) Neither Disagree nor Agree, (4) Agree and (5) Strongly Agree.Scores given by the panels are quantitative expression on whether they agreed with the statements.A blank column was provided by the side of each statement for comments.In the subsequent rounds, the quantitative scores and comments were complied.The Delphi results from the initial survey were placed next to each item.This is to allow the participants to re-evaluate their earlier comments and scores for each item with respect to the collective quantitative and qualitative results.

Delphi Survey Technique
Delphi survey technique was employed in this research to get a consensus from a group of academics in Malaysia regarding the measured variables' contextual validity.Delphi survey technique is a combination of quantitative and qualitative method (Dalkey & Helmer, 1963).It systematically combines opinion and assessment by experts towards achieving a mutual agreement as well as consensus (Linstone & Turoff, 1975).It is an alternative to communication among participants (Adler & Ziglio, 1996;Skulmoski et al., 2007).Another advantage of Delphi procedure is that the participants are allowed to express their opinions freely, without undue social pressure to conform to other panel members in the group.Through a series of iteration, participants were allowed to refine their views and comments on other panels' suggestion.
The optimum number of repetition of Delphi procedure determines the rigorousness of the findings.Altschuld (1993) suggested three rounds (repetition of procedure) are sufficient for stability of information.Therefore, in this study three rounds of Delphi procedure were conducted.The convergent input of respondents for Delphi procedure was assessed using median and partial quartile deviation as suggested by Holden (1992), and Rowe and Wright (1999).(Refer Table 2)

Delphi Round One
Panels' responses toward each item in the questionnaire for round one was analyzed to determine the median, percentile and quartile deviations.Table 3 shows 12 items possess a quartile deviation value of less or equal to 0.500; therefore all 12 items achieved high consensus.Eight items achieved moderate consensus, determined by the quartile deviation value of greater than 0.500 but less or equal to 1.000.Four items possessed quartile deviation value greater than 1.000; implying low consensus or no consensus.Prior to distribution of Delphi Round Two modifications were made on the five items based on the panel's comment.The panel suggested the items to be contextualised into higher education setting in Malaysia.Based on the experts' feedback, risk-taking attributes and risk tolerance in the context of undergraduate students at the university is viewed differently from the context of business.The item related to this comment is 'I can cope with the failures'.It is commented by one of the reviewers that the statement is too ambitious to be achieved by the students.It was suggested that the statement be changed to 'I can cope with unexpected problems'.
The panel also viewed that the ability of undergraduates to come up with a new product or services is far-fetch hope for undergraduate students in Malaysian higher education.One of the item which is related to this comment is 'I can develop a breakthrough product which is ready penetrate global market opportunity'.A general comment made by the reviewers is that the statement is too ambitious and only extraordinary entrepreneur can achieve this level.It is suggested that the term 'breakthrough' term should be replaced by the word 'new' so that it become less difficult to be achieved by undergraduate student.The revised item is 'I can develop a new product which is ready to penetrate global market opportunity'.
The statement 'I engage in innovation for enhancing the life quality' also received comment by the reviewers.The concept of 'life quality' in the item 'I engage in innovation for enhancing the quality' was considered too wide.The word 'human' was suggested to be included, hence the change to 'I engage in innovation for enhancing the human life quality'.
It was also argued by the reviewers that the statement 'I am able to create a business gap for new opportunity' is too idealistic to be inspired by the students and whether students have the capacity to create a business gap.The alternative suggested was 'I am able to find a business gap for new opportunity' with the word 'find' used instead of 'create' so that it is achievable from the perspective of the students.
The word 'nation' in the statement 'I want to improve my nation through innovation' was considered too much to expect from the students as it covers a bigger population.Thus, it is recommended that the word be replaced by 'society' since 'society' is closer to the student and it can it interpreted as just the local community where the student lives.Thus, the statement becomes 'I want to improve the society through my innovation'.Table 4 summarizes the comments given by the Delphi study panel in Round One.The word 'nation' was considered too much to expect from the students as it covers a bigger population.Thus, it is recommended that the word be replaced by 'society' since 'society' is closer to the student and it can it interpreted as just the local community where the student lives.
I want to improve the society through my innovation.
I engage in innovation for enhancing the life quality.
(Partial deviation, x=1.500) The concept of 'life quality' was considered too wide.The word 'human' was suggested to be included.
I engage in innovation for enhancing the human life quality.I can cope with failures.(Partial deviation,x=1.500)Risk-taking attributes and risk tolerance in the context of undergraduate students at the university is viewed differently from the context of business.The statement 'I can cope with failures' is too ambitious to be achieved by the students.

I can cope with unexpected problems.
I am able to create a business gap for new opportunity.Partial deviation, x=1.500 The statement 'I am able to create a business gap for new opportunity' is too idealistic to be inspired by the students and whether students have the capacity to create a business gap.'I am able to find a business gap for new opportunity' with the word 'find' used instead of 'create' so that it is achievable from the perspective of the students.
I am able to find a business gap for new opportunity.
I can develop a breakthrough product which is ready to penetrate global market opportunity.Partial deviation, x=1.500 The statement is too ambitious and only extraordinary entrepreneur can achieve this level.I can develop new product which is ready to penetrate global market opportunity.

Delphi Round Two
Table 5 shows the median, percentile and quartile deviation for round two.A total of 12 items possessed quartile deviation of less or equal to 0.500; indicating high consensus.Nine items achieved moderate consensus, with a quartile deviation value greater than 0.500 but less or equal to 1.000.Three items possessed quartile deviation value greater than 1.000; indicating low consensus or no consensus.The comments in Delphi Round Two were considered and refinement of item statements in the questionnaire was made prior to distribution of Delphi round three.The result from the Round Two Delphi Study showed some improvements in terms of level of consensus achieved plus the value of partial quartile deviation.Nevertheless, based on the comments given by the reviewers, some modifications were made to several items to make the items more relevant for students at higher education institutions.The main concern highlighted by the panels is the difficulty level of the items.The reviewers felt that the items could be toned down so that the traits stated are achievable by the students in higher education institutions.Although modifications have been done based on the comments given at the Round One Delphi Study, the results of Round Two Delphi Study showed that those items were still considered problematic in terms of appropriateness in the context of higher education scenario.
The statement 'I can cope with unexpected problems', was considered to be quite negative, 'Challenge' was the preferred word to show the positive quality of a student as an entrepreneur.The statement 'I can develop a new product which is ready to penetrate global market opportunity' was modified to be more practical as the students are just at the beginning phase of joining the business world.The students are still at the learning process therefore, 'global' was suggested to be excluded.
For the item 'I engage in innovation for enhancing the human life quality', 'human life quality' was changed to 'the benefit of human kind'.'The benefit of human kind' was considered a straightforward phrase, which can be easily understood by university students.The item 'I am able to find a business gap for new opportunity', was modified to exclude 'find a business' gap since it is considered too much for university students as they are not experienced enough to analyze the gap in real world business.Finally, the word 'improve' in the statement 'I want to improve the society through my innovation' was replaced with 'contribute' so that the statement will not be too optimistic.According to one of the reviewers, 'improve' seems to be quite pushy, 'contribute' is considered to be more diplomatic and a softer trait to be exhibited by novice innovator or entrepreneur such as university students.Table 6 summarizes the comments and revised statements in Delphi Round Two.The word 'improve' in the statement 'I want to improve the society through my innovation' was replaced with 'contribute' so that the statement will not be too optimistic.The word 'improve' seems to be quite pushy, 'contribute' is considered to be more diplomatic and a softer trait to be exhibited by novice innovator or entrepreneur such as university students.
I want to contribute to the society through my innovation.
I engage in innovation for enhancing the human life quality.
Partial deviation, x=1.000 'Human life quality' was changed to 'the benefit of human kind'.'The benefit of human kind' was considered a straightforward phrase, which can be easily understood by university students.
I engage in innovation for the benefit of human kind.
I can cope with unexpected problems.

Partial deviation, x=1.500
The statement 'I can cope with unexpected problems', was considered to be quite negative, 'Challenge' was the preferred word to show the positive quality of a student as an entrepreneur.
I can cope with unexpected challenges.
I am able to find a business gap for new opportunity.

Partial deviation, x=1.500
The item 'I am able to find a business gap for new opportunity', was modified to exclude 'find a business' gap since it is considered too much for university students as they are not experienced enough to analyze the gap in real world business.The statement 'I can develop a new product which is ready to penetrate global market opportunity' was modified to be more practical as the students are just at the beginning phase of joining the business world.The students are still at the learning process therefore, 'global' was suggested to be excluded.I can develop new product and market opportunity.

Delphi Round Three
Table 7 showed the results for Delphi round three with 14 items having quartile deviation value less or equal to 0.500; indicating high consensus.Ten items achieved moderate consensus, determined by the quartile deviation value greater than 0.500 but less or equal to 1.000.No item obtained a quartile deviation value of greater than 1.000; indicating low consensus or no consensus.All items achieved high and moderate consensus implicating the panels are satisfied with the items in the questionnaire.Through the Delphi procedure consensus, contextual validity of the 24 items was achieved confirmed by the twenty-panel members.The contextual validity of the instrument is established i.e. the instrument conforms to its conceptual definition and each constructs' measured variables are unidimensional.The instrument is contextually suitable to be deployed in the context of undergraduates in Malaysia.Table 8 summarizes the items.

Discussion and Conclusion
The way this Delphi survey measure innovation orientation, entrepreneurial orientation, innovation motivation, creativity and entrepreneurial self-efficacy is more of deductive approach.This research simultaneously created a new set of survey instrument for measuring undergraduates' innovation orientation in Malaysia.The measurement items connect how undergraduates perceived innovation orientation and its factors.Prior to Delphi process the researchers make a preliminary literature review to define the items for establishing a set of items to start the consensus process.Conceptualization of constructs took abstract definition and produced measured items that linked to the constructs.Agreements on each of the twenty four items that represent their constructs also signify how innovation orientation can be enabled among undergraduates in Malaysia.In the Delphi survey, the aspect of item clarity and explicit meaning develops sound explanations for a construct.A single construct has several definitions, and the Delphi panel may disagree over the definitions thus consensus among experts is required (Neuman, 2011, pp.201).
One of the strength of Delphi method is the operationalization course deviates from rigid process because as the study continues, participants can revisit each of their own comments and other participants' comments.This gives them the chance to learn more about the issues based on the interests of other participants.It is an interactive process and participants can explore options, and may rethink about their own comments.Minimum requirements for agreement was made based on computation of all median value, percentile value and quartile deviation value for each item in Delphi questionnaire.In this research, to reach consensus effectively the refinement of items' statements focus on improving four items that received low agreement and one item that received marginally moderate agreement.
Delphi technique was used due to its advantage for overcoming unfair and impasse strong emotions dispute during the consensus process.The process begins with theoretical frameworks and definition of each constructs.
Then, a group of academics was selected to seek consensus on the set of 24-item measures of innovation orientation survey.The academics who participated in the Delphi study represent a different constituency within public university in Malaysia.The selecting criteria for a participant are their availability to work within the budgeted time plan and work experience in higher education.
On the other hand, Delphi technique was employed to improve reliability of the survey instrument particularly on clarity of constructs' conceptualization.Sound-quality measures develop credibility of a research finding.
Reliability of an instrument can be increased when each measure indicates unidimensionality, meaning that they indicate only one concept.However, if only one measured item used to measure a construct it will lead to bad or imperfect measurement (Neuman, 2011).Therefore, the use of multiple items for a construct increases reliability.
In this research innovation orientation has four items, entrepreneurial orientation has five items, creativity has four items, entrepreneurial self-efficacy has six items, and innovation motivation has five items.As a conclusion, multiple items are more stable than single item measure and captured the essence of the construct.
This research is not without limitation.The limitations of this research are the lacks of ability to read nonverbal cues because researchers infer conclusion based on the written response system.The researchers need to be discrete in investigation and comprehend each participant's comments.The triangulation process of participants' comments is quite time-consuming for the researchers.In short, most of the limitation of the study is due to the inability of the researcher to employ consensus techniques that are available in face-to-face settings.

Table 5 .
Level of consensus on items in Delphi round

Table 1 .
Relationship between error reductions to size of participants

Table 2 .
Determination of consensus

Table 3 .
Level of consensus on items in Delphi round 1

Table 4 .
Summary of comments and revised item after Delphi Round One

Table 6 .
Summary of comments and revised item after Delphi Round Two

Table 7 .
Level of consensus on items in Delphi round 3

Table 8 .
List of items