Employability Skills Assessment Tool Development

Research nationally and internationally found that technical graduates are lacking in employability skills. As employability skills are crucial in outcome-based education, the main goal of this research is to develop an Employability Skill Assessment Tool to help students and lecturers produce competent graduates in employability skills needed by the industry. The employability skill Assessment Tool were developed using the Kepner-Tregoe (K-T) method. Samples were 107 employers from five types of Malaysian manufacturing industries. The findings showed that employers agreed on the importance for all seven of the employability skills; interpersonal skills, thinking skills, personal qualities/values, resource skills, system & technology skills, basic skills and informational skills. These skills were ranked and chosen as items for the Employability skills assessment tool. The tool developed was tested and validated by manufacturing employers and lecturers in institutions. The agreement coefficient was found to be between substantial agreement and almost perfect agreement.


Introduction
Technology is going through rapid changes that require a more skilled workforce to fulfill the needs and requirements of industry (Taylor, 2005).There is gap between graduate attributes not only their employment readiness but also their employability skills (Freudenberg et al., 2011).
Employability skill is often debated with various interpretations biased toward stating that employability skill is a preparation for graduates to successfully get jobs and to develop in their careers.Employees with employability skills and competent in technical skills are considered an asset to employer.The human capital theory states that through social psychology perspective, "employability" is not only about shaping talent, techniques, and experience for an individual to get a job, but more toward the ability to do the work.The main goal is the critical ability, reflective to convince and upgrade an individual with the skill other than his special or technical skill (Buck and Barrick, 1987;Lankard, 1995;Saterfield et.al., 1995;Fugate et.al.,2004;Rothwell & Arnold, 2007).
In term of competence development, competence is define as the ability of a person to handle and cope or complete a task or job successfully.This capacity covers the perceptual motor skills cognitive factors, affective factors, personality traits and social skills (Ellström &Kock, 2008).Using this definition as a point of departure, the notion of qualification may now be defined as employability skills that are required by the employer.

Employability Skills Issues
Findings from Ramlee (2002) on the role of vocational and technical education in the industrialization of Malaysia from the perception of educators and employers states that technical graduates in Malaysia has mastered their technical skills.Employers however felt dissatisfied because their employees lacked motivational skills, communication skills, interpersonal skills, critical thinking, problem solving and entrepreneurship skills.
Syed Hussain (2005) found that 62.3% of graduates in the technical field are still jobless because they lack employability skills rather than technical skills required by the industry.Research done by Mohamed Rashid (2005) on polytechnic graduates also found that about 50.5% of technical graduates of Polytechnic are jobless for almost nine months of the year because they lack employability skills.He found that these graduates are weak in communication, writing, and computer skills.Therefore, they suggested that human resource departments have to provide several short courses to help upgrade these skills.Kathleen (2005) found that employers were not satisfied with the application of technical graduate because they lack the abilities for non-technical skills through her research on developing employability skills of technical graduates in United States.
The study for the Higher Education Funding Council for England on the impact of employability skills teaching and learning on graduate labour market prospects by Cranmer (2006) states that there is a mismatch between the skills acquired at the university with the skills needed in employment.Answers from the Line managers interviewed revealed that graduates lack mostly are the employability skills.
Ministry of Higher Education of Malaysia (MOHE) put emphasis on employability skills of graduates through this statement "…to produce competent graduates to fulfill national and international manpower needs, the graduates must be employed in their relevant fields within six months of their graduation".Meanwhile results from 2008 tracer study done by MOHE (2009), out of 13,002 first degree technical graduates, 35 percent (4,616) were unemployed within that period.Meanwhile out of 5,515 diploma technical graduates, 25 percent (1,428) were unemployed within that period.
In order to ensure that graduates are able to survive in the arena of employment today they must have employability skills for continuous development of new skills and adaptation to changes (Australian Council for Educational Research, 2001;Kearns, 2001).(2006) states that to compete in the 21 century workforce employability skills is a necessity.Carnevale et.al (1991) identified six declining basic workplace skills which is learning to learn, communication skills, creative skills, self-esteem, interpersonal skills and leadership.This has become employers worries as it begin to affect profit margins The Secretary's Commission on Achieving Necessary Skills (SCANS, 1991a) were given the task to investigate whether high school students were capable of meeting the demands of the workplace.SCANS identified five competencies important for students to enter the workplace which are 1) identifying, organizing, planning, and allocating resources, 2) working with others, 3) acquiring and using information, 4) understanding complex interrelationships, and 5) working with a variety of technologies.SCANS also categories foundational skills into three group that is basic skills, thinking skills and interpersonal qualities.It is predicted that jobs in the future will need people that can apply knowledge into their work.These workers must have creative and problem-solving skills which employers can build on.Lovejoy (2000) states that in order to integrate employability standards into the curriculum, technical educators are required to use innovative teaching techniques.While designing the curriculum, planners should involve the industry partners so that they can incorporate real workplace procedures and systems in instructional strategies.This can then be incorporated in the teaching and learning process by using innovative teaching methods and implementing authentic assessment.Students will be able to apply skills in real-life situations.(Riebe et al., 2010).

Problem Statement
Based on the research done by Fitrisehara et al. (2009) they found that students have a moderate level of employability skills, information skills were found to be the lowest score and personal quality the highest score.They found that there were significant differences between employability skills and the students' field of studies.The findings also showed employability skills acquired from different field of studies varies significantly.The findings showed that students in the electronic field have the most employability skills compared to others.
Meanwhile the study on technical graduates done by Ahmad Zaini (2005) found that 80.000 were still unemployed, and these graduates are largely depend on academic qualifications to get a job but with less nontechnical skills or generic skills required of employers.Employers tend to find competent workers from abroad because of local graduates lack of employability.Data from the Economic Planning Unit (2006) stated that the unemployment rate rise from 3.1% (2000) to 3.5% (2010).
A study conducted to explore the importance of employability skills of engineering graduates through employers' perspective in various fields of engineering in Peninsular Malaysia shows that the employers put high level of interests in employability skills as must have skills from graduates.Meanwhile investigation on the undergraduates' core competencies ability to meet with the requirements set by the employers and to analyse the effectiveness of personal qualities and employability skills development in private university in Malaysia, shows that the undergraduates were all highly competent in possessing the said personal qualities and skills.However, such skills as critical analysis, planning, problem solving, oral communication, decision making, and negotiating report a slightly higher level of mismatch between employers' and undergraduates' perception on their importance and development in the University (Husain et al., 2010).
There will be a mismatch between what is acquired with what is needed in the industry if there is imbalances in the curriculum content (Mohd Yahya, 2003).There is no specific tool to assess the employability skills of technical students which want to pursue job in manufacturing industry.
Therefore this study aims to develop employability skills assessment tool is crucial.Higher education stakeholders in Malaysia have expressed growing concern about employability to be possessed or acquired by students to become more employable.In Malaysia, the importance of employability skills is beginning to be emphasized by the Ministry of Higher Education by giving instructions to higher education institutions to inculcate and evaluate their students' employability skill.Developing an employability skills assessment tool to measure and assess individual levels in Malaysia manufacturing industry is aim to help higher educational institutions prepare their students with sufficient and suitable employability skills before graduating.

Method
Research design of the study used descriptive quantitative approached and developmental research design.The scope of development in a research context is analysis and planning for development, evaluation and utilization.Quantitative data for this study were obtained through questionnaires.The items in the questionnaires has been adapted from Secretary's Commission on Achieving Necessary Skills or SCANS (1991b) and modified according to Malaysia's context.Seven constructs of employability skills are: 1) basic skills, 2) thinking skills, 3) resource management skills, 4) informational skills, 5) interpersonal skills, 6) system and technology skill, 7) personal quality skills.

Sampling
The sample consists of 107 employers from five categories in manufacturing industry labeled as: 1) electrical and electronic product (E&E), 2) metal-based products (MB), 3) machinery and equipment (ME), 4) transport equipment (TE), and 5) other kinds of product industry (OT) (MIDA, 2008).The objective of choosing employers from various categories is to observe how important employability skills are to these different kinds of employers.1 was, typically, Operational Manager (49.3 percent and a majority) Supervisor (39.3 percent) and Chief Executive (11.2 percent).The employers' experience distribution was: 33.7 percent 6-10 years, 22.4 percent 3-5 years, 21.5 percent more than 20 years, 19.6 percent around 11-15 years, and 2.8 percent less than two years.The instrument for survey was pilot test and the result in Table 2 shows the reliability (Cronbach Alpha) of seven items in the employability skills domain.The highest score are basic skills (0.93), followed by the system and technology skills (0.92), informational skills (0.91), thinking skills (0.89), resources skills (0.87), interpersonal skills (0.87), and personal quality (0.85).All the variables used in this study showed Cronbach Alpha values more than 0.6 indicating that the chosen item are consistent and reliable.

Quantitative Data Analysis
Descriptive and inferential statistics were used to analysed the data.Descriptive analysis involves frequencies, percentages, mean and standard deviation whereas inferential analysis involves t-Test and ANOVA.Frequencies and percentages are used to explain the demographic items of respondents whereas mean and standard deviation are used to analyze level of employability skill of respondents.T-test is conducted to identify the differences between employability skills with the demographic variables.ANOVA is used to examine whether there are any differences among the employability skills based on respondents' field of study.A significant level of 0.05 is used by the researcher.The interpretation of the mean score of the employability skills findings using the following indicators: 4.50 -5.00=Most Important, 3.50 -4.49= Important, 2.50 -3.49=Moderate, 1.50 -2.49=Low, and 1.00 -1.49=Very Low.

Employability Skills Development Process
An Assessment Tool Development Model is produced to gives a guideline and concept how to develop an assessment tool systematically.Employability Skills Assessment Tool Development Model as shown in Figure 1 is a concept developed by the researcher after undergoing the process developing the assessment tool.This method gives a guideline and concept to develop an Assessment Tool and how it is developed systematically.This method is not specially for developing Employability Skills Assessment Tool only, but can be used to develop other kinds of skill assessment.
This model suggested four phases to be considered when developing an assessment tool.Those phases were identifying the item of the assessment tool, determining the weight factor, determining individual score level, and verifying the assessment tool.

Figure 1. Employability Skill Assessment Tool Development Method
Analysis of the data using Statistical Package for Social Science (SPSS) was done to identify which employability skills to be chosen and which to eliminate according to their importance to the employers' needs in each type of manufacturing industry mentioned above.Thus, the employability skills which were chosen were ranked as skills item to be assessed.The method that was identified suitable to develop an assessment tool in assessing employability skill was the Kepner-Tregoe method (K-T Method) (Kepner and Tregoe, 1981).It is a method or analysis frequently used by employers in deciding the performance level of employees.It contains several steps in determining the most rational decision, which are: 1. Defining the set of criteria needed in decision making.The criteria are the objective to be achieved.

EMPLOYABILITY SKILLS ASSESSMENT TOOL DEVELOPMENT METHOD
2. Listing the criteria according to weight factor, from scale 10 (the most significant) to scale 1 (the least significant).
3. Assessing all options in decision making.
4. Giving scores to sub-criteria from scale 10 for highest skills and scale 1 for the lowest skills.
5. Repeating the score process for all the criteria.
6. Multiplying relative scores with weight factor for all criteria.
8. Comparing total scores for all options.
9. Choosing the highest score option.
In the first phase of developing the assessment tool, one has to identify aspect of the skills to be assessed in an individual.These aspects can be identified through various sources such as literature review, survey, assessment tool instrument, or panel of experts.Nonetheless, the item selected for the assessment Tool has to be tested for its reliability through certain processes such as statistical test, interviews, experts' consensus, and other suitable research method.
The second phase is determining the weight factor on items of the assessment tool which was selected.The weight factor value can be determined through two methods -the panel of experts and mean score.Before setting the weight factor value, the range and suitable weight factor has to be determined.The range in question is the mean score range of the item in the assessment tool.This range can be determined through a panel of experts or mathematical equation.
The third phase is to set the employability score for the individual.The score scale can be designed or adopted from any other related and relevant measurement assessment tools.The score should be scaled and graded.A score form should be used in order to assess the employability performance of individual assessed.
The fourth phase is to verify the assessment tool developed.Verification should focus on the assessment tool system efficiency.The assessment tool developed is tested by the employers and lecturers.Checklist and interviews of the employees and lecturers are conducted to see if the agreement coefficient can be accepted.

Results
Results will be discussed in two sections which are the important aspects of employability skills and followed by the development process of the assessment tool.

Important Aspects of Employability Skills
Table 3 shows the results of the important aspects of employability skills.Employers from all five categories of manufacturing industry agree on the important aspects of employability skill.The mean score for each employability skills showed little difference, and each of the means showed medium ranges of standard deviations, indicating a relatively small variability in the distribution.From the mean value, the skills were ranked according to most important to less important, and the weight factors were assigned as in Appendix A.

Setting Employability Skills Score Level
The score scale and achievement level is then built in the third phase.The score and level is used to assess the employability performance of individual evaluated as shown in Table 4.The score scale and achievement level, has been validated by experts to verify the scale and level.

Validation of Employability Skills Assessment Tool
Validation of the Employability Skills Assessment Tool as mentioned is the fourth phase in the assessment tool development process.Validation is done by 10 employers and 10 lecturers.11 aspects listed in the checklist to see the effectiveness of the tools are;1) Suitability, 2) Clarity, 3) Measurable, 4) Aspects compatible to industry, 5) Clarity of rubrics, 6) Tested result meet expectations, 7) Tool can measure employee and student performance, 8) Tool easy to use, 9) Tool can measure all categories of students and employees, 10) Rubrics and Scale helps evaluator and 11) Tool can be used in industry and institution.The agreement coefficient in Tables 5 and 6 is done through the Cohen Kappa statistical measure of inter-rater agreement.Since k value takes into account the agreement occurring by chance it is considered to be more robust measure.The result of coefficient 'k' from employers and lectures analysis as shown in Table 5 and 6 showed the value is from 0.72-0.91.According to Sim and Wright (2005), the interpretation of the coefficient 'k" values show that the employability skills assessment tool developed is in between moderate agreement (0.61-0.80) and strong agreement (0.81-1.00).

Discussion
The research findings indicated that the aspect of interpersonal skills such as leadership ability, teaches others and work with cultural diversity has the highest mean score.Many employers suggested that this would be an outcome where employees integrated aspects of the personal values and employability skills.Key leadership in the organization are encouraging co workers and team members to focus on work goals, motivating peers, taking responsibility for themselves and others, participating in and facilitating change and improvement.According to Alston et al. (2009), team work and dedication were thought by employers as extremely important followed by the ability to lead.According to Zirkle (1998), leadership skill were seen very important by most vocational instructors in order to be develop in schools as part of the preparation to enter the workplace.
The next most important aspect of employability skills to be looked at is the thinking skill, problem solving, creative and innovative thinking, knowing how to learn and reasoning has the highest mean score.The employers indicated problem solving skills were essential for both entry level and existing employees.Problem solving required by employers would be varying according to the level of complexity of the job and the types of problem employees would encounter.According to Emery (1999),an employee who has acquired problem solving skills will be able to make creative solution, are practical and independent.He added that reasoning skills is important because it can help employee to make quick and logic decision or making conclusion out of them.
According to Margison (1999) to a make a sound conclusion it is essential to have knowledge.
Result by employer's shows that employers placed great importance on communication skills (reading, speaking, writing and listening) with mean score over 4. 05-4.35.The skills ware also the most frequently mentioned by employers as being crucial across all employees and positions.Employers placed different emphasis on the different forms of communication.Sometimes verbal communication (speaking) was emphasized, at others written communication was emphasized.Having empathy and listening to customers was sometimes mentioned as the key aspect of communication skills.Many employers said that employees who could not read and write but also be able to explain complex in a manner easily understood and who could listen and consider the implication of what was being said to them (McLeish, 2002).
Aspect of resource skills showed that all aspects (manages time, manages money, manages materials and resources, manages human resource and manage risk) has high mean score from 4.21 to 4.46.The pressure on industry to grow and find new markets is leading to the identification of this area of skill as critical to employability.The industries recognize that they need to continually provide new services and products in order to extend current markets or secure new markets.The role of employees is developing these new service and products is seen crucial.The employers suggested that employees must no longer relied on managers to take initiatives in developing the business or identifying new approaches.The identification of initiatives and enterprise skills as critical employability skills reflects new perspective from employers.Students in the financial management and business studies need to be expose to business and industrial world in order to understand more (Godfrey, 1997).
For the aspect of informational skills, the sub-skills 'organizes and maintains information' showed an moderate mean score (mean = 3.73).The ability to organize and maintain information is essential in management work.Employers also stated that planning and organizing skills is important, employees need to be independent and good at time management and manage priorities in order to survive because they know when to work alone and when to ask for support.In short, employees need to be able to manage time, manage self and work alone, be resourceful, make decision because they might be required to manage projects from conception to completion (Fitrisehara et al., 2009).
For the interpersonal skills, working as a team is seen as a very important skill.Employers stress that every position in any size of industry require some kind of teamwork hence it is essential for employees to understand cultural, gender and age factors in ensuring effective teamwork.The skill to be able to transfer between individual work and team work is an advantage (McLeish, 2002).Bunn and Stewart (1998) reported that skills to use technologies are very important to develop basic skills.Employees in the production field need to be able to apply technology to perform tasks because contemporary industries frequently used the latest technologies in their work.De Leon and Borchers (1998) found that 80% of employers agreed that the requirement to apply technologies to execute duties is extremely high.Yahya Buntat (2004) in his research also supported this notion.
The aspect of 'adaptability and flexibility" has the moderate mean score overall (Mean=3.48)but employers feel that personal values such as self-esteem, sociability, self-management, responsibility, integrity, work safety, and conscientiousness is important no matter how good their academic qualification they have.These values are seen to contribute to harmony and productivity within workplace and to developing good relations with customers.They are also seen as important to leadership in the workplace.
For the developed employability skills assessment tool, comments from the employers which have tested the tools are positive and the value of agreement k, is considered high from 072-0.91.Individual interviews with employers and lecturers are also done as the research methodology for the tool validity.The following comments indicate employers and lecturers view about the employability skills assessment tools and its functions:  Useful for evaluating staffs and students on areas to be difficult to quantify.With this tool, evaluation of employability skills becomes more transparent and strategic.
 The tools also provide value to the academic fraternity where assessments done are in line with the industry requirement and expectation.To enhance the tool further it should incorporate the competency leveling used by large majority of industries. The tool is fast to provide information.
 Good system to evaluate manpower performance appraisal.
 Tool can be used to evaluate employability skills of students.
 Tool is good for lecturers to assess and improved student employability skills.
 It helps lecturers to identify aspects of employability skills and assess these skills.Self assessment can be done by students.
The developed Employability Skills Assessment Tool is an analysis instrument used in making decisions on students, graduates and employees.This developed assessment tool could be used to compare the decisions made on two or more individuals in order to select the best among them.This developed assessment tool could help simplify the assessment of individual employability skills as well as produce a firm and consistent assessment.The Employability Skills Assessment Tool Development Model can be used as a guide to develop any skills assessment tool.The Employability Skills Assessment Tool Development Model emphasizes that each of the employability skills item should not be seen as equal in importance but should be differentiated according to its importance in the line of work.

P1 = Rater 1
UTM = University Technology of Malaysia UPNM = National Defence University of Malaysia P2 = Rater 2 UiTM = University Technology of Mara, Malaysia UniKL = University of Kuala Lumpur UTeM = University Technical of Melaka, Malaysia As a result of the completion of the development phase, an Employability Skills Assessment Tool was developed as shown in Appendix A.

Table 1 .
Demography of employers

Table 4 .
Employability skills performance level