Creating Core Competencies and Workload-Based Key Outcome Indicators of University Lecturers’ Performance Assessment: Functional Analysis

This research aims to create and validate the quality of performance assessment using functional analysis. The researchers employed a design-based research method to create core competencies and their workload-based key outcome indicators as a preliminary study encompassing two phases, before formulating a standards-setting appraisal model to assess university lecturers in a public university, Thailand. The researchers began with documentary research to identify core competencies of university lecturers from three clusters of educational programs, namely science and technology, health science, and humanities and social sciences. An innovative prototype of university lecturers’ core competencies was developed based on the obtained results from the first phase. A total of five experts and 17 users participated to validate the quality of the innovative prototype. The preliminary results reveal that there are four core competencies of university lecturers, namely teaching, research, academic service, and preserving arts and culture. Moreover, there are 13 workload-based key outcome indicators and 27 elements that resulted from the four core competencies related to the specific research university in the Thai context. Moreover, the quantitative results of the content validity index from the rating scales of the five experts indicate that the conformity index is 0.78 or higher. However, the qualitative interview results regarding the 17 users from four focus groups imply that there is a gap regarding the accuracy of current performance appraisal between lecturers’ core competencies and their actual workload. Therefore, the dean should make the necessary adjustments based on the context.


Introduction
The role of the university lecturer has a great impact on knowledge and cognitive development for society and the nation (Blašková, Blaško, & Kucharþíková, 2014). Therefore, it is a highly demanding job that requires core competencies and continual enhancement of professional knowledge and social competencies. This enables university lecturers to conduct scientific research and transfer the scientific results to students for their future development (Blašková, Blaško, Jankalová, & Jankal, 2014). University lecturers' work performance can not only have a significant impact on higher education implementation but also support the dynamic and effectiveness of the education process (Akbar Ali & Si, 2015). The usefulness of performance assessment, in general, can be categorized as the main contributor to the quest for reward and publishment, as a standard to authorize the assessment, provide feedback to the university to serve as individual career development, determine the purpose of the training program, and support the detection of organizational problems (Akbar Ali & Si, 2015). define the specific characteristic of the key outcome indicators as capturing the essence of the desired result, specifically related to the achievement of university lecturers' performance assessment. Furthermore, the key outcome indicators must be measurable, considering the repeatability of assessment, the precision required for measurement, and the resources needed for measurement (Allen et al., 2012). Next, the achievable and attributable characteristics refer to the performance assessment system to identify what changes are anticipated as a result of the involvement and whether the results are realistic. In other words, attribution requires that changes in the targeted developmental issue can be linked to the involvement (Hocking et al., 2008). The key outcome indicators must be relevant and realistic to establish levels of performance that are likely to be achieved practically and, thus, reflect the expectations of stakeholders (Allen et al., 2012). Finally, the time-bound characteristic refers to the progress of work performance to be traced cost-effectively at the desired occurrence for a set period (Leagnavar et al., 2015).
The selected research university is a public university in Khon Kaen province, Thailand. Since it is an established university, the human resource department has a performance assessment system that consists of two main components: 70 percent of the performance assessment system is used for measuring university lecturers' work achievement, while the other 30 percent is used for measuring university lecturers' behavioral performance (Khon Kaen University, 2015). The core competencies have been identified in accordance with the guidelines provided by the Civil Service Commission and have been used by the research university since 2011 (Office of the Higher Education Commission, 2018). Five core competencies are assessed in the performance assessment system, namely (i) Service Mind, (ii) accumulation of their careers (Expertise), (iii) focus on achievement (Achievement Motivation), (iv) Teamwork, and (v) adherence to integrity and ethics (Integrity). As a result, the main aim of this research is to create core competencies and workload-based key outcome indicators for performance assessment of university lecturers in this public university using functional analysis. This is followed by examining the quality of the created core competencies and workload-based key outcome indicators for assessing the university lecturers' work performance.

Research Design
The researchers chose the documentary research design during the first phase so that they could use the official documents as sources of material (Ahmed, 2010) to identify core competency expectations from three different clusters of educational programs. The documentary research design was deemed suitable because it could be used to assess a set of documents for historical and social value to create a larger narrative through the investigation of multiple documents surrounding the university lecturers' work performance. Using this type of material in the research entails that the documents are recorded as secondary data sources because they contain material "not specifically gathered for the research question at hand" (Steward, 1984, p. 11).
The expert reviews research design was used in the second phase as a usability-inspection method. As a result, the five experts examined the developed innovative prototype of university lecturers' core competencies and workload-based key outcome indicators with the goal of identifying usability problems and strengths (Harley, 2018). The researchers emphasized the experts' past experience and knowledge of usability principles. Moreover, focus group interviews were conducted for four groups of real users to review a set of specifications or more abstract versions of the users might interface. The focus group interviews were performed via planned discussion with a small group of real users to obtain their considerations and ideas on the quality of the developed innovative prototype of university lecturers' core competencies and workload-based key outcome indicators. The focus group interviews were practicable for illuminating the variation of viewpoints held by members of the four groups of real users. Moreover, these focus group interviews were feasible for methodological triangulation with the five experts' evaluation.

Research Procedure and Research Participants
The research involved a preliminary study prior to formulation of a standards-setting appraisal model utilizing a design research method (Reeves, 2006;Vongvanich, 2020). The preliminary study, composed of two phases, was employed to determine the core competencies and workload-based key outcome indicators for assessing the university lecturers' work performance. In the first phase, researchers conducted documentary research to investigate the roles, duties, workloads, and working conditions of university lecturers from three different clusters of educational programs, namely science and technology, health science, and humanities and social sciences, of a public university in Thailand.
The obtained results from the first phase were used to design and develop an innovative prototype of university lecturers' core competencies in the second phase. There were two groups of participants involved in the second phase to validate the quality audit performance of the innovative prototype, to examine whether the created core competencies matched the workload-based key outcome indicators. The first group comprised five experts, namely three experts from the areas of teaching, research, and academic services in higher education, one expert specializing in educational measurement and evaluation, and one who is a key individual involved in assessing university lecturers' work performance. These five experts were required to rate the innovative prototype in terms of content validity.
The second group consisted of four focus groups as users of the innovative prototype. The researchers employed a purposive sampling technique to select the four focus groups. The first focus group consisted of three informants who are the faculty's performance appraisal practitioners. The second focus group consisted of nine university lecturers, three from each respective cluster, namely science and technology, health science, and humanities and social sciences. The third focus group was the dean or associate dean from each cluster, totaling three informants. The final focus group included the rector and vice-rector of the human resource division, a total of two informants. A total of 17 informants participated in user groups for four cycles of the interview, using the user experience research method.

Data Analysis
Sources of data from the documentary research and focus group interviews were analyzed using content analysis. Researchers coded or broke down the text into manageable code categories for analysis. Once the text was coded into categories, the codes could then be further categorized into themes to summarize data even further. The documentary data were analyzed using conceptual content analysis, whereby the concept of university lecturers' core competencies was chosen for examination of the occurrence of selected terms in the data. Terms can be indicated in the documents implicitly or explicitly. The researchers needed to decide the level of implication and base judgments on subjectivity for reliability and validity issues for implicit terms.
Content analysis was used to analyze the focus group interview data. The researchers determined the presence of certain words, themes, and concepts within the given qualitative data. This helped the researchers to quantify and analyze the presence, meanings, and relationships of such words, themes, and concepts. They then could make inferences about the messages within the texts of the four groups of real users.
Functional analysis was employed to validate the identified core competencies by the five experts. First, the five experts selected the main functions and objectives of work according to both the workload and positioning standards. They began by overviewing, drawing conclusions, and summarizing the core competencies along with their operational roles. This was followed by breaking down each core competency into its indicators and elements. All identified core competencies together with their workload-based key outcome indicators were examined to formulate an innovative prototype, the so-called core competencies measurement model.
The quality audit of the core competencies measurement model was validated using the item content validity index (I-CVI). A good I-CVI should be 0.78 or higher. The I-CVI was calculated using the formula below (Polit, Beck, & Hungler, 2006): Nc identifies a number of experts assessing items at a consistent level N identifies the total number of experts I-CVI identifies content validity index for each item Calculation of item-I-CVI, let experts consider the conformity assessment measure into four levels:

Results
The results of this research are presented according to the research aim stated previously. The preliminary results comprise the workload-based key outcome indicators and elements based on the conceptualization of university lecturers' core competencies. These results are followed by examination of the quality of the identified core competencies and their workload-based key outcome indicators, as well as related elements, to assess university lecturers' work performance.

Results of Documentary Analysis
The first phase of the documentary research results provides a list of core competencies that were hypothesized as the measurement for the work performance of university lecturers. The results reveal that university lecturers' core competencies comprise four categories, namely teaching, research, academic service, and preserving arts and culture. The comparative results of the three different clusters of educational programs, namely science and technology, health science, and humanities and social sciences, indicate that there are differences in core competencies, except for preserving arts and culture.
On the one hand, the teaching competency of university lecturers from the cluster of science and technology is focused on improving teaching documents. On the other hand, the teaching competency of university lecturers from the health science cluster emphasizes using teaching material with advanced technology to interact with their students. Moreover, the university lecturers also practice specific professional practices in their teaching. However, the teaching proficiency of university lecturers from the humanities and social sciences cluster reveals that they are more concerned about supervision of student performance and consultation of students' classroom research.
The majority of university lecturers from the science and technology cluster possess research skills as they are research project leaders. Moreover, they have published their research results in international database journals with the impact factor, which indexed at Quartile 1 to 2, and have been first authors or corresponding authors. The documentary results reveal that university lecturers from the health science cluster also possess research competency and publish their research results in international journals with the impact factor. Moreover, they utilize these research results to benefit communities and society as well. However, the results reveal that the research competency of university lecturers from the humanities and social sciences cluster is lacking compared to the other two clusters because these lecturers only publish in national journals or certain international journals recognized within their specific field of study.
The results reveal that the three clusters are performing in academic service competency differently. For example, the majority of university lecturers from the science and technology cluster are receiving scholarship from either external or international agencies to support them to become project leaders in providing academic services. Meanwhile, university lecturers from the health science cluster are mainly providing academic services that have a high impact on communities and society. Finally, university lecturers from the humanities and social sciences cluster boast educational innovation as their prior academic service to society.

University Lecturers' Core Competencies and Their Key Outcome Indicators
The researchers utilized Wyborn et al.'s (2018) suggestions to synthesize the documents related to core competencies of university lecturers to create an innovative prototype. This was followed by using the user experience research method for quality audit performance of the innovative prototype of university lecturers' core competencies. Finally, researchers employed functional analysis to determine the workload-based key outcome indicators and elements derived from the identified core competencies. The preliminary results reveal that there are four core competencies of university lecturers, namely teaching, research, academic service, and preserving arts and culture. Furthermore, there are 13 workload-based key outcome indicators and 27 elements that resulted from the four core competencies with regard to the specific research university in the Thai context. Table 1 details the core competencies, their workload-based key outcome indicators, and their elements, while Figure 1 demonstrates the distribution of core performance mode using the functional analysis method.

Quality Audit Performance Results Through User Experience Method
The researchers employed the user experience method to conduct four cycles of focus group interviews with four key groups of informants who were involved directly in assessing university lecturers' work performance. These four key groups consisted of practitioners, university lecturers, the dean or associate dean, and the rector or vice-rector of the human resource division of the research university. These persons are currently using the guidelines provided by the Office of the Civil Service Commission (Office of the Higher Education Commission, 2018). The researchers intended to obtain wide-ranging views from users' perspectives to determine the quality of the performance results of core competencies and workload-based key outcome indicators in terms of their appropriateness and consistency.
The researchers practiced triangulation of interview data from four perspectives to enhance the validity of the collected data (Gay, Mills, & Airasian, 2011). According to Gay et al., the compelling viewpoints of qualitative research remain in the triangulation of numerous methods, data collection, and data sources to obtain a more detailed illustration of the idea under research and also to enable researchers to cross-check information. Content analysis was utilized to analyze the obtained interview data by identifying, analyzing, and reporting the themes within the data (Braun & Clarke, 2006).
The following verbatim interview records from the group of practitioners are responsible for performance appraisal commented about the gap of the current performance appraisal to measure accurately between university lecturers' core competencies and their actual workload. Moreover, they suggested that the standard-setting appraisal system should be a reference only, the dean of each faculty has to make necessary adjustments based on their context.
The following verbatim interview records from the group of practitioners are responsible for performance appraisal, and mention the failure of the current performance appraisal to measure accurately the difference between university lecturers' core competencies and their actual workload. Moreover, they suggested that the standard-setting appraisal system should be a reference only and that the dean of each faculty should make necessary adjustments based on their context: "The indicators for assessing performance are quite abstract. The empirical evidence used to support the assessment was not clearly stated. As a result, the assessors used their discretion to assess, and allowed their subordinates (university lecturers) to have full scores for all categories, including core competencies and behavioral outcomes." "There should be a standard-setting performance appraisal system that is the central standard of the university, and each faculty can adjust using additional and appropriate details depending on the context of the faculty." The second group comprised the university lecturers who revealed that the empirical evidence to support the core competencies was not defined clearly as a standard. They complained that the level attained in the performance appraisal should indicate clearly how university lecturers can improve in future to obtain a higher level of performance assessment results. The verbatim interview records are presented according to the related issue as follows: "Empirical evidence requires such as performance reports to assess core competencies is not clearly defined as a standard." "The level of performance assessment results in each assessment cycle did not show clearly what kinds of improvements we had to make for the next round to have a higher level of performance assessment results." The third focus group comprised of deans or associate deans who have the role of assessors. They commented on the core competencies used to assess the university lecturers' work performance, stating that the broad characteristics, such as good service (Service Mind), do not make clear how they can be measured based on lecturers' workload. They suggested that the core competencies in the current work performance appraisal should be used as the central standard of the university and established based on the practical workload. Moreover, faculties should be allowed to further adjust in accordance with their context. The following verbatim interview records reflect the deans' or associate deans' views regarding the current work performance assessment system: "Core competencies that assess the performance of university lecturers in the present have broad characteristics such as good service (Service Mind), etc. It does not specify how it is measured by their workload." "The core competencies used in the university lecturers' work performance assessment should be established based on the practical workload. It should be the central standard of the university and the faculties can further adjust according to their context." The final focus group consisted of the rector and vice-rector of the human resource division. Their interview results indicated that the current performance appraisal should reflect the areas that need further development.
For example, if a university lecturer has achieved the highest rating of expectation, the assessment criteria should be adjusted accordingly. Furthermore, they recommended that all parties should be encouraged to be involved because mutual recognition based on actual practices, clear indicators, and criteria is important for every individual to accept and understand. Finally, they agreed that performance assessment results must reflect the strengths and their indicators that need to be developed individually. This is expected to substantially benefit future personnel management of the organization. These themes are derived from the following verbatim interview records: "Performance assessment itself should be an assessment to reflect the areas that need further development.
If the university lecturers being assessed have achieved the highest rating, expectations and assessment criteria should be adjusted accordingly." "Determining the core competencies used in the assessment system should emphasize the participation from all parties involved and mutual recognition based on actual practices. There are clear indicators and criteria that can make everyone accept and understand." "Performance assessment results must be able to reflect strengths and points that need to be developed individually. This is for the benefit of further use in personnel management of the organization."

Discussion
This is a preliminary study mainly aimed at creating and validating the university lecturers' core competencies and workload-based key outcome indicators before researchers develop a standards-setting appraisal measurement model for a public university in Thailand. Therefore, rating the performance assessment tool is a fundamental technique to ensure that the job analysis can support consistency for university lecturers' work performance. For this work, the major concern was to offer reliable and valid means of collecting data and to focus on the critical core competencies to create an innovative prototype. The results of this preliminary study have successfully approved the quality of this innovative prototype with validation from five experts and 17 users.
It can be concluded that such a robust appraisal measurement model can provide university lecturers with meaningful and quality feedback and generate consistent use of performance appraisal data for administrative decisions such as merit pay, promotion, and tenure (Akbar Ali & Si, 2015;Lohman, 2021). Since past researchers (Cordoso, Tavares, & Sin, 2015;Herdlein, Kukemelk, & Türk, 2008) have raised questions about the quality of performance appraisal practices and noted poor alignment with institutional missions, the researchers would like to recommend the ideas of Allen et al. (2012), Hocking et al. (2008), and Leagnavar et al. (2015 to confirm the key outcome indicators so that they are specific, measurable, achievable and attributable, relevant and realistic, and time-bound, to solve the problems of poor alignment. Finally, a key suggestion for improving university lecturers' work performance is to enhance their core competencies through professional training so that they can conduct better scientific research and transfer the scientific results to their students