Knowledge Management and Usability Model for Knowledge Management System

Many studies and works have been done to produce a Knowledge Management System (KMS) in which employees of any organization can access the organization’s sources of information and solutions. However, there is still no standard knowledge measurement and usability model that can assist KMS user to select or evaluate the appropriate KMS. The aim of this paper is to analyze how the ISO Consolidated Usability Model suggested by Abran, Khelifi, Suryn and Seffah can be used in measuring knowledge and evaluating usability for any Knowledge Management System. The methodology used is a user-satisfaction questionnaire developed based on the ISO Consolidated Usability Model


Introduction
Knowledge Management System (KMS) is becoming a trend nowadays enabling employees of any organization to access the organization's sources of information and solutions.For example, using a KMS, a programmer of an IT company could know the existing libraries that he/she could use in his/her current project.Sharing this information organization wide can lead to more effective software design and it could also lead to ideas for new or improved software features.
However, the lack of a usability measurement framework for KMS may drive people away from using it.It also impedes a systematic comparison among KMS providing a similar functionality.Nevertheless, this situation can be overcome if appropriate usability models are in place.These models will assist KMS administrators in determining which provider best fits the organization's needs.Furthermore, it is most important to notice that, usable system is a must to ensure satisfied and returning users.
In determining usability of a software system, one must have specific knowledge about the end-user of the software systems.In ISO 9126-1, the first part of ISO 9126, usability is defined as "set of attributes that bear on the effort needed for use, and on the individual assessment of such use, by stated or implied set of users" (Wikipedia, 2009).Usability here comprises of: • Learnability -learning effort for different users, i.e. novice, expert, casual etc. • Understandability -how system functions can be understood, relates to user • Operability -ability of the software to be easily operated by a given user in a given environment.
The usability of product depends on the nature of the user, the task and the environment.For the end-user, software usability means how fast and efficient he/she can complete the expected task.For managers, software usability is a criterion that must be considered in selecting a product.For software developers, software usability means issues related to design and documentation.
From KMS perspective and basing on ISO Consolidated Usability Model, usability refers to the capability of the KMS to be effective, efficient, satisfactory, learnable and secured under specified conditions.In particular, the usability model proposed in ISO Consolidated Usability model is enhanced.This model is validated using user satisfaction questionnaire given to Universiti Putra Malaysia KMS' users.We do not claim the model to be exhaustive, rather a starting point where new attributes and measures can be introduced as further experience is developed.
This paper is structured as follows: In Section 2 we give a brief overview of Knowledge measurement and usability, while in Section 3 the some usability models are reviewed.In section 4, we apply the ISO Consolidated Usability Model to KMS.Section 5 show the methodology used while in section 6 the results are discussed.The last section summarizes this paper and proposes future work

Knowledge Measurement
What is knowledge?According to (Trans4mind, 2009) , knowledge means: 1) Knowing that (facts and information) 2) Knowing how (the ability to do something) Thus, in an organization knowledge can refer to the information that an employee may have know and the ability of an employee to perform assigned tasks.It can also mean facts and information that an organization has.In (Lethbridge, 1994), knowledge is defined as "any form of information that one might be able to manipulate in one's brain … involving the categorization, definition and characterization of things and their relationships".Now, how to measure knowledge?In Computer Science and other fields, knowledge can be measured using metrics.In (Wikipedia(b), 2009) metric is defined as a "standard unit of measure, such as meter or gram, or more generally, part of a system of parameters, or systems of measurement, or a set of ways of quantitatively and periodically measuring, assessing, controlling or selecting a person, process, event, or institution, along with the procedures to carry out measurements and the procedures for the interpretation of the assessment in the light of previous or comparable assessments".There are two types of metrics; open-ended and closed-ended metrics (Lethbridge, 1994).A closed-ended metric refers to measurements that fall within a particular range and it can range from zero to one.An open-ended metric refers to measurements which one of the ends of its range is not absolutely fixed.
In Software Engineering, there are many metrics such as LOC (Lines of Code), Function points and COCOMO (Constructive Costs Modeling).In this paper, we will use metrics that are applicable to usability only.In ISO 9126 (international standard for the evaluation of software), usability is defined as the how easy and effective a software is.Usability here comprise of learnability, understandability and operability.
The next sections will describe in detail on usability and usability metrics.

Usability
In (Galin, 2004) usability is defined as "requirements deal with the scope of staff resources needed to train a new employee and to operate the software system".Boehm et. al. (1978) defines software usability as "the extent to which the product is convenient and practical to use".The measures of usability include measures of internal attributes and external attributes (Fenton, et. al., 1998).The internal attributes are: 1) well-structured manuals 2) good use of menus and graphics 3) informative error messages 4) help functions

5) consistent interfaces
External attributes of usability: 1) entry level: in terms of experience with similar classes of applications 2) learnability: speed of learning, hours of training required before independent use 3) handling ability: speed of working when trained, errors made when working at normal speed The usability of the system plays a big role, not only in customer satisfaction but also in terms of additional functionality and life-cycle costs.There are many works that have been done to produce good usability models.We will further describe these works in section 3 consolidated usability model.
Usability metrics are the measurement used in measuring the usability of software, in other words, quantitative methods of measuring usability.Metrics can help managers in tracking design process or in deciding to purchase products.Examples of metrics; Percentage of Task accomplished, Time to achieve one task, Time to learn, Time spent on errors and Percentage of Task achieved per unit of time.

ISO Consolidated Usability Model
From our study, we found out that there is little attention in the usability during the development of any type of software.
Software developers mainly stress more on the features of software and getting it done while neglecting the important aspect of a software which is usability.Therefore, in this paper, we aim to analyze how the ISO Consolidated Usability Model suggested in (Abran et. al., 2003) can be used for guiding any organization or institution in developing usability measurement for their KMS.

Consolidating the ISO Usability Models
According to Abran et. al., 2003, software usability can be characterized based on the target audiences since each audience has a specific expectation of the software usability.For the end user, good usability software will be software that allows him to achieve the expected task more efficiently.For the manager, usability will help him in deciding whether to select a product or not.
The authors used ISO 9241 and ISO 9126 as the basis of their studies.They argued that a clearly defined model must be established so that it can clarify usability definition and objectives of software through specified measurements.The paper argued that ISO 9126 is unclear in detailing the measures for each level and also lack of guidance in assessing results of measurement.The paper also stated that ISO 9241 is still not adequate as it does not tackle the learnability characteristics.Hence, it suggested a consolidated model, which was based from ISO 9241-11 but added with two additional characteristics: learnability and security.
The consolidated model is shown in figure 1 below.
Our paper will be based on this ISO Consolidated Usability Model.In order to ensure a usable KMS one must study design of user interfaces for the system and also the security part.We evaluate this model using Universiti Putra Malaysia KMS, the SSM.The SSM is the KMS system to evaluates the performance of the staffs.This is detailed out in methodology section.
We believe this model is also a good candidate for future work of KMS usability model.

Applying the ISO Consolidated Usability Model to KMS
In this paper we have selected ISO Consolidated Usability model as the basis for KMS usability model.In Table 1, the definition of the different attributes along with their relevance is shown.
Each of the sub characteristics above will determine whether a given KMS is usable or not.Thus, attributes need to be defined for each sub characteristics so that measurement can be done.
Each of the sub characteristics above will determine whether a given KMS is usable or not.Thus, attributes need to be defined for each sub characteristics so that measurement can be done.
Based from ISO Consolidated Usability Model, effectiveness is measured by: 1) Percentage of tasks accomplished 2) Ratio on failure of handling 3) Percentage of tasks achieved per unit of time The following table 2 shows what we can measure to determine the effectiveness of a KMS.
Efficiency was measured by: 1) Repetitions' number of failed command The following table 3 shows what we can measure to determine the efficiency of a KMS.
Satisfaction was measured by: 1) Percentage of users' favorable and unfavorable comments 2) Number of times that user expresses his frustration 3) Rating scale for users' satisfaction with functions and characteristics The following table 4 shows what we can measure to determine the user satisfaction of a KMS.
Security was measured by: 1) Access audibility 2) Access controllability 3) Data corruption prevention 4) Data encryption Security was more on whether participants were asked for password during the first time using the system and also when the system fails.The following table 5 shows what we can measure to determine the security of a KMS.
For learnability sub characteristic, we consider the capability of the software to enable the user to learn how the software achieves its aim.Learnability was measured by time to learn as shown in table 6.

The enhanced Usability Model
The following figure 2 shows the ISO Consolidated Usability Model together with the attributes that we suggested in previous section.
Based on the measurement given for each sub characteristics, usability of a KMS can be calculated accordingly.However, in this paper, we only validated the ISO Consolidated Usability Model using a simplified questionnaire as stated in the Methodology and Results and Discussion sections.

Methodology
In order to evaluate the suggested model in section 4, we developed a questionnaire based on the measurements laid out in the model and also sample taken from Software Usability Measurement Inventory (SUMI).
The usability questionnaire supports user subjective satisfaction with the KMS effectiveness, efficiency, security and learnability as well as with the attitude the system induces in users during its usage.Participants indicate level of their agreement with a questionnaire statement on a three-point Likert scale.

Results and Discussions
For this study, we have chosen a group of Universiti Putra Malaysia KMS's users, mainly lecturers from Fakulti Sains Komputer & Teknologi Maklumat, Universiti Putra Malaysia.
These participants have used the KMS at least once.Therefore, we assumed that they are quite familiar with the flow of the KMS.

Procedures
We use quantitative data in order to meet our objective by giving out structured questionnaires.In preparing the questionnaires, we employed the following steps: • Categorized the questions by employing the characteristics in the ISO Consolidated Usability Model which are Effectiveness, Efficiency, Satisfaction, Security and Learnability.

•
Chose type of data to be collected.Each question should have 3 likert scale or at least a Yes and No answer for a quick and easy data analysis, as well as open question to obtain detailed information.
The questionnaires consist of both closed questions and open-ended questions.For the closed questions, Likert Scale was used in which participant was asked to indicate his or her degree of agreement with the statement.For open-ended questions, participant was asked on the main reason of using the system.
What we have done during the questionnaire sessions are:

•
Explain to the participants the purpose of the sessions • Give participants around 10 to 15 minutes to fill up the questionnaires • Ask for any additional suggestion from the participants

Measures
The validity of the model was determined from participants' answers to the questionnaire; all questions used a three-point Likert scale.The five aspects that are taken into considerations: effectiveness, efficiency, satisfaction, security and learnability.
In this study, for effectiveness, we only look at in terms of participants' ability to complete given task.For efficiency we look at participants' satisfaction with the system efficiency in terms of: repeating certain processes, error recovery and also usefulness of online help.For satisfaction, we look at whether the participants agree that system has all the expected functionalities.Security was more on whether participants were asked for password during the first time using the system and also when the system fails.For learnability, we look at the number of participants who agreed that the system is easy to learn.

Accomplished results
We received 14 respondents for this study.Table 6 shows the results of this study.
Here we can see 71.4% participants aware that they must use the system to fill up their activities for assessment of their work performances.
From table 7, we can see that most participants agree that the system is not effective.
Based on the responds, we can see from table 8 that most participants agree that the system is not efficient as: • They need to repeat previous tasks if system stops suddenly The online information is not very helpful • It takes a long time for the system to recover from error From table 9, most of the participants do not satisfy with the system.Based on our short interviews with them, we gather that: • System requires many steps • Users need to scan their documents before entering data into the system.If not the system will not allow the users to save the transactions From table 10 more than 60% respondents agree that the system is not easy to learn.
From table 11, the system is good in terms of security.
Looking at the accomplished results we can see that the Universiti Putra Malaysia KMS still need more enhancements in order to ensure its usability.We believe using this enhanced Usability Model, any KMS usability also can be determined.

Conclusions and Future Work
In this paper, we apply the characteristics suggested in ISO Consolidated Usability Model to see how this model can be used in KMS.This model is validated using user satisfaction questionnaire given to Universiti Putra Malaysia KMS' users.From the results, we can see that this model can be used to develop a usability measurement framework for any KMS.We do not claim the model to be exhaustive, rather a starting point where new attributes and measures can be introduced as further experience is developed.The KMS users must be able to complete their task at a given time The KMS must be easy to learn to achieve one task 6) Number of good and bad characteristics recalled by users 7) Number of available commands not called upon

Table 2 .
Measurement of Effectiveness KMS

Table 3 .
Measurement of Efficiency KMS

Table 7 .
The main reasons for using KMS Figure 1.ISO Consolidated Usability Model Figure 2. ISO Consolidated Usability Model