Using “USE tool ”: Usability Evaluation Method for Quality Architecture in-Use

The main priority of Malaysian healthcare design quality is to organize an informational domain of a patient-oriented care design by patient experience to usable environment. The usability evaluation is an appropriate qualitative research design dealing with a process concerning the understanding of the user and context of use. This paper provides strategies for evaluating quality architecture in use from user experience and approaches for analyses of applicable qualitative data. Case studies have been conducted to explore the usability of three replacement hospitals in Peninsular Malaysia using “USE tool ” evaluation method introduced by Hansen, Blakstad, and Knudsen (2011). It is a five-stage evaluation process focusing on the following questions: for what, what, where and whom, why, and lastly, the final report as an action plan and input for the improvement of a building quality environment design in use. The process of data analysis is based on thematic analysis principles using NVivo 9. The findings indicate that (1) the quality of care is the positive users’ experience feedback on the usability of physical environment design that fulfils their needs and expectations, (2) there is a strong relationship between the usability physical environment criteria and overall patient satisfaction, and (3) the usability evaluation is useful for benchmarking and creating comparative databases, which optimally accommodates the needs of users and acts as a learning feature for improving the existing or future design.


Usability of Healthcare Design
Hospital is a complex design and its purpose is a value to design quality, which is to satisfy users on their positive experience by meeting their needs and expectations either by physical or psychological (Dilani, 2010;Lee, 2011;Bishop, 2008;Ulrich et al., 2004;Haron, Hamid & Talib, 2012). Moreover, its facility interrelates with functions that must accommodate the constant movement of people with physical ability and ease of use. Those factors are the main driver and reflection of effectiveness, efficiency of service delivery as well as contribute to patient health outcomes (Lee, 2011). Therefore, the planning and design of hospitals need to focus on creating spaces and environment for human-based facilities (Nawawi, 2006;Mohamad, 2010). However, in the local context, according to Aripin (2010) most of the surveys are related to the quality or value of healthcare design using Post Occupancy Evaluation (POE), which has been carried out not noticeably on the patients, staff and visitors but more on the building equipment and services. In addition, feedback from the interview with the medical planner from the pilot study shows that POE will be carried out after they get a complaint from a client (hospital) or end user (occupant) if the facilities do not function properly. In contrast, usability is an extension and improvement of POE but emphasizes on the user experience and perspective, and the main criteria in evaluating the ability of design measured based on their own experience (Alexander, 2008).

Usability Concept and Evaluation in Built Environment
The study of usability emerged from a diverse field and based on some views it was first developed in the 1950s in Human Computer Interaction and is widely known in relation to applications within User Centred Design (UCD), Usability Engineering (UE) and user experience (UX), and associated with the friendliness criteria (Fenker, 2008;Gulliksen, 2006). The usability research in built environment is associated with the International Council for Research and Innovation in Building and Construction (CIB) Task Group 51 "Usability of Buildings 2005", Workshop W111 -"Usability of Workplaces 2-2008" and "Usability of Workplaces 3-2010".
The CIB group is established to apply usability concepts and provide a better understanding of the user experience in buildings and workplaces. The term usability describes user experience of service or product, whether or not a product is fit for a specific purpose. It is adopted from the international standard on usability and a part of usability parameters '[…] effectiveness, efficiency and satisfaction with a specified set of users can achieve a specified set of tasks in a particular environment' (ISO,1998) and usability should be addressed to 'specified users to achieve specified goals' (Alexander, 2008).
Usability measures the quality in use and usability evaluation is for tracking the process of quality in use (Bevan, 1995;Voordt, 2005;), a process of understanding the interaction between facilities and its use, and the characteristics of that interaction (Mohamad, 2010). Hence, POE evaluation is still about the building rather than the user experience and primary process of the occupants, and the difference between POE and usability, which is the evaluation, depends on:  Context specificity (Lindahl et al., 2011)  Situated action (Fenker, 2008)  Cultural sensitivity (Lindahl & Granath, 2006) and habits (Sinkkonen, 2000) in (Alho & Nenonen, 2008).  Elements of user experience (Alexander, 2006)  Characteristics (Mantyla, 2001 in (Alho & Nenonen, 2008)  Serviceability (Hansen & Knudsen, 2006;Strawderman & Koubek, 2006).

Method and Reflection to the Research Objective
Having reviewed the literature on usability evaluation and the results of the pilot study, USEtool is a usability evaluation that determines the patients' experience and the quality involve high value from their experience and the context of use the environment. This tool was introduced by CIB W111; Research Report Usability of Workplaces Phase 3 in 2010 and this USEtool handbook was studied from the usability research projectmethods and tools from 2007 to 2009 (Hansen, Blakstad, & Knudsen, 2011). It presents a systematic approach of five-stage mapping process reflecting the objectives and the research questions of the study. It is a process description, detailing how building owners and facilities managers can gather user experience from existing buildings as a basis for improving them and focusing on the following questions: who, what, where and why (Kärnä, Nenonen, & Junnonen, 2010). This scenario is associated with the theory of the built environment, which tends to be oriented on the process of how it is created and supplied, and how it functions. This requires an attitude to architecture that recognizes the value of people and matter as the very reason for its existence (Kuhn, 1962;Lang, 1983) in Vischer, J. C. (2008). The understanding of the USEtool evaluation being applied to the main study as a systematic process of collecting, analysis, presenting and reporting the data is shown in Figure 1.

Stage 1: Defining the Evaluation (For What?)
The process of collecting data for this study used the usability evaluation, which is a tool for tracking quality architecture in use. The aim of this evaluation was to identify the patient lead's usability of healthcare physical environment design from their experience. It was associated with the cause and effect phenomenon between users (patients and visitors) and their context of use (the hospital environment) as shown in Figure 2.
The conceptual framework referred to the understanding of usability concept and the purpose of usability evaluation. The conceptual framework described the relationship between the usability evaluation and how the usability criteria were identified. It identified the usability criteria from the overlapping process between the patient's expectation versus the real situation (context to be used) and the situation of the environment, facilities or service that should function properly.
The evaluation and exploration of usability criteria and the value of design depended on the physical environment design, facilities acceptability, and user-friendliness. In addition, the usability criteria were also influenced by:  The users' situation -how the environment matched their characteristics, difficulties and family needs.
 The users' emotion during their experience finishing their task -how the environment made their journey more efficient and effective to use.  The users' expectations towards the environment before they entered the hospitals, either expectations for improvement from the previous experience or expectations of how the environment could make their experience comfortable, convenient, and efficient to use.

Stage 2: Mapping (What?)
The mapping process was based on the framework according to the findings of the pilot study and the understanding of usability concept, the relationship between usability, and attribute to quality architecture in use. Basically, the data collection used two tasks: semistructured interview (Task A) and walk-through journey experience (Task B) consisting of two components, face-to-face interview and activity observation. The interview questions and observation keys referred to nine aspects of the meaning and measurement of usability of building criteria by (Voordt, 2005;

Stage 3: Walk-Through Usability (Where and Whom)
In this study, qualitative case study research design was used to study the complex phenomena within their contexts of use. The main case study referred to three hospitals, and it was a continuous process from the pilot study but with improvement and appropriate method that have been clarified. This study used multiple case studies because if the study represented a single case study, it would not have been able to give instructions to design a variety of hospital sizes for users of different backgrounds. Moreover, the selection of multiple cases was to identify the trend, similarity and gap of usability criteria, and the physical environment domain from the different sizes and types of hospitals. The idea of multiple cases is, it can give the researcher predict the similar results across cases or predict the contrast results based on a theory (Yin, 2003).
Furthermore, Picker Institute Europe (2009) reported that qualitative research is suitable to be used when dealing with patients' needs because it is an appropriate method and more user-friendly without much effort to understand the questions and the respondents can express the answers in more detail, and can control the situation according to their suitability and comfort (Sheldon, Sinayuk, & Donovan, 2009). It is difficult to measure quantitatively environments that produce a positive experience with valuable, memorable and enjoyable experience (Alexander, 2010). The evaluation was based on the phenomena between the context of use (users) and its situation as intended to be used (environment) as well as its reflection of the research questions, which contributed to the qualitative research design:  How could the usability of physical environment design influence and produce quality architectural in use?
 Why the patients concerned about their journey were experience process and how did their expectations of the physical environment design reflect support to the way it was used by them?  What were the nature of the differences and relationship of patient experience and expectations in contributing to the physical environment design usability?
The results of the pilot study were summarized, mapped and formed into the main key questions for the main studies of walk-through journey experience using interview and direct observation. This combination method gave the respondents a chance to talk freely about their expectations during the trip and to gather in-depth information about the usability issues. However, there were a few factors that had to be considered during the implementation of this task. These were:  The focus of investigation explained to the respondents before the walk-through session included the purpose of task, the start and end of the task, and the estimated duration of the task.  The flow map explained the probability of stopping and waiting patients as a guide for the interviewer for the interview process. It was to clarify the scope and limitation of the area involved in order to know the patients and visitors' expectations ( Figure 3).
In every pit stop, the respondents' reaction would be observed and their opinion asked as the attribute and the domain of the ability of the environment. It was also to gather the answer on how the environment helped the respondents by identifying the quickest route to the desired location.  Using layout as a tool to support the identification of location and attribute of the usability issues highlighted by the respondents. The researcher took part by accompanying the respondents and thus the researcher became an active participant in the events being studied.  The unstructured observation being done, in which the end users did not realize that they were being observed and the evidence was natural and spontaneous. The direct reactions included patterns of informal interactions between users and their surrounding as an intermediate sense of what was seen, heard, felt, and expressed.  The observations were recorded using a camera and video without disturbing the privacy and territory of patients due to the ethical requirements. The pictures were presented and reviewed on why it happened and what should have been essential to ensure that the environment design was able to be used by the patients and visitors. In this session, field notes were also a major research instrument to support this participant observation session.  Taking notes -divided into three stages. First, the meaning of what was observed from the perspective of the participants, describing the close approximation of comments related directly to the observation activity, describing the possible gap or the impact on the respondents in that situation. Second, the characteristics of the users and the characteristics of the environmental situation, setting and scenarios. Third, taking note of the start, finish time, date and day. This process that combined text and photos from the various stops provided useful, comprehensive documentation that was easy to communicate. Examples of field notes can be seen in Figure 4.

The Scope of Study: Where
In this study, the walk-through session was limited to the area frequently used by the patients and visitors. According to Rahim and Samad (2010) in the study of audit accessibility of Malaysian public hospitals, the usability universal design in public space by Afacan and Erbug (2009), and the results of the pilot study, the frequently used areas were:  Hospital entrances and exits: Identifying and approaching the entrance and exit, and manoeuvring through them.  Area of obtaining service including the service desk, outpatient area lobby area, consultation, pharmacies, and specialist clinic waiting area.  Public amenities such as a food court, public telephones, restrooms, recreational area and seating units.
 Circulation and walkway environment such as ramps, elevators, escalators, hallways, and corridors.

Unit of Analysis: Sample
According to (Olsson et al., 2010), patients in a hospital are service receivers and children, parents, patients' relatives, family members and friends are indirect service receivers as shown in Figure 5. Hence, this study focused on the patients and visitors because they are the main and real users of hospitals (Leebov, 2003). This is www.ccsenet.org/jsd Journal of Sustainable Development Vol. 6, No. 12; in line with the purpose of the hospital, that to be at the heart of patients; it must be understood from the perspective of patients (Dilani, 2010;Joseph et al., 2009;Stern et al., 2003). According to Dumas and Redish (1993), reviews by design experts are not considered usability tests although such evaluations may be conducted as part of the overall development process. As Dumas and Redish (1993) pointed out, "If the participants are more experienced than the actual users, you may miss problems that will cause the product to fail in the marketplace. If the participants are less experienced than the actual users, you may be led to make changes that are not improvements for the real users"... p.23. However, the study did not focus specifically on the differences of gender, age, ethnic, socioeconomic and cultural backgrounds since healthcare given equal treatment to all patients irrespective of their race, gender, socioeconomic status and cultural backgrounds.

Analyze the Main Case Study Data: Application of Thematic Analysis
In this stage, the principles of inductive thematic analysis, which is a process of interpreting raw data analysis from interview transcripts and direct activity observation field notes, are for the development of the taxonomy, themes, and theory. The data was transferred to NVivo9 to facilitate the coding and to maximize the effectiveness and efficiency in sorting and merging the data according to themes. The identification of overarching themes based on the theoretical framework integrated themes from related previous research and results of the pilot study.
The cross-case analysis was used to identify the similarity of usability criteria, the pattern of patients and visitors' expectations, and the context of use experience from the three hospitals. This method has allowed the researcher to identify the gap between the users' expectation and needs (demands) and what the hospital offers (supply). This process was done manually with colour indicators aligning the usability parameters to six major domains. This process was developed from the concept mapping approaches by Jackson and Trochim (2002) and method on merging and sorting the theme (Patton, 2002;Lincoln & Guba, 1985;Miles & Huberman, 1994;Strauss & Corbin, 1990;Ryan & Bernard, 2003).
The domains constituted the overarching, interconnected yet independent structures that provided the framework encompassing patients and visitors' experience of the physical environment. The domains were relevant to all the respondents and participants' experience. However, each participant of the interaction are varies. The alignment with this domain was indicated by the evidence for a single dimension concerning the influence of a patient's diagnosis of their consciousness of the physical environment.

Stage 4: Why?: Validation the Main Study Finding (Qualitative Result) With Quantitative Approach
In this stage, each of the results of three cases was tabulated in order to identify the relationship between the issues. Then, the results were validated and triangulated within the main case study results and other multiple methods in order to support the main findings. Construct validity was attained by establishing a high significant level of correlation with other measures of the same constructs or concepts (Shi & Tang, 1997;Said, 2006). Correlating the data from different categories of respondent or task would show convergence of positive or negative responses of that situation and indicated high measurement validity (Patton, 2002). This cross-validation process is a validation strategy in qualitative research and known as methodological triangulation (Bishop, 2008;Said, 2006). This strategy can use multiple methods to study the same or single phenomenon in order to improve the results by capturing from different sources and to manage effects of missing data (Patton, 2002).
In this study, the validation process was a reflection of the reliability of research and reliability referred to the extent that consistent results were obtained when a particular measure was applied to the respondents (Shi & Tang, 1997). However, in this task participants were asked to respond to the full data set obtained from the cross-case analysis results using questionnaire (quantitative approach) to cross-check the mismatch within the main case study results (qualitative approach). All the questions, which used the Likert scale with five scales (strongly disagree to strongly agree) for participants to choose, were distributed via email. Participants were divided into two groups: (a) distribution among experts involved in hospital design and those involved in hospital service, and (b) distribution among the patients and visitors from the existing and other hospitals. In other words, the data were triangulated to reveal different aspects of empirical reality (Patton, 2002) of responses towards the hospitals' physical environment and its context of use, and the results of this task were verified to support the research findings.

Stage 5: Final Report
Include In this stage, the data from three sources, namely the main study results, the task results from expert review, and the patients and visitors' views were triangulated in order to interpret the resulting concept from the www.ccsenet.org/jsd Journal of Sustainable Development Vol. 6, No. 12; study of a communication gap between the medical planners, designers, facility management and other professionals with the end users (patient and visitors). The results were expected to:  Act as a guide for future hospital design, closer to a patients' needs with community involvement  Act as a development strategy to manage the facilities  Improve the user experience and to support the effectiveness and efficiency of the physical environment  Act as a systematic evaluation of the quality of healthcare design and architecture in use.

Discussions
Overall, the analysis of this study showed that respondents could focus and talk freely about the usability of the physical environment. However, it was influenced by how their experience meets their expectations before they come to the hospital. This is linked to the individual's interest in the physical environment for the capacity to support the things that they want to do, and for the friendliness context influencing such situation that;  Improvement from past experience if the experience gives negative feedback. The effectiveness and the efficiency of spatial designs offered in order to meet the patients and visitors or family member's expectations, needs and goals.  The facilities offered are to meet or suitable and able to be used for every characteristic and category of patients and visitors as well as their family members, which come from different age, physical difficulty and health condition.  Patient satisfaction also associated to the activities, atmosphere, environment and facilities offered during the waiting process besides. Besides that, patient satisfaction and usability criteria are also related to the fulfilment of the needs and expectations from the patient's family members and friends.
Accordingly, this study is devoted to facilitating more balanced relation between the supply and demand of design quality with nine criteria, and all the criteria are associated with satisfaction. The main study results show that satisfaction level is related to usability criteria and it supports the overall criteria. User satisfaction and usability criteria exist if the context of use experiences meet the users' expectations with effective and efficient physical environment design as it intended to be used and how the design benefits and supports patients and their family's needs. Moreover, Nielsen (1994) stated that satisfaction is an individual's usability dimension and according to Rasila, Peggie and Kerosuo (2010), besides the value of usability and it is also the main criteria influencing the usability perception, which contributes to other instances of usability criteria.
Usability evaluation is an iterative evaluation process combined with different methods to find the maximum number of usability problems (usability criteria) rather than relying on one method. It is a systematic method to facilitate the evaluation and data collection appropriate for the qualitative study. This study has provided a definition of usability concept in the healthcare environment that is applicable to an architectural design context. In addition, a sequential "USEtool" evaluation based on this definition has been demonstrated as an efficient method for evaluating the usability of a Malaysian hospital design.