Construction and Validation of a Tool for the Assessment of the Oral Health in Primary Health Care Through the Perspective of Patients

Objective: To construct and validate a questionnaire to evaluate the quality of oral health services in primary health care, from patients. Methods: Initially a theoretical model of evaluation of Primary Health Care was elaborated, based on the evaluation of primary care and integrality in primary care. This model served as the basis for the script of a focus group with patients, aiming to verify the attributes perceived as important for such evaluation. The focus group results substantiated the first version of the questionnaire. Content validation was performed through a committee of experts (five teachers/researchers) and face validation in two pre-tests (37 patients each pre-test). For construct validation, factor analysis was performed and reliability (Kappa coefficient) and internal consistency (Cronbach's alpha) were verified. Results: Thirty questions were considered for exploratory factor analysis. The anti-image matrix of covariances showed the need to exclude fourteen questions (values <0.5). After this initial analysis, 16 questions remained in the questionnaire. The KMO test, considering the 16 questions, presented a value of 0.84. Cronbach's alpha was 0.919. The final version contains 16 questions divided into two dimensions: my health unit and the care in my health unit. Conclusions: The questionnaire allows a strategy that easily evaluates oral health services in primary care, based on the perception of patients.


Introduction
The evaluation of systems and programs has occupied an increasing space in the process of consolidating services in the health area. Assessments are seen as an important part of an effort to redirect professional and institutional practices. They can produce data necessary for accountability to society on the use of committed resources, as well as on the progress of the programs currently in force, empowering management with appropriate tools in the various processes decision-making (Felisberto, 2006).
In the international context, a considerable volume of studies, aimed to evaluate Primary Health Care-PHC, have been observed (Friedberg, Hussey, & Schneider, 2010;Balarajan, Selvaraj, & Subramanian, 2011;Schoen et al., 2009;Nicholson, 2012;Hogg & Dyke, 2011;Doran & Roland , 2010). Patients' view as a protagonist in this evaluation process is essential and necessary since it represents a judgment about the characteristics and quality of the services offered, in the perception of those who receive the service (Souza et al., 2008). However, patients are not yet configured as fundamental subjects in the process of evaluation of health services (Esperidião & Trad, 2005), which is a mistake. Patients' views are seen as an effort in which individual and citizenship rights are reaffirmed, contributing to empowerment, in a political and social dimension (Kleba & Wendhausen, 2010).
There are questionnaires capable of evaluating the performance of PHC services from the perception of patients, highlighting the Primary Care Assessment Tool-PCATool (Shi, Starfield, & Xu, 2011), the European General Practice Evaluation Instrument (EUROPEP) (Grol & Wensing, 2000); the Interpersonal Processes of Care -18-item version -IPC-II (Stewart et al., 2007) and the Veterans Affairs National Outpatient Customer Satisfaction Survey -VANOCSS (Borowsky et al., 2002). However, these instruments do not satisfactorily evaluate specific oral health, presenting few questions that directly address this area. What happens in practice is a shortage of instruments focused on this type of evaluation (Lima, Cabral, & Vasconcelos, 2010).
The objective of this work is to construct and validate a questionnaire for the evaluation of oral health in PHC from the perception of patients, aiming to fill this existing gap and ultimately contribute to the improvement of practices and management in this field.

Methods
The study was developed in Belo Horizonte, capital of the state of Minas Gerais, Brazil. With approximately 2,500,000 inhabitants, the municipality is divided into nine health districts and has an organized municipal health system, within the PHC, with the inclusion of 76% of the population in the Unified Health System-SUS, the Brazilian health policy.

Participants
The participants of this research were adult patients (more than 18 years) of both genders and with a minimum experience of 3(three) years of effective use of the oral health service provided by the health system (that is, to seek the service for treatments and not only aiming at emergency care). University professors participated in content validation.

Theoretical Framework for the Preparation of the Questionnaire
For the construction of the questionnaire, named Oral Health Assessment in Primary Care-User (OHAP-user), initially, a theoretical model was constructed, contemplating PHC attributes aimed at evaluating primary health care (access, coordination of services, integrality) (Starfield, 2002) and to evaluate integrality in primary health care (reception, bond-accountability, and quality of health care) (Silva Júnior et al., 2008). A representation of this theoretical model is set out in Figure 1.

Focus Group
In the next stage, a focus group was performed. The focus group was the data collection strategy chosen to allow, through an explicit interaction between participants, under the conduction and facilitation of the researcher, to explore people's views and experiences about different aspects of daily life. It is possible to have a shared perception on the theme studied, covering more broadly the object of study, as well to know what people think and why they think about a particular topic, in this case, health services (Kitzinger, 1995).
The inclusion criteria aimed to ensure the heterogeneity and multivocality of the participants, according to the different districts of the municipality. Thus, it was requested to dentists based in different basic units, covering the nine health districts, indicate patients who had a potential interest in participating in the research. No calibration of dentists was required.
Subsequently, the researcher responsible for data collection contacted the patients indicated by dentists, when clarifications were made about the research (the objective, importance of their participation, anonymity), as well as the invitation to participate in the focus group. After attempts to reconcile the availability of those who had an interest in participating, there were six participants left, who effectively attended the activity. The researcher who conducted data collection had previous experience in conducting qualitative research and had no link with the others involved (patients and dentists of the basic units).
Data collection took place in 2015, in a room specially provided by the Research Institution, which provided the necessary adaptation (comfort, silence, tranquility) for the realization of the focus group. The expenses for the movement of patients to this site were assumed by the researchers. Initially, the participants were again clarified about the research objectives, oriented about their participation in the group, how to avoid overlapping speech and to always speak toward the recorder (in the center of the circle). All 6 subjects remained until the group was completed. There were no interruptions during data collection.
For the realization of the focus group, a semi-structured script was used, elaborated from the theoretical model. After the first conversation with the objective of clarifying and relaxing the group, the motivating question was asked: "How is your arrival, your entry into the dental service at your health unit? Then the script was explored, in continuity with the established conversation.
The activity was considered finished when all the topics defined in the script were covered. The recorder was only turned off after everyone had left the site, allowing observations to be recorded after the formal termination of the group. The content obtained was transcribed by the same researcher who conducted the focus group. The data obtained were processed using the content analysis method. The first stage, called pre-analysis, involved the first contacts with the documents to be analyzed and the formal preparation of the material. Subsequently, a fluctuating, exhaustive and repeated reading of the texts was made, which allowed the transformation of raw data into themes and subsequently the obtainment of categories. Next, inferences were made from the data already treated, qualitatively analyzing the themes and categories that constituted the patients' perception of oral health care in PHC. This process was carried out by two researchers, simultaneously and independently and after discussion between the researchers, the categories were consensed (Minayo, 2014). No calibration of the researchers was required since this stage was qualitative.

Development of the Questionnaire
The categories obtained in the focus group allowed the questions that make up the questionnaire to be developed. Initially, the same configuration used in the PCATool was chosen, containing an introduction with the characterization of the interviewee, including identification, the basic unit that attends, schooling and family income.
Next, the questions were elaborated, and seven existing questions in the PCATool were used because they corresponded to categories pointed out by patients in the focus group. Another 36 questions were developed by two researchers who later discussed the most appropriate writing, the presence of positive and negative questions, technical terms, until consensus was reached, making up a total of 43 questions.
We chose to use a Likert scale, with three response options: yes/always (value three), sometimes (value two), not/never (value one), plus the option do not know (value zero).

Validation of the Questionnaire
Initially, content validation was performed. This stage was attended by a committee of experts, composed of five teachers/researchers, with experience in primary care and/or the construction of questionnaires. The experts gave their opinion on the pertinence and clarity of the questions included, issuing suggestions that contributed to the process, suggesting a more appropriate wording or exclusion of questions. After this phase, the questionnaire had 42 questions and was submitted to a first pre-test with patients, aiming at face validation.
At this stage, the application took place in two basic units located in two health districts of the city, including 37 patients, present at the reception of health units. They were invited to participate, informed in relation to the research objectives and guided by the researcher responsible for how to fill out the questionnaire. The answers were analyzed by the researchers and made alterations based on the difficulties pointed out by the patients at the time of completion. Two questions were excluded and one was dismembered for being ambiguous. Thus, the questionnaire now has 41 questions.
A second pretest was performed, with 37 new patients, selected with the same criteria as the first pre-test, in two other health districts and, at this time, the questionnaire was considered adequate, since difficulties expressed by patients in the first pre-test were not manifested again.
For the evaluation of reproducibility, 45 patients were selected, five from each health district of Belo Horizonte. The questionnaires were applied and reapplied, with an interval of 14 days, a time considered appropriate for this type of activity (Streiner & Norman, 2006) and evaluated by the weighted Kappa coefficient.
To evaluate the internal consistency and validation of the construct, a sample of 164 patients was considered. This number was defined based on the ratio of 4 individuals per question (4x41), considered adequate for factor analysis (Bartlett, Kotrlik, & Higgins, 2001). To compensate for possible losses, the sample was increased by 10%, totaling 180 patients, divided equally into the nine health districts (20 patients per health district). From the list of health units that had dental services, a simple drawing of one health unit per district was made. The questionnaires were applied in the health units themselves, where patients were approached at the reception and invited to participate.
Exploratory factor analysis was used to analyze construct validity. Kaiser-Meyer-Olkin (KMO) and Bartlett's sphericity tests were performed to verify the adequacy of the data for factor analysis. The method used to extract the factors was the latent root and the orthogonal rotation used was varimax, and the factors with self-values >1 were chosen. For the composition of the factor, the questions that presented load (loading) above 0.40 were considered. This technique allows identifying as few factors (or dimensions/constructs) that best explain the correlations between the indicators' questions. The internal consistency was assessed by Cronbach's alpha (Cronbach α), which can vary from 0.00 to 1.00, considering that the higher the coefficient the more homogeneous the instrument is. The behavior of each item in the questionnaire was also analyzed, calculating the item-total correlation and corrected item-total coefficients (correlation coefficient of the individual item with the total score of the scale). In this stage, questions whose corrected item-total correlation was less than 0.20 were excluded. The SPSS 17.0 program was used for statistical analysis.

Results
The mean age among patients was 40.43 (SD = 12.59) (18 to 68 years), 78.9% were female, 35.5% attended incomplete elementary school and 33.7% reported monthly family income between 2 and 3 minimum wages (600 to 750 US$, approximately). As for occupation, the majority (53.6%) reported working on elementary activities, such as housework, kitchen helper or street vendors.
The focus group allowed us to verify that, for patients, the most representative categories for assessing dentistry in PHC are: the health unit (specifically from access, reception, service provision, the physical structure and supplies, the referral and counter-referral system); the reception and its relationship with the creation of the bond (being welcomed and recognized); service with a strong emphasis on the humanized relationship between professional and patient and on teamwork and; the critical citizen view of public management.
A total of 45 pairs of answers (test/retest) were considered in the reproducibility analysis for the 41 questions analyzed. The kappa weighted coefficient in the first analysis was considered insufficient (Kappa = 0.19-0.75). Two questions were excluded, due to very low reproducibility (Bartlett, Kotrlik, & Higgins, 2001). As the questionnaire had three answer options (yes/always, sometimes no/never), it was decided to aggregate the positive response options (yes/always and sometimes) and redo the calculations. Once this has been done, the kappa values weighted for each item ranged from 0.27 to 1.00, with only two questions presenting Kappa values<0.55.
One hundred and eighty patients answered the questionnaire in the phase of evaluation of internal consistency and construct validity. Fourteen questionnaires were discarded (filling errors) remaining for the analysis, 166 questionnaires. Descriptive analysis, maintaining the original scale, showed 11 questions with a high percentage of answers do not know (above 80%). These questions were excluded from the questionnaire.
After these correct answers, 30 questions were considered for exploratory factor analysis. The correlation matrix showed no multicollinearity between the variables (r < 0.726). The anti-image matrix of covariances showed the need to exclude fourteen questions (values < 0.5). After this initial analysis, 16 questions remained in the questionnaire. The KMO test, considering the 16 questions, presented a value of 0.84. The matrix of the components showed that two factors were extracted, which explained 84.0% of the variance of the data. Table 1 shows the factor loads, which represent the correlation of the item with the factor. Table 2 presents the factorial loads of each question that composes the questionnaire, and it can also be observed to which dimension each question is linked. Table 3 presents the questions that made up each dimension. The questionnaire validation process, therefore, allowed it to present, in the end, 16 questions distributed in two dimensions. The analysis also yielded very good internal consistency results for complete scale (α Cronbach = 0.9193). A good internal consistency was also shown for the 2 domains: My health unit showed a Cronbach of 0.8923, the service at my health unit of 0.8324. The dentist who attends me at the clinic has a good relationship with me. 0.3131 0.6483 The dentist who attends me at the clinic is concerned about my health. 0.3645 0.6461 I am confident in the work that the dentist who attends me at the clinic does. 0.5536 0.0692 The work of the dentist who attends me at the clinic is of good quality. 0.7436 0.3232 My oral health needs are solved in the same quality as the treatment given by a private dentist.

0.1918
In general, the post where I am served is clean and organized. 0.3786

0.6117
The equipment used is good and works well. 0.2180 0.7282 The health clinic I go to has the materials that the dentist who serves me needs. 0.2439

0.7424
The treatment with the dentist who attends me at the clinic is fast.  Table 3. Dimensions obtained after factor analysis and respective questions, in the construct validation of the OHAP-user questionnaire, 2015

My health unit
It is easy to get dental treatment at the clinic where I am served.
It is easy to get emergency dental care at the clinic where I am served.
Talking to the dentist who attends me whenever I need it is easy.
The dentist who attends me at the clinic does even more than I need to.
I recommend the health clinic where I have dental treatment.
I understand well what the dentist at the clinic explains to me.
I am confident in the work that the dentist who attends me at the clinic does.
The work of the dentist who attends me at the clinic is of good quality.
My oral health needs are solved in the same quality as the treatment given by a private dentist.
I manage to solve my mouth problems at the health center.

The service at my health unit
The dentist who attends me at the clinic has a good relationship with me.
The dentist who attends me at the clinic is concerned about my health.
In general, the post where I am served is clean and organized.
The equipment used is good and works well.
The health clinic I go to has the materials that the dentist who serves me needs.
The treatment with the dentist who attends me at the clinic is fast.

Discussion
Questionnaires produce important data for evaluation processes and their development and use should be surrounded by great care. These concerns should be even greater in relation to research that seeks to evaluate a particular social phenomenon, such as public policies, in the case of this questionnaire developed.
A self-applicable version was chosen, since questionnaires sent by mail or filled in interviews, whether validated, have less accuracy (Rosen & Olsen, 2006). In addition, there are the daily difficulties faced in the daily life of services, which require agile mechanisms for the evaluation of the actions developed. In this sense, the elaboration of a questionnaire with only sixteen questions, divided into two dimensions, contributes to this agility. It is added that the questionnaire developed was completed by the study participants with minimal intervention by the researcher, an advantage for its purpose.
The use of the focus group from the theoretical framework chosen (Starfield, 2002;Silva Junior et al., 2008) proved to be an important tool in identifying patients' perceptions, enabling the categories considered important by them in the evaluation of health services to be used (Nassar-McMillan & Borders, 2002). This fact is important, since the elaboration of the new questionnaire was not based only on what advocates the literature for evaluation in the area, but especially by the vision of those who effectively use the service.
The results of the test-retest allowed the reproducibility of the questionnaire to be verified. Although the interval was Kappa 0.27-1.0, only two questions showed agreement <0.55: "It is easy to get dental treatment at the health unit where I am attended"; and "the treatment with the dentist who attends me at the health unit is fast". The two dimensions obtained presented sufficient validity and reliability for its application in other contexts. All 16 questions have a factor load within the established parameters. Cronbach's alpha values were extremely good and, although there is an understanding that values above 0.9 for these tests can be a sign of redundancy of questions (Bartlett, Kotrlik, & Higgins, 2001), it is clearly perceived that they evaluate different aspects expected in daily life in health services.
The choice of the two-dimensional model resulted in an explanation of variance of approximately 84%, a value greater than that observed in other questionnaire validation processes for PHC evaluation, including PCATool (Almeida & Macinko, 2006;Harzhein, 2006). A cumulative variance close to 84% of an event as complex as the user-service interface indicates OHAP-user as a reliable instrument for assessing oral health services in PHC.
The questions of the final version of the questionnaire address all attributes initially defined for evaluation of integrality in PHC (Silva Júnior et al., 2008), but do not address the attributes integrality and coordination (Starfield, 2002). It is recognized that the validation of the questionnaire resulted in the exclusion of a considerable number of questions (n=25) from the original version, which included the other attributes that made up the theoretical framework. But although not all the dimensions pointed out by patients remained in the final format, it was observed the maintenance of what is indeed important for them: they want to have access to services when necessary, be welcomed by professionals and have their oral health demands resolved with quality, both from a technical point of view and of the relationship between professional and patient. That is, they want that in addition to the existence of services, in a given place and with appropriate times, there is the possibility of effective use in them with quality, which is supported by other studies (Souza & Roncalli, 2007).
It is important to clarify that the integrality attribute helps compose the theoretical framework, but does not represent it in its entirety. It is stated that integrality is related, in summary, to the complete variety of procedures aimed at the needs of the patient, which should be available and provided when necessary for problems that occur most frequently (Starfield, 2002). Given the polysemy of the term integrality it would probably be impossible for the questions of the questionnaire to address all their meanings, but it is possible to observe, among the questions remain, the presence of concerns related to practices professionals, who are not limited to the clinical aspect, or access, aspects that are linked to the concept of integrality (Silva, 2014) and that contribute to the expanded clinic (Fonseca, Nehmy, & Mota, 2015).
The absence of issues related to coordination, already mentioned, can be supplied by maintaining the question "the dentist who attends me at the health unit does even more than I need". This absence can express the patients' understanding that their needs are met, not requiring referrals to other levels of the system. It can also mean that in practice there is no reference or, what patients actually expect is what is done in PHC. This fact, however, must be seen with care, since many of the oral health demands presented by patients in the basic units are not solved in these units, requiring referral to specialized services and access to these remains a critical node for the reality of oral health in Brazil, for example (Sanchez & Ciconelli, 2012).
Regarding the coordination attribute, it appears that in another study, aimed at the validation of the children's version of PCATool for Brazil (Harzhein, 2006), there was also an exclusion of issues that addressed coordination in services, which may represent a challenge to be overcome for future questionnaires that want to evaluate PHC and that have this attribute in their theoretical basis.
As limits of this work, it is pointed out that not all patients invited to the focus group wanted or could participate generated a small universe of participating patients, which may have omitted perceptions and unreported experiences.
The OHAP-user, therefore, presents itself as a possibility, given the scarcity of instruments for its purpose. It concludes by stating that the construction and validation of this questionnaire enabled an assessment strategy that is easy to perform for oral health services in PHC, based on a theoretical framework that contemplates dimensions aimed at this level of care, through the view of adult patients. The results obtained may sensitize managers to qualify health services and professional practices and, ultimately, contribute to the construction of a public health system at the height of their existence, which is the patients themselves.