Validity of the Diabetes , Hypertension and Hyperlipidaemia ( DHL ) Knowledge Instrument among Medical Students of Karachi

Background: While there have been a number of studies on DM, hypertension and hyperlipidaemia, an instrument which assesses knowledge based on all three conditions has neither been established nor authorized in Pakistan. Hence, the focus of this study was to establish a pretested extensive questionnaire to evaluate medical students’ understanding of DM, hypertension, hyperlipidaemia and their medications for use. Methods: A pre-validated and pre-tested DHL instrument was employed on 250 students of Dow Medical and Sindh Medical College and on 45 physicians working in a leading teaching hospital of Karachi. The DHL knowledge instrument was then distributed a second time to the very same set of students, after a period of 2 months, at the end of the foundation module, once they had received some basic formal medical education including diabetes and CVS diseases. Results: The overall internal consistency for the DHL instrument failed to comply with the set standard of more than or equal to 0.7 as our results yielded Cronbach’s α of 0.6. Overall the average difficulty factor of 28 questions is 0.41, which highlighted that the instrument was moderately tough. The mean scores for all domains were substantially lower in the students section in comparison to that of the professional section, which had remarkable impact on the overall mean(SD) knowledge score (40.58 ± 14.63 vs. 63.49 ± 06.67 ; p value = 0.00). Conclusion: The instrument can be used to recognize people who require educational programs and keep an account of the changes with the passage of time as it could help in differentiating the knowledge levels among its participants based on their educational status.


Introduction
Diabetes mellitus (DM), a disease caused by the inadequate production of insulin in the body or the body being unable to properly use the insulin that is produced, thereby resulting in hyperglycaemia (Caitlin Hagan Medical Producer, 2012) and complications such as hypertension and hyperlipidaemia; is a major worldwide health-care burden.The WHO estimates that between 2000 and 2030, the world population will increase by 37 percent whereas the population of diabetics will increase by 114 percent.By 2030 India and China will continue being the countries with the greatest population of diabetics (79.4 million and 42.3 million) (Wild, Roglic, Green, Sicree, & King, 2004).According to IDF (International Diabetes Federation), Pakistan had around 6 million diabetics in 2003 and by 2025 the number of diabetics is expected to rise to well over 14.5 million.Currently, Pakistan has the seventh largest diabetes affected population in the world, and will take fourth place by the year 2025.Research has shown that by providing education regarding diabetes by healthcare professionals on medications and self-care practices, clinical results and the standard of life for patients have drastically improved (Colleran, Starr, & Burge, 2003).
However the spread of knowledge does not guarantee a change or betterment in results, therefore an assessment of diabetes-related knowledge is important to establish an educational program and gauge its effectiveness.In previous years, a great number of studies have been carried out using various instruments to assess the knowledge regarding diabetes amongst the population; the Michigan Diabetes Knowledge Tool (MDKT) (Fitzgerald et al., 1998), Diabetes Knowledge Questionnaire (DKQ) (Garcia, Villagomez, Brown, Kouzekanani, & Hanis, 2001), Ped Carb Quiz (PCQ) (Koontz et al., 2009), and Diabetic Numeracy Test (DNT) (Huizinga et al., 2008), all of which have been validated.Many of these devices were authorized in the United States, while some were approved in Australia (Garcia, Villagomez, Brown, Kouzekanani, & Hanis, 2001) or the United Kingdom (Koontz et al., 2009).Moreover, these devices were mainly constructed using the English Language (Fitzgerald et al., 1998) while others were translated into different languages, including Spanish (Garcia, Villagomez, Brown, Kouzekanani, & Hanis, 2001), in order to meet the needs of the local population.On the other hand, these instruments suffered from certain limitations as well.The MKDT determined general knowledge and use of insulin (Fitzgerald et al., 1998), DKQ evaluated the general understanding regarding diabetes (Garcia, Villagomez, Brown, Kouzekanani, & Hanis, 2001), the PCQ evaluated carbohydrate food recognition, counting and used it to calculate insulin dose (Koontz et al., 2009), while the DNT calculated numeracy skills for diabetes (Huizinga et al., 2008).The populations reviewed by these instruments also varied, from adults with type 1 or type 2 diabetes by the MDKT (Fitzgerald et al., 1998), to adults with type 2 diabetes only the DKQ, and children with type 1 diabetes using the PCQ or DNT (Koontz et al., 2009) (Huizinga et al., 2008).In addition there were a number of general complications with the tests: some were too long, while others were opinionated or not specific enough.
The knowledge of a medical student regarding diabetes and its complications, hypertension, hyperlipidaemia and their medications can be used to determine the effectiveness of educational programs.While there have been a number of studies on DM, hypertension and hyperlipidaemia, no instrument that assesses knowledge on all three conditions has been developed or validated, that too in Pakistan.Due to the extremely low literacy rate in Pakistan and other developing countries, the validity and veracity of the information collected through these instruments would remain doubtful.Hence, the goal of this study was to establish a pre-tested extensive questionnaire to evaluate students' understanding regarding DM, hypertension, hyperlipidaemia and their medications to be used by Pakistanis suffering from type 2 DM.

Formation of the (DHL) Knowledge Tool
We employed pre-designed and pre-tested DHL knowledge instrument of Pauline et al. which was drafted for the Cardiovascular Risk Factors Intervention Strategies (CORFIS) test.The goal of this original test was to evaluate and possibly establish a chronic disease management strategy for patients from Malaysia.The tool was then utilized to evaluate the levels of diabetes-associated understanding among medical students and medical doctors in Pakistan.The validity and legitimacy of the DHL instrument was authenticated by a pilot survey on a group of 30 medical students, before its implementation on final target population.The result of pilot survey was not included in the final study.This questionnaire was then distributed to 45 practicing community and hospital physicians, as well as 250 medical students.The DHL tool was established in the English Language as it could be comprehended by most individuals.The questionnaire comprised of twenty-eight questions was categorized into 5 sub-fields, with every question having the choice of correct, incorrect and don't know as an answer.The sub-fields are: diabetes (5 questions), hypertension (5 questions), hyperlipidaemia (5 questions), medications (8 questions) and general issues (5 questions).

Student Section
Students who were recently enrolled into Dow and Sindh medical college of Karachi, had received no formal knowledge regarding diabetes and could converse in English, were inducted into the survey.Students filled out the DHL instrument twice: first upon admission and then after a period of 2 months, which is before and after the foundation module of their medical curriculum.

Professional Section
In order to achieve a more impartial evaluation on the legitimacy of the DHL instrument, physicians of Civil Hospital Karachi (CHK) were included as the professional group in this multi-centre study.Physicians had a greater understanding of diabetes as compared to the students, therefore the instrument was tested on them once only.

Procedure
Students were included from two medical colleges in Karachi, Dow medical and Sindh medical college.The first part of the questionnaire was comprised of demographic variables like age, gender, BMI etc. and second part consisted of DHL instrument questions.The questionnaire was administered the first time before the start of the foundation module.Students completed the questionnaire themselves, within 10-20 minutes.The DHL instrument was then distributed a second time to the very same set of students, after a period of 2 months, at the end of the foundation module, once they had received some basic formal medical education including diabetes and CVS diseases.An informed consent was obtained and the purposed of the study was explained briefly to each student before administering the questionnaire.The study protocol was approved by Dow University of health sciences.

Statistical Analysis
Statistical Package for Social Sciences (SPSS) version 20 was used for data entry and analysis.The answer to every question was marked as either correct, incorrect or I don't know.Examination was carried out by marking 1 point for a right answer and 0 for a wrong or don't know answer.The marks were first calculated and then the total changed into a percentage.Further analysis of each sub-field score was then performed by adding the total number of correct answers and converting them into a percentage.0 indicated knowledge of the lowest level, whereas 100 indicated knowledge of the greatest level.

Analysis of the Individual (Dichotomous) Items
McNemar's test helped obtain the test-retest reliability of all the individual items.Difficulty factor is explained as the number of people opting a correct answer to the question and the total number of correct responses being divided by the sum total of responses, themselves.It is used to indicate how difficult a certain question is.The greater the difficulty factor, the simpler a question would be.A value greater than 0.75 is considered poor, since the question would be far too simple.Questions having values between 0.3 and 0.7 are considered to be the most efficient.The ideal level is considered to be 0.5 10 .The DHL instrument's internal consistency was evaluated with Cronbach's a coefficient.Cronbach's a value ≥ 0.70 is regarded as an excellent internal consistency.By removing an item if there is a remarkable increase in Cronbach's a, therefore omitting that question would raise the homogeneity of the scale 11 .Furthermore, the average knowledge score along with standard deviation was obtained for each field.This total highlights the degree of knowledge in each field.As the data collected was not typically distributed, a number of non-parametric tests were applied.Wilcoxon Signed-rank test differentiated between any remarkable difference present between the results of the test and the retest.Furthermore, Mann-Whitney U test analyzed whether or not there was a remarkable difference between student and professional sections.Lastly, spearman's rho obtained a correlation coefficient of the field scores from the students section at test-retest.

Results
A total of 250 students and 45 physicians were included in this cross sectional survey.The student participants comprised of 152 (60.8%) males and 98 (39.2%) females with the mean age of 21.75 ± 2.35.Demographic characteristics of student participants are illustrated in Table 1.

Psychometric Properties of the DHL Knowledge Instrument
Internal consistency for the overall DHL instrument failed to comply with the set standard of more than or equal to 0.7 as our results yielded Cronbach's α of 0.600, which indicated poor inter relatedness or heterogeneous constructs.Nonetheless, keeping 0.5 as threshold value of Cronbach's α, the results (Table 2) demonstrated that the DHL instrument's internal consistency stayed close to the value of Cronbach's α-0.600 with the elimination of any of the components.Hence, all 28 components were preserved for further analysis.Overall the average difficulty factor of 28 questions is 0.41, which is moderately acceptable with a mixture of higher than 0.7 (4 questions) and lower than 0.3 (14 questions).This highlighted that the knowledge instrument was moderately tough.
Moreover, test-retest reliability was assessed in 250 students after 8 week duration.From the 28 components, only four (components 5,9,15 and 27) were seen to be non-remarkable (p value &gt; 0.05) and indicated high correlation coefficient (0.16-0.31) (Table 2).In addition, there was no difference in the test-retest marks for fields such as Diabetes, General Issues along with total score (Table 3).However, domains like HTN, hyperlipidaemia and medications showed statistical difference (p value &lt; 0.05) between test-retest scores.This finding highlight that the pre-validated knowledge instrument achieved partial stability only.
Table 3 illustrates the baseline understanding marks for students and physicians section.The average marks for all fields were substantially lower in the students section in comparison to that of the professional section, which had remarkable impact on overall mean(SD) knowledge score (40.58 ± 14.63 vs. 63.49 ± 06.67 ; p value = 0.00).Low correlation is observed in test and retest scores across the domains.Demographic factors including sex, age, BMI, race, monthly income and exercise, were not connected with understanding scores.

Discussion
Our study shows that the professional group of physicians are far more educated regarding diabetes when compared with the group of medical students.Similarly, knowledge regarding diabetes related diseases amongst the physicians is also greater as compared to the students.The lack of knowledge of the risk factors of DM among medical students is alarming.Therefore, a remarkable spread of knowledge regarding the disease through various programs seems to be a crucial factor in many preventive health behaviours.
The knowledge instrument employed in our study operated suitably for only few of the psychometric components analysed.The overall Cronbach's was only 0.6, which was not up to the mark, as far as the standard is set for such an instrument.Strikingly, the level of difficulty for all of the components was considerably low (0.15-0.75) with an average of 0.41.All of the questions had a level of difficulty less than 0.75, which demonstrated that the DHL instrument was moderately tough to answer for students.Moreover, this was supported by the average baseline understanding marks for our instrument which was evaluated to be 40.58.Whereas, other studies (Fitzgerald et al., 1998;Huizinga et al., 2008) which validated their instruments based on diabetes-related understanding acquired greater marks which varied from 44.4-61.0%, the survey by Koontz et al in specific, gained a score of 87.0%.Students in the current study demonstrated a greater understanding of diabetes, hypertension and hyperlipidaemia (41.76, 49.52, and 45.28) and less knowledge on medications (30.06).This finding is not unusual, as not all the students would have prior knowledge of certain medications related to the various illnesses.On the other hand, these three diseases are inter-connected, the students need not only be aware of the cardiovascular risk factors related to diabetes, but even medications linked to them.
The dependability of the DHL instrument, simply outstanding when seen through the results of the test-retest.From 28 components, only 4 did not show a substantial difference in the results of the test-retest.This influenced the field scores of hypertension, hyperlipidaemia and medication, causing the former two scores to increase and the later to decrease.Component 5 (Average fasting blood sugar ranges between 4 and 6 mmol/L), component 9 (We can sense whether or not our blood pressure is high) and component 15 (Omega-3 supplements more effective against "bad" cholesterol compared to the medicines prescribed by doctors) showed a substantial improvement in the retest, demonstrating that students may have perused the information after the first test.On the other hand, component 27 (30 minutes of exercise every week decreases the risk of generating heart issues) was answered correctly at retest by fewer students.One plausible reason is, the students "guessed" the answer correctly in the first test, however upon the retest they could not recall their former answer and therefore "guessed" incorrectly.
The overall understanding marks of the DHL instrument were greater in the professional section (physicians) as compared to the student section (63.49% versus 40.58%) including all the domains.Health care professionals are supposed to have a greater degree of understanding of diabetes and its related concerns in comparison to the students.Demonstrating the DHL instrument, is a legitimate instrument which can be utilized to differentiate the degree of understanding of diabetes amongst the people with different educational status.
Although this study has supplied useful information regarding the state of awareness of DM and its complications amongst students and professionals, certain limitations must be acknowledged.As the study participants were medical students, they had a general understanding of the issue at hand, it is also important to note the sample size of the students, which was only two hundred and fifty.Therefore, it is recommended that similar studies should be carried out by targeting various other institutes, thereby increasing the sample size and targeting students from fields other than medicine.Accurate administration of our questionnaires (written in English) depended on the translation of the interviewer, which could have in some way introduced a translation bias, as not all the students were fluent in English.Further studies using the Urdu versions (local language of Pakistani students) of the DHL instrument are also needed to be able to be more representative of the knowledge amongst Pakistani students.Nevertheless, present findings have laid the groundwork for similar studies in other parts of the country.

Conclusion
The psychometric assessment indicated that DHL tool had a high difficulty factor and it indicated considerable potential to be used as a device to evaluate students' understanding regarding diabetes and its related risk factors in Pakistan.The DHL instrument can even be utilized to recognize individuals who require educational programs and keep an account of the changes with the passage of time as it could help differentiate the knowledge levels amongst its participants based upon on their educational status.

Table 4
depicts the collation of the psychometric properties of our DHL instrument with other authorized instruments which were used in different parts of the world to evaluate knowledge on diabetes.

Table 2 .
The psychometric properties of the DHL instrument

Table 3 .
Knowledge scores of the professional and students group at test and retest (by domain)

Table 4 .
Assesment of psychometric properties of the DHL instrument with other tools for determining knowledge