Mosquito-Borne Arboviruses in Brazil: Assessment of Apps Based on the Mobile Apps Rating Scale (MARS)

Background: In Brazil, the prevalence of arboviral diseases, such as dengue, Zika, and Chikungunya, transmitted mainly by mosquitos, has increased alarmingly. In recent years, numerous free mobile apps tackling this issue have become available for various purposes and users. Objectives: This study aimed to systematically survey and evaluate these apps using the Mobile App Rating Scale (MARS). Methods: The survey was performed on Google Play Store and sought to identify these apps adopting the descriptors “Chikungunya”, “Dengue” and “Zika”. The MARS scale was used by two researchers to evaluate the apps following their translation to Portuguese and subsequent validation. Student's T-test, Kappa statistics, and Cronbach's alpha coefficient were employed to evaluate the interobserver agreement and the reliability of the scale. Results: Most apps (20 out of 29 or ~70%) were created to disseminate basic information about arboviral diseases to the population or for entertainment. There was an agreement between the two researchers for all parameters of the MARS scale, except for the engagement (p=0.002). The Cronbach's alpha coefficient indicated good reliability. Conclusions: The use of the MARS scale has shown that most of the evaluated apps were developed to share information about arboviral diseases in an interactive way, but they do not necessarily have the purpose of influencing their users to change behaviours related to vector control or the prevention of arboviral diseases, which the authors feel would be a more appropriate aim for future app development.


Introduction
In Brazil, the prevalence of diseases caused by arboviruses and transmitted by mosquitoes, such as dengue, Zika and Chikungunya, has increased alarmingly in recent years (Leta et al., 2018). Until September 2020, a total of 928,282 cases of dengue, with an incidence of 441.7 per 100,000 inhabitants, were registered by the Ministry of Health (Brazil, 2020;. The traditional vector control programs, based on vertical methods to eliminate breeding sites with the use of insecticides, have failed due to the increase of resistant mosquito populations, the high level of adaptation of mosquitoes to urban environments and infrastructural problems in Brazilian cities (Lima-Camara, 2016;Donalisio et al., 2017;Lowe et al., 2018); (Donalisio et al., 2017).
One of the most effective control programs is community participation in monitoring mosquito breeding sites (Abel Mangueira et al., 2019;Costa et al., 2020;. However, the existing programs do not effectively involve the population in the practices of elimination of these sites, leaving the community in the position of spectators (Abel Mangueira et al., 2019;Costa et al., 2020;. Thus, health promotion and health education measures that transcend the dissemination of knowledge are necessary in order to boost the engagement of the population. Recently, several studies have made efforts to elucidate the elaboration of educational materials for health promotion based on models of behaviour change, as well as the importance of using these theories for the involvement of the communities in health actions and for the adoption of individual protection behaviour (World Health Organization, 2012). One of the strategies that have been used to evaluate behaviour change involving communities and populations is the use of mobile learning and software applications (apps) for mobile devices (World Health Organization, 2012;McKay et al., 2018;Abel Mangueira et al., 2019). Available anywhere and at any time, these devices are very promising for the development of apps based on behaviour change theories to promote health education (Masterson Creber et al., 2016;Taj et al., 2019;Yang et al., 2015).
The rapid proliferation of smartphone apps has made it difficult -for their users as well as for health professionals and researchers -to define which ones have greater potential and quality (Stoyanov et al., 2015;Cummings et al., 2013). In 2014, there were more than 100,000 apps falling into the health, fitness, or medical categories, which doubled the market size of two and a half years previously (Xu & Liu, 2015). However, there is still a lack of evidence and methodological strategies to effectively evaluate the quality of the apps for health promotion, since the use of star ratings on Web pages is very subjective and based on unknown sources.

The Mobile App Rating Scale (MARS)
In 2015, Stoyanov and colleagues (2015) proposed a scale for the classification and the evaluation of the quality of these apps -the Mobile App Rating Scale (MARS) (Stoyanov et al., 2015). This scale has four sections, one of which is specific and modifiable to suit the application that is to be evaluated (Stoyanov et al., 2015). The items of the MARS scale are scored using a five-point Likert scale (1 inadequate, 2 poor, 3 acceptable, 4 good, and 5 excellent). The first section provides descriptive information about the apps. The second evaluates the objective quality of the application and it is organised into 19 items divided into four categories: engagement, functionality, aesthetics, and information quality. The third section contains four items that evaluate the overall subjective satisfaction of the user. The fourth section is an application-specific subscale that evaluates the perceived effect on knowledge, attitudes, the user's intentions regarding behaviour change, the search for help to promote change, and the likelihood of changing the identified target behaviours. The reliability of this scale was also examined, showing good results (Stoyanov et al., 2015). Recently, the MARS has been translated, validated, and adapted to be used in different countries and to be effective for rating apps built around health issues (Stoyanov et al., 2015;Terhorst et al., 2020;Martin Payo et al., 2019;Domnich et al., 2016;Messner et al., 2020).

Objectives
This study aimed to use the Mobile App Rating Scale (MARS) to systematically survey and evaluate the apps related to the issue of arboviral diseases that are available for different purposes and users in Brazil. To achieve this goal, the MARS scale was translated into Portuguese and validated before being used to assess the quality of these mobile applications.

Methods
The analysis of the apps for mobile devices was performed using the search terms "Chikungunya", "Dengue" and "Zika", referring to three of the main arboviral diseases transmitted by the Aedes aegypti mosquito. Preliminary screening was done based on the name and description of the app on Google Play Store, excluding apps that had fewer than 500 downloads, were duplicates (found using different search terms) or paid apps as well as those whose names were not associated with the search terms. After submitting the apps to the exclusion criteria, they were reviewed by team members and then downloaded for evaluation ( Figure 1). The screening was based on the Android operating system and Google Play Store; since, in Brazil, more than 95% of the population uses the Android system. The screening was performed over the period of three months and a total of 29 Apps were selected to be evaluated.
After the screening phase, the MARS was translated from English into Portuguese and a pilot study was then performed to validate the tool. In this study, ten undergraduate biology students and four researchers (SS, RSM, VAA, IDF) performed the analysis of one of the apps identified in the screening phase using the translated scale. After addressing their critical comments regarding translation and other suggestions, the assessment was replicated with the final version of MARS. The Portuguese version of MARS can be obtained from the authors on request.
Before evaluating the 29 Apps, we followed the methodology suggested by Stoyanov and colleagues (2015). After watching training videos, three researchers used the Portuguese version of the scale to evaluate five apps not included in the final sample. Following this training phase, the 29 apps were independently analyzed by two of them (researchers A and B). The compiled data were analyzed with the R program. Numerical variables representing the scores were analyzed using the normality test (Shapiro-Wilk) and, considering that most of them had a normal distribution, the Student's T-test was performed to compare the means obtained by the two researchers (p>0.5).
The Kappa test was used to evaluate the interobserver agreement for binary categorical variables and for variables using the Likert scale. To interpret the results, we used the Landis and Koch (1977) table, which establishes values above 0.81 as almost perfect agreement; between 0.61 and 0.80 as strong agreement; from 0.41 to 0.60 as moderate agreement; from 0.21 to 0.40 as reasonable agreement; between 0 and 0.20, the agreement is weak and if it is zero, it is insignificant (20). To perform the comparative evaluation of the reliability of the MARS scale between the two observers, Cronbach's alpha coefficient was used. It varies from 0 to 1. Values above 0.7 indicate good reliability of the evaluated score questionnaire.  Table 1 presents a brief description of the 29 apps selected for analysis. It was verified that only four apps (14%) were developed by the state or federal government with the purpose of disseminating information to the population, facilitating the identification of mosquito breeding sites or individuals affected by arboviral diseases. One of these apps, "Zika Zero", was widely disseminated by media outlets, such as television and radio. Four other apps were designed by university researchers or non-governmental organizations to collect epidemiological data or to facilitate the georeferencing of cases of arboviral diseases or breeding grounds. Furthermore, the dissemination of updated scientific information was the focus of the application created by the World Health Organization (WHO).

Results
Most apps (20 out of 29 or ~70%) were created by individual or professional developers to disseminate basic information to the population or for entertainment; several "smash the button" games were developed. Interestingly, one of the apps offered a sound frequency that was supposed to be insect repellent. In all, 23 of the 29 apps had the general population as their target audience; four targeted health workers, one was for basic education teachers and the final one was aimed at children. Table 2 shows the scores obtained by the 29 apps, in each of the parameters of the MARS scale, comparatively analyzing the evaluations of researchers A and B. It was observed that eight of the 10 apps with the highest average score were well evaluated by both researchers, showing strong agreement. Although the scores attributed by the different evaluators were not identical, the ranking of the apps tended to be considerably similar. Using the MARS scale, the two apps with the highest scores were "Dengue", developed to assist health professionals in the management of epidemiological information; and the "SP X Dengue" app, an educational game developed by São Paulo state government. There was also an agreement between the evaluation of the researchers regarding the two apps with the lowest quality score (Mosquito Attack and Dengue x Chik x Zika).    Table 3 shows the results of the t-student test comparing the mean scores attributed by the two researchers for each of the MARS parameters. It was found that there was an agreement between the evaluation of the researchers for all of these parameters, except for the engagement (p=0.002). By using a Likert-type scale with five response options for each of the items assessed on the MARS scale, Kappa statistics indicated that nine parameters had no significant p-value. Table 4 shows the interobserver agreement values of the variables in which the Kappa p-value was significant (p<0.5). It was verified that the Kappa values varied from 0.20 (customization) to 0.92 (application focus); three of 13 parameters had an almost perfect or strong agreement (above 0.61); three varied from 0.41 to 0.60, showing a moderate agreement; six varied from 0.21 to 0.40, being classified as reasonable agreement; and just one in thirteen had a Kappa value of 0.20, showing a weak agreement.
Finally, the internal consistency of the MARS was estimated using Cronbach's alpha coefficient and the results showed good reliability. A small difference between the results of the two evaluators was observed: 0.93 and 0.77 for researchers A and B, respectively (Table 5).

Discussion
For the first time in Brazil, the MARS scale was translated and used to evaluate mobile apps. Most of the Brazilian apps were created by individual developers or companies, with little or no contribution from health professionals and organizations, and aimed to share information about arboviral diseases in an interactive way. However, they do not necessarily intend to influence their users to change behaviours related to vector control or the prevention of these diseases.
One of the issues identified in the present study was the screening process using search terms on the app store.
Using the three selected descriptors (Dengue, Zika, and Chikungunya), the survey led to a great number of apps that had no direct connection to these arboviral diseases. In Brazil, the term "zika or zica" is used in common parlance to refer to bad luck and these words were extensively employed in this sense by developers on the app store.
Another issue was the durability or lifetime of the apps on the Google Play Store. Due to the challenges mentioned above, the screening of the apps was a lengthy process. In addition, many months were dedicated to the translation and validation of the Portuguese version of the MARS Scale as well as to the training of the researchers. During this process, several apps were excluded from Google Play Store. Given that most of the apps were developed by individual initiatives for entertainment purposes, this possibly occurred due to their financial limitations (Xu & Liu, 2015).
Most of the analysed apps were developed for the dissemination of information or entertainment purposes. A few excellent apps, such as the WHO ZIKA APP, fulfil their aim and manage to deliver high-quality information to health professionals and citizens. Some other apps, despite succeeding in providing information in an interactive way, do not necessarily intend to influence their users to change behaviours related to vector control or the prevention of arboviral diseases, which the authors feel could have huge benefits, especially amongst the more popular apps. Where this has been a specific aim of an app (or a project), the public health benefits are significant (Cummings et al., 2013). Moreover, the content made available on the apps was often not developed by health workers or organizations, meaning that conceptual mistakes, misconceptions or misunderstandings are possible or, indeed, inevitable (Cummings et al., 2013).
In the literature, many studies have addressed the contribution of behaviour change theories to the development of digital health technologies (Hartin et al., 2016;Wang et al., 2019;Glanz & Bishop, 2010). These theories, for example, have been successfully used to develop apps focusing on physical activities and weight loss (Yang et al., 2015). Despite this, the Brazilian apps analyzed in the present work did not adopt these theories to promote behaviour change in order to reduce the incidence of arboviral diseases.
This study corroborates previous findings demonstrating that the MARS scale is a reliable tool to identify high-quality apps (Stoyanov et al., 2015;Terhorst et al., 2020;Martin Payo et al., 2019;Domnich et al., 2016;Messner et al., 2020). For example, the "RS Contra Aedes -Agents" was classified as one of the five apps with the highest scores in the engagement section, general average score and subjective quality average. This app was developed by the Federal University of Rio Grande do Sul (UFRGS) in partnership with the State Department of Health. Targeting community health workers, endemic disease control agents and military officers, this app aims to help control and monitor home inspections for the elimination of mosquito breeding sites. The application is equipped with a geo-referencing system that provides real-time information to the health workers who are performing inspections and allows them to create reports, and record the date of the visit, address, type of property and location, that can feed the database of the Department of Health. The app also provides information about the diseases transmitted by the Aedes aegypti mosquito and its prevention, also offering advice for pregnant women to prevent congenital Zika syndrome. These features combine to make this app an exemplar of what the authors would like to see developers aim for: informative and instructive, but also adding to knowledge and supporting behavioural change.