276
Views
0
CrossRef citations to date
0
Altmetric
Research Article

WhatsApp-based intervention in urban Colombia to support the prevention of arboviral diseases: a feasibility study

ORCID Icon, , , , , & show all

ABSTRACT

Arboviral diseases remain a significant health concern worldwide, with over half the world’s population at risk for dengue alone. Without a vaccine or targeted treatment, the most effective strategy of prevention is vector management with community involvement. mHealth interventions, like WhatsApp, offer promising results for engaging communities and promoting healthier behaviors. This study explores the feasibility of integrating WhatsApp in vector control activities to improve arbovirus prevention in Colombia. A mixed-methods approach was employed to assess the WhatsApp-based intervention. WhatsApp messages were sent to 45 community women for 5 weeks to increase their knowledge and practices about dengue, Zika, and chikungunya. Pre-and-post surveys and focus group discussions were conducted in community settings to measure the feasibility and acceptability of this intervention. Chat reviews were done to assess the usability of users. A total of 1566 messages were exchanged in 45 WhatsApp chats. High acceptance and good usability (82% of users used the app for replying) were reported in this study. WhatsApp messages were perceived as short, clear, and enjoyable. Users liked the frequency, and design of messages. Pre- and post-surveys demonstrated improvements in the knowledge and practices of arboviral diseases. The intention to apply this knowledge in practice was reflected in a significant improvement, particularly in cleaning the laundry tank once a week (pre 62.1% to post 89.6%, p < 0.008). This study suggests that using WhatsApp as an additional tool could be a feasible, acceptable, and affordable strategy for improving the adoption of better practices in the prevention of arboviral diseases.

Introduction

Arboviruses are a diverse group of insect transmitted viruses that cause human diseases such as dengue, Zika, and chikungunya (DZC) which are all transmitted by Aedes aegypti and Aedes albopictus and are widely distributed in tropical and subtropical regions around the world [Citation1]. These three arboviral diseases are devastating with over half of the world’s population at risk. Dengue is one of the most rapidly increasing vector-borne diseases, and results in 100 million cases reported annually [Citation2]. Chikungunya and Zika have also caused large epidemics in the Americas and elsewhere that rapidly spread through a large, susceptible population [Citation3]. Vector management is currently the primary available prevention strategy as the first dengue vaccine still has limitation in its use and several others are under development [Citation4]. The most common vector control method in many countries is the application of the larvicide temephos to domestic water containers or space spraying with an insecticide for adult mosquitoes, followed or combined with biological control. However, the limited success of these methods can be attributed to increasing insecticide resistance [Citation5,Citation6], as well as to cultural and social factors related to living conditions and to behaviors and practices of the population that favor the proliferation of Aedes spp. breeding [Citation7].

Initiatives such as integrated vector management (IVM) have been promoted to tackle these issues through a functional integration model to reduce vector densities and ultimately disease incidence [Citation8]. In line with this approach, there have been programs that involve community mobilization and behavioral change for improved dengue prevention and control. However, the difficulty of these programs lies in finding ways to achieve the active and sustainable participation of local people. Well-constructed social strategies such as Communication for Behavioral Impact (COMBI [Citation9]) that incorporate information and communication technologies (ICTs) could play a role in increasing community engagement of potential risks and encouraging collective action but have not led to a sustainable vector control.

During the last decade, mobile-based interventions have received considerable attention to improve population access to health information and care [Citation10,Citation11]. This approach articulated with health education can be beneficial for improving health outcomes [Citation12,Citation13]. Research in Peru and Nepal has demonstrated that delivering health messages through Short Message Services (SMS) can promote positive changes in human behavior, improving dengue practices and consequently decrease vector densities in households [Citation14,Citation15]. The use of mobile devices in the health field is referred to as mobile health (mHealth) [Citation16]. Since the arrival of advanced mobile devices called ‘smartphones’, there has been a special focus on integrating or creating mobile applications (apps) for a wide range of health-related needs [Citation17], including arboviral diseases. Novel mobile applications have been developed for different purposes in the prevention and control of DZC. To date, these technologies are mainly aimed at the collection and reporting of cases and symptoms, monitoring outbreaks, tracking mosquitoes, and mapping breeding sites and hotspots [Citation18]. However, only few studies have explored the use of mobile phones in health education and promoting behavior change.

WhatsApp Messenger (WhatsApp Inc.) is one of the most popular mobile applications worldwide, with approximately 2 billion active users [Citation19]. Although its primary purpose is social networking, the popularity of WhatsApp has meant that it has been widely used as a channel for sharing news and information [Citation20]. This feature has been leveraged for health systems and programs for improving communication between patients and professionals or health authorities and communities, favoring the spread of health-related information [Citation21], creating reliable communication channels and promoting healthier behavior in different target populations [Citation22,Citation23]. Despite these benefits, no research has explored the use of the WhatsApp application in the prevention of dengue, Zika, and chikungunya yet.

In Colombia, there is increasing interest to implement technological solutions in vector control operations such as development of software for epidemiological analysis, mosquito tracking [Citation24,Citation25] and mobile applications for data collection [Citation26]. However, these initiatives are only focused on vector surveillance rather than improved education or community engagement. In 2019, a national survey by DANE (National Statistics Department) reported a massive increase in ownership of mobile phones. Approximately 95% of households have at least one mobile phone and around 80% of mobile phone users have a smartphone [Citation27]. Additionally, a recent digital report showed that a high proportion of Colombian users (95.6%) use their phone to access internet and 41% check their health symptoms online, which is higher than the worldwide average (25%) [Citation28]. WhatsApp is the most downloaded messaging app among mobile users in Colombia [Citation29]. Moreover, a recent study shows that instant messaging and social networks are the communication methods preferred by Colombians [Citation30].

Researchers from Freiburg University in Germany (Center for Medicine and Society, ZMG) and from the State Health Institute of Norte de Santander (IDS) in Colombia tested a novel vector control method, ‘insecticidal coating for water tanks’, as an alternative to controlling Aedes spp. mosquitoes in Cucuta and its two adjacent municipalities in Colombia. These areas share the main land border crossing with Venezuela, alongside certain unauthorized points of entry. Over the past seven years, forced displacement and migration waves have led to over 2.8 million migrants and refugees fleeing from Venezuela to Colombia [Citation31], with around 337,000 of them staying in the State of North Santander [Citation32]. These populations are also at risk and could benefit from the insecticidal coating. The coating is applied in water tanks for laundry which are the main mosquito breeding places in the city. Safety assessments and long-term effectiveness of the method were proved in scientific papers [Citation33–35]. This vector control method aims at involving local communities as much as possible as active partners who ensure the acceptance of the measures, support the promotion and participate as community volunteers in the painting of water containers in their neighborhoods. In the communication between vector control staff and community, digital devices such as smartphones could have a major role. Additionally, a sub-survey was conducted in 2000 households in Villa del Rosario and Los Patios as part of the baseline study to assess the transmission risk of arboviral diseases and people’s willingness to apply insecticidal coating [Citation36]. It was found that 91.7% of the population had smartphones with WhatsApp, and 81.1% of individuals were interested in receiving preventive messages on their mobile phones. Given these facts and the possibility to introduce WhatsApp as part of vector prevention efforts, this study aimed to test the feasibility of a digital (mHealth) intervention to provide health education and social engagement for community women living in the metropolitan area of Cucuta, Colombia. The intervention focused on the prevention of dengue, Zika and chikungunya considering the following areas: knowledge of arboviral diseases, preventive practices, promotion of novel vector control tools and other relevant health information.

Methods

The study aimed to explore whether a WhatsApp-based intervention for community women can improve the knowledge and practices related to arboviral disease prevention. Further, the study aimed to validate and assess the acceptability of using WhatsApp messages as a service for health promotion targeting communities affected by dengue, Zika and chikungunya.

Setting

This study was part of a large-scale project on the effectiveness of tank-coating for reducing dengue incidence in pre-selected neighborhoods with the highest rates of DZC in two urban municipalities (Villa del Rosario and Los Patios) within the metropolitan area of Cucuta, the capital of Norte de Santander, Colombia [Citation36]. Villa del Rosario is located at an elevation of 440 m and covers an area of 228 km2, whereas Los Patios is 410 m high and has a spread of 131 km2. Both areas are close to the Venezuelan border and heavily affected by migration. The registered population is 92,661 inhabitants in Villa del Rosario and 78,409 in Los Patios. The average annual temperature of 27°C provides ideal conditions for the breeding and spread of Aedes spp. mosquitoes, the vectors of DZC [Citation37].

In relation to the latest distribution of dengue reported by the National Health Institute, the incidence rate in Cucuta stood at 229.3 per 100,000 individuals in 2021, which was higher than the national rate of 165.0 per 100,000 [Citation38]. Villa del Rosario and Los Patios had dengue incidence rates of 196.3 and 197.5, respectively. However, these figures were estimated during the COVID-19 crisis. Prior to the pandemic, the average annual dengue incidence rate in Cucuta was alarmingly high at 400 cases per 100,000 between 2009 and 2019 [Citation38]. Recently, a large study conducted in Cucuta, Villa del Rosario, and Los Patios estimated a high vector infestation of 25.1% in the overall house index, which was above the 5% assumed by WHO, indicating a high risk of epidemic transmission [Citation36]. Traditionally, these communities have used laundry tanks, known as ‘lavaderos’, for washing clothes and cleaning houses. This type of container was reported as the most productive breeding site for Aedes spp. mosquitoes, as they are frequently uncovered and situated outside of the house [Citation36].

Study design

The current study used an explanatory sequential mixed method design following the guidelines by WHO [Citation39] to evaluate a WhatsApp-based intervention as part of the above-mentioned implementation trial for dengue vector control. A pre-and-post design without a control group was conducted to measure the change in knowledge and practices of community women during August – October 2019. After the follow up survey, two focus group discussions were conducted to validate the structure and content of messages and assess the perceptions of the WhatsApp intervention. Furthermore, the usability was also measured through chat reviews. Recommendations were provided for the adoption of the WhatsApp message intervention. During COVID-19, all implementation trial activities had to be stopped. Nevertheless, communication with community members was maintained. After this period, formal monitoring of the mHealth intervention restarted and compliance was followed informally until July 2023. Results and analysis of the first phase before the interruption are included in the present study.

Study participants and recruitment

Participants were community women aged 18 years and above, from areas of Villa del Rosario and Los Patios. The involvement of women can be effective in vector control programs at the community level [Citation40]. Women are usually more active in reduction of mosquito breeding sites because they are often responsible for household water management and family care, particularly in low-and-middle-income countries [Citation41,Citation42]. Engaging a group of women in the identification and elimination of the most productive containers has shown to be successful in reducing mosquito abundance [Citation43,Citation44]. Therefore, this study contacted women associations with National Colombian programs such as Families in action and Family, Woman and Childhood. These associations identified and contacted women leaders to be recruited in this study. Women were eligible for this feasibility study if they had (i) a smartphone with good internet connection, (ii) experience using WhatsApp, and (iii) lived in the study areas. Eligible women were invited to a meeting to explain further details of the study. After providing their verbal and written informed consent, the participants were selected and registered in the WhatsApp application. The selection process of participants was developed in four steps (see ). We ended up recruiting 50 participants for sending WhatsApp messages, but the study had a loss-to-follow up (n = 21) which resulted in 29 participants being included in the survey/FDGs analysis. Furthermore, this study examined the recorded chats of all participants (n = 45) to assess the application’s usability.

Figure 1. Flow of participants throughout the study.

The flow diagram shows the selection process of the participants throughout the study.
Figure 1. Flow of participants throughout the study.

WhatsApp messages design and development

WhatsApp is an existing application worldwide which was implemented during this study as a mHealth app for health education, health promotion, behavior modification and health communication of arboviral diseases. WhatsApp (version 2.19.241) for Android device was used since all participants had this operating system in their smartphones. Reminder messages were sent through the WhatsApp broadcast function using a simple smartphone model (Moto G). The project name was mobile health because the primary purpose was to disseminate health information with emphasis on arboviral diseases.

All WhatsApp messages were developed following the national guidelines for the prevention and control of DZC [Citation45] and recommendations of previous studies on mobile technology for dengue [Citation18]. In total, 20 short text messages (no more than 350 characters each) were designed for this study. Two additional text messages were incorporated to welcome and thank for the participation in this study as well as one voice message was included for Valentine’s Day in Colombia. Emojis were included in each text messages for making them more readable, lively and user-friendly. An example of WhatsApp messaging is presented in . All messages were written in Spanish language and reviewed by local health staff and national and international experts in arboviral diseases. Messages are described in the additional file 1.

Figure 2. WhatsApp messaging.

Figure legend 2: presents a screenshot of the WhatsApp messaging using the broadcast function which was captured by the facilitator (researcher AMG) who sent the messages from the project’s mobile phone.
Figure 2. WhatsApp messaging.

Messages have four components: (1) health education, (2) health promotion (3) behavior modification, and (4) health communication. These components aimed to improve knowledge, promote a new novel vector control tool for water containers, increase intention-to-change behavior on preventive practices against arboviral diseases and disseminate relevant health information.

Health education

The WhatsApp intervention contained 5 messages with educational information on the three target diseases. Vector and its habitat (breeding sites), warning signs of DZC, and transmission patterns of arboviral diseases were included in this component and then validated by national and international health experts and adjusted by local health staff. Example of message: ‘A breeding site is any water reservoir inside and outside the house where the Aedes mosquito lays its eggs, which develop to larvae and eventually into new mosquitoes that can transmit dengue, Zika and chikungunya’.

Behaviour modification

Eight messages on how to prevent DZC with emphasis on the protection of domestic water containers were developed based on the Communication for Behavioural Impact (COMBI) tool. This approach employs behavioral models as well as communication and marketing theories to engage individual and families in the acquisition of recommended healthy behaviors and encourage the adoption of those behaviors toward the prevention of dengue [Citation9]. Messages were designed as reminder, but also some of them were formulated as questions or instruction for encouraging users to respond through any type of communication (e.g. photo) when they participated in a preventative practice. Some examples were: i) ‘I hope you have not forgotten to clean your laundry tank, especially the walls, corners and joints of your tank’, ii) ‘Did you change the water in the flowerpots and/or the water of your pets’ bowls?’.

Health promotion

Five messages with promotional information on the new vector control method (insecticidal coating) and how to use it were designed following the instructions of insecticidal coatings which were developed by Inesfly Corporation (manufacturer of the insecticidal coating) and tested by the authors [Citation35,Citation36]. e.g.: ‘A new method to combat mosquitoes in your home will soon arrive. This is a transparent protective coating that should be applied to the dry walls of your laundry tank. Let’s participate!’.

Health communication

The purpose of this component was to provide general information on the national and regional health service. One message was illustrated with the slogan of the current national campaign against dengue ‘Cut the wings of dengue’ developed by Ministry of Health (MoH) [Citation45] and a text message related to health staff. e.g.: ‘Allow the staff from the Departmental Health Institute enter your home and follow the instructions related to Aedes mosquito control to prevent dengue, Zika, and chikungunya’.

WhatsApp intervention

All women who had signed up for participation in the WhatsApp trial received a private welcome message, reminding them of saving the phone number. The WhatsApp broadcast function was used to send messages to each participant which allowed easy distribution and anonymity protection in comparison with WhatsApp groups. Participants did not have access to information from others. It was necessary for each participant to save the project number in their own mobile phone as a condition for receiving broadcast messages via WhatsApp, according to the regulations of the app. The WhatsApp infographic function served as a process indicator to monitor message reception, which was previously accepted by the participants through informed consent.

WhatsApp messages were sent anonymously to each registered WhatsApp chat by the author of the study (facilitator). The facilitator was responsible for managing the WhatsApp application and interacting with participants (). The facilitator responded to messages during working hours, ensuring a timely response within one day. They had the freedom to ask questions if they had any doubts about the intervention or were interested in topics related to the management of arboviral diseases and the use of insecticidal coating Messages were sent at 10 a.m. every day (excluding weekends) which was previously agreed upon by everyone. Participants were allowed to respond to messages using text messages, voice messages, videos, and any other reactions since this WhatsApp intervention was designed for a bidirectional communication. In the pre-standardized messages, three were designed to prompt behavior change by asking questions like ‘Did you already covered or turned around all buckets, pots, and bottles in your house?’. It was not mandatory for participants to interact or respond to each message. Participants could choose to withdraw from the study at any time. Participants were also instructed not to forward these messages to anyone until the end of the study. The period of intervention was 23 workdays (approximately 5 weeks).

Table 1. Role of the facilitator.

Research instruments

Pre-and-post-intervention surveys were designed and adjusted following questions on Knowledge, Attitudes, and Practices (KAP) of dengue in Colombia [Citation46]. They were validated by experts and local health staff through participatory workshops. Pre- and post-surveys were disseminated in person during two meetings, the first one for registration in the study and the second one for feedback and closure of the study. The pre-survey had five sections: sociodemographic information, knowledge, practices data and willingness to participate in vector control activities. The post-survey had the same sections with additional questions related to the novel vector control tool and acceptability of the intervention. Extracted information from surveys is found in the additional file 2.

Immediately after the post survey, all women were invited to participate in focus group discussions (FGDs) which were conducted by three female interviewers (AMG, MAC, RCS) who were trained to conduct qualitative research. A total of 29 women participated in two FGDs. They were designed to gather information on women’s perceptions and acceptability of the WhatsApp-based intervention and to validate the WhatsApp messages. The FGDs included five themes on women’s experience with the messages, users’ satisfaction, content of the messages, time and frequency of the messages, and suggestions to improve the WhatsApp program. Both FGDs lasted between 40 and 55 min. The discussions were audio-recorded and transcribed for analysis. Examples of questions were: How did you feel using WhatsApp for dengue prevention? What was your favorite message? How do you think these messages could help you increase DZC prevention?

The WhatsApp chat reviews were collected using the export chat function from the same application to obtain text script of each chat. Text scripts received as input the raw transcript of all messages by the facilitator, participants, and content of each message. Scripts were transferred into a database of the Excel program to evaluate user’s interaction and usability of the intervention. Furthermore, scripts were also extracted and categorized by type of communication of users (text, voice, emoji, among others), usage (frequency of messages by the facilitator and users) and type of user (active or inactive). Extracted chats are found in additional file 3.

Statistical analysis

The quantitative data collected before and after the intervention study were analyzed using the Statistical Package for the Social Sciences (SPSS) version IBM SPSS Statistics 29.0.1.0. The levels of participants’ arbovirus knowledge and practice were scored. They were the primary outcome indicators of the study. The score attributed to each question ranged from 0 to 1, with a maximum score of 29 points as the total number of participants was 29. Therefore, only 1 point was marked for the correct answer and 0 points were marked for the wrong answer. The differences between pre and post scores were tested using the McNemar test. A p-value <0.05 was considered statistically significant.

The qualitative data were analyzed by themes. After reading all FGD transcripts, the research team identified patterns and used inductive coding to identify keywords and concepts to develop a list of common codes to analyze the data. To ensure the reliability of the analysis, a qualified qualitative researcher independently coded the de-identified transcripts. The two coding schemes were then reviewed, discussed, and developed into the final coding scheme by the research team. A total of five themes were analyzed using the grounded theory approach for the qualitative analysis [Citation47]. Additionally, FGDs were conducted followed the guidelines by WHO [Citation48]. The text was extracted from transcriptions to generate analysis content, themes, and findings, and then translated to English.

Chat reviews were calculated with the total number of messages per frequency of messages sent by women and the facilitator. Each interaction of women was also evaluated to estimate the usability in terms of active users who read the WhatsApp messages and users who used the app at least replying once (any type of communication was accepted as a reply).

Results

Fifty women were registered in the WhatsApp messaging application to receive the mHealth intervention for 5 weeks, but only 45 of 50 women received the WhatsApp broadcast messages. The main challenges were failure of saving the WhatsApp project number on cell phones and having an inactive WhatsApp connection. According to the infographic function, 4 of the 50 initial participants did not save the number and 1 participant had her WhatsApp offline, thus they were unable to receive all messages until follow-up. After the intervention, 29 women completed the evaluation questionnaire and participated in the FGD (see above ).

Sociodemographic characteristics of the study population

The baseline characteristics of the 29 women are described in . Most community women lived in Villa del Rosario (62.1%), had a mean age of 40 years (range 20–68 year, SD 12.09), had completed secondary studies (44.8%) and were housewives (44.8%). None of them reached a university degree.

Table 2. Sociodemographic characteristic of study population.

Knowledge of arboviral diseases

shows a considerable improvement of knowledge levels of women about arboviruses-transmitting mosquitos (mosquito’s name: 58.6% to 89.7; p < 0.004; mosquito’s gender: 55.2% to 82.8%; p < 0.008) and the water sources for the development of Aedes spp. vectors which improved by 27.6% (p < 0.021). The post-intervention knowledge of women about breeding sites improved by 17.3% but it was insignificant regarding the warming signs of DZC (p < 0.180). Participants were familiar with the symptoms of dengue and have likely heard about how to prevent it because they came from dengue-affected areas where prevention campaigns have been conducted before. The three most frequently mentioned symptoms were fever (93.1%; n = 27/29), headache (58.6%; n = 17/29), and skin rashes (31,0%; n = 9/29). After the WhatsApp intervention, all the women (n = 29) felt well informed about the transmission and prevention of DZC.

Table 3. Knowledge of community women about DZC.

Practices about the prevention of DZC

presents the proportion of women who were aware of the need to take action regarding two specific activities for preventing DZC transmission. Before the intervention, a high proportion of women knew about the recommended frequency of changing water in pet bowls (93.1%; n = 27) and cleaning laundry tanks (82.8%; n = 24). However, in practice, only around 62% of them followed these recommendations. After the intervention, there was a significant improvement in the adoption of DZC prevention measures. The number of women (excluding 8 who had no pets) who changed water of their pets’ bowls at least every 2 days increased to 33.3% (p < 0.016), and those who cleaned their laundry tanks at least once a week rose by 27.5% (p < 0.008). These findings indicate a positive change in the DZC preventative practices in community women.

Table 4. Practices for the prevention of DZC in women.

Acceptance of WhatsApp intervention

The WhatsApp intervention for improving knowledge and – more importantly – action for the prevention of DZC was well accepted by the 29 women participating until the last phase (participating in FGDs). All of them responded that WhatsApp is an excellent tool for disseminating information on arboviral diseases and improving the willingness to participate in vector control operations, particularly in the application of insecticidal coating for water containers.

From FGDs, women perceived that receiving WhatsApp messages for DZC prevention were a good, useful, and motivating strategy. They agreed that the messages could be used as a source for prevention of arboviral diseases.

  • - ‘It seems very good to me, the idea of WhatsApp messages because that way one can communicate it to the family and it is something that one can do at home, pick up things, clean the jars, but one doesn’t know that there is a good strategy to eliminate the mosquito’ (Participant 11, VdR)

  • - ‘The messages helped us, for example in my home they helped me a lot to prevent, to be more vigilant about the things that … avoid breeding sites and avoid mosquitoes breeding and also avoid a dengue or Zika infection’ (Participant 4, LP)

  • - ‘The messages were very motivating, very excellent, very clear and they helped to reinforce that… this is commitment that one has’ (Participant 1, LP)

Reminder messages via WhatsApp were the favorite channel to send health information in comparison with alternatives media such as flyers and Short Messages Services (SMS).

  • - ‘One never reads them [flyers], instead the messages, it’s like you’re so connected to reality when you read one every day. That’s the way it is. But this flyer, one never reads it’ (Participant 2, VdR)

  • - ‘SMS are boring’ (Participant 4, LP)

  • - ‘I do not read text messages via SMS because there are a lot of advertising for other things, I trust WhatsApp for the health promotion’ (Participant 13, VdR)

Some women also mentioned that the messages help them to increase awareness and adopt a better behavior concerning the elimination of Aedes spp. mosquitoes.

  • - ‘Be aware, awareness. To be aware there is an enemy and it is in the house’ (Participant 3, VdR)

Even, some of the women sent pictures through WhatsApp doing preventative activities, particularly washing tanks which were also registered in the chat reviews.

  • - ‘I even took a picture washing the laundry tank … so that you see that we are attentive to the project because it is very good and beneficial for everyone’ (Participant 18, VdR)

However, some practices were not followed due to technical problems with WhatsApp messaging, the lack of water in some communities or some participants were negligent.

  • - ‘This last [message] did not reach me’ (Participant 2, LP)

  • - ‘I changed my number’ (participant 11, LP)

  • - ‘I forgot it’ (participant 5, LP)

  • - ‘Since I didn’t get water, I emptied the tank and I already had little water’ (Participant 11, VdR)

  • - ‘I don’t get water every day, and I’m not going to throw it away’ (Participant 17, VdR)

Some women suggested that they could be part of this project as ‘multipliers’ and willing to sensitize their communities.

  • - “‘Let’s all share on social media and more that we start to be part of a project, yes! I talk to my neighbor, my family, and so on. What we want is to improve. Well, like us to spread the word, like us to be multipliers and carry out this awareness’ (Participant 1, LP)

Furthermore, the use of WhatsApp was recommended for other health projects, beyond the insecticidal coating.

Content of WhatsApp messages

During the intervention period, a total of 1,566 messages were exchanged in 45 WhatsApp chats over 5 weeks, of which 72% (n = 1132) were sent by a facilitator and 28% (n = 438) were sent by women participants. A list of intervention messages sent by the facilitator is displayed in Supplementary Additional file 1. Emoji were the main type of communication used by women (51.6%) to express their satisfaction or agreement or compliance, followed by text messages (27.6%) (). Since the intervention messages were the same across the chats, analyses are presented with all WhatsApp chats combined.

Table 5. Type of response message sent by community women to the facilitators.

Each message was read and discussed by participants during the FDGs. The content of messages was short, clear, and enjoyable. Generally, women liked the time (10 am), frequency, size, and design of messages, in particular the emojis in the end of every message.

- ‘Something that is important is that the messages are short so that people are going to read it quickly’ (Participant 11, VdR)

- ‘It’s short!, It’s very clear’ (Participant 8, LP)

- “Yes at 10, Yes [every day]. We all want you to write to us (Participant 13, VdR)

- ‘All the messages were very enjoyable, even my son liked the messages especially the emojis at the end. they and even their children liked the messages especially the emojis at the end’ (Participant 6, VdR)

No one reported difficulty when using the app or asked for training on how to use it because they were familiar with the use of this digital tool. However, some words were suggested to be corrected such as ‘Infectious disease’ is preferred over ‘contagious’ and the ‘elevated tank’ should better be called ‘aerial tank’.

Usability

From the fifty women registered in this study, 45 women (90.0%) received WhatsApp messages. Among these active users, 39 (86.6%) of them were able to read all messages as verified by infographic function. Some technical issues meant that a few women did not manage to read all the messages because they initially did not save the project number (which made it impossible to deliver messages by broadcasting function), intermittent internet, and changing phone numbers.

In terms of response, it was found that 37 of active users (82.2%) used the app for replying at least one message and 18 of them (40%) were actively involved replying more than 10 messages and 7 users sharing a photo when they did wash their water container. Only 8 (17.7%) participants did not respond messages even once, most of them coming from Los Patios (n = 7). The messages that received the most interaction were those whose purpose was to promote insecticidal coating and preventive practices toward dengue (more than 30 replies). The messages that received the least interaction were those related to health communication (only 3 replies). No difference was observed between the frequency of days of the week they used the app (average response messages per day = 9 replies).

Costs of WhatsApp intervention

Financial costs are presented according to three major cost items (personnel, material, and technology), and program period (preparation and implementation) (). Over the 5-week period, the total financial costs were US$876 (U.S. dollars); the main cost item was staff cost (facilitator). This amount may increase if the number of users increases as it could require personnel with IT skills, more technological capabilities (computer and software), and licensing permits of WhatsApp providers and service.

Table 6. Cost of intervention over 5 weeks.

Discussion

This is the first study to use the WhatsApp application as a health digital tool for supporting the prevention of arboviral diseases. The study introduced and described the design and development of text messages for a mobile application and the testing in urban communities of Colombia. One of the major findings was demonstrating that the incorporation of mHealth-based interventions through an existing and well-known application such as WhatsApp in vector control programs is feasible and acceptable for certain population groups, particularly women. Previous studies have also demonstrated that the WhatsApp-based intervention is feasible for improving health outcomes in women [Citation49,Citation50].

Improving knowledge and practices

Although from the beginning, participants showed prior knowledge of certain prevention measures to reduce mosquito breeding, there was a significant increase in the application of this knowledge, particularly in actions associated with water containers. After the intervention, there was a clear trend toward the improvements of preventive practices, especially cleaning the laundry tank once a week (pre 62.1% to post 89.6%, p < 0.008). These results may suggest that WhatsApp may not only increase knowledge, but also improve preventive practices. This result is similar to other implementation studies conducted in Peru and Nepal that show improvements in knowledge and positive changes in the practices toward the dengue prevention when participants are exposed to health information through SMS messages [Citation14,Citation15]. However, the integration of communication technologies such as mobile applications or SMS into vector control programs should depend on local needs, technological capabilities, and preferences of the population [Citation18]. Further studies should be conducted at scale to reconfirm this finding.

Acceptability, usability and cost

The acceptance of the WhatsApp messaging for the prevention of arboviral diseases was extraordinarily high in community women who perceive this intervention as a useful and motivational strategy to increase awareness and adopt better behaviors. The mobile health intervention was highly accepted by participants who responded in the quality interviews. Unfortunately, a notable number of participants were lost during the follow-up due to external circumstances reported in informal discussions. Weak internet connectivity, user mistrust, lack of training has been reported in previous studies as barriers in the adoption of a mobile intervention for the prevention and control DZC [Citation18]. However, many participants in this study continued to being a part in the large project. Women showed a particular interest and willingness for participating in the application of the new insecticidal coating which was also reported in the FDGs and chat reviews. In terms of usability, this study showed high proportion (86.6%) of users who read all messages and those (82.2%) who used it for responding and solving doubts. These findings are consistent with other studies which reported high acceptance and usability in mobile applications for the prevention and control of arboviral diseases [Citation51–53] and other diseases [Citation49,Citation54]. Researchers have highlighted that WhatsApp, which is known by the population as a daily tool to interact with friends and relatives, allows a receptive space to exchange health information and solve doubts not commonly discussed during health care [Citation50]. Despite high acceptance and good usability, some technical problems were presented, particularly with broadcast message reception, which need to be considered for further investigation.

The use of WhatsApp broadcasting is an affordable strategy that enables the specific targeting of groups such as community leaders, community volunteers, patients receiving home care, and DZC cases that require follow-up. This facilitates and maintains close cooperation and communication with the vector control or public health staff. Personnel cost can be avoided by task change in the communication departments of the MoH at central and local level. Although there is little evidence related to costs of digital technology in arbovirus management, according to a study, the costs associated with implementing a mobile application for disease prevention and surveillance can be influenced by factors such as community engagement, promotional activities, and one-time investments in technology [Citation55]. The introduction of a mHealth program could also require an initial investment which should be considered in further research. It is important to note that privacy and data protection should always be incorporated. National laws should be reviewed, such as those in Colombia where regulations like Law 1581 of 2012 govern the protection of personal data [Citation56]. Organizations planning to use WhatsApp for broadcasting should ensure compliance with these laws and regulations, which include obtaining consent from individuals before processing their personal data and implementing security measures to safeguard data.

Suggestions for improvement of future studies

For the large intervention trial, WhatsApp can definitely be recommended for the prevention of DZC considering the following aspects:

  • - The close cooperation between local health authorities and other local associations is essential for increasing and sustaining the community participation as women could be multiplicators in their neighborhoods.

  • - Importantly, saving the WhatsApp contact in order to receive the broadcast messages was a condition in this study. It is important to supervise the registration process and mobilize friends and neighbors to remind each other. Vector control staff could also help during their routine inspection of household.

  • - It is recommended to directly save the participants’ contacts in cloud and do backup when recruiting. The creation of a broadcast list is facilitated thereby since manual adding of contacts into the phone and later into WhatsApp can be avoided. Gmail allows for data to be exported in vCard format to a phone’s sim card.

  • - WhatsApp business might be essential for a high number of participants and better chat management. For example, contact numbers should be placed in folders by area, and automated and timed messages should be set.

  • - Codes should be assigned to participants during the registration process for ensuring data protection.

  • - Motivation should be done creatively. People need to be reminded in a comfortable way of simple prevention steps, to not only follow commands of local health authorities but actually understand and self-apply vector control.

  • - WhatsApp should not be abused for the promotion of other purposes (such as commercials); messages should be short, clear and enjoyable.

  • - Staff cost can be avoided by task change in the communication departments of the MoH at central and local level.

Potential limitations

This study had a limited observation period, possibility of ‘politeness bias’, and it is important to note that findings from the larger implementation trial are not included in this manuscript. While the absence of data from the larger trial may limit the depth of our analysis, it is worth mentioning that during the course of the trial, positive assessments and high compliance were observed among women. Furthermore, employing a mixed methods design enabled an analysis between the different pieces of the study, revealing high levels of agreement. This strengthens the reliability of our findings within the scope of this study.

Another limitation was the small sample size, which was not powered to evaluate behavior change. Authors recognize the significance of a more diverse sample and a control group for conducting a more insightful analysis. The analysis was focused on a sample representative of women, as they play a crucial role in vector control efforts at community level [Citation40]. However, participants' loss exceeded our expectations (35% of women did not follow up on the qualitative analysis). This challenge was similar to another study that used WhatsApp messages for improving breast cancer knowledge in women [Citation50]. Despite these challenges, the findings of this study allowed the assessment of the feasibility and relevant information concerning the implementation of a well-known mobile application for promoting preventative messages in arboviral diseases.

Conclusion

This study introduced and tested the use of WhatsApp for the prevention of arboviral diseases demonstrating to be a feasible, acceptable, and affordable strategy. The content of messages was validated and tested by community women as part of a large intervention trial which is being conducted in an urban community of Colombia. In general, the results obtained suggest that WhatsApp could be incorporated as an additional tool in traditional health communication actions and vector operations for combating DZC.

List of abbreviations

mHealth=

mobile health

DZC=

dengue, Zika, chikungunya

IVM=

Integrated vector Management

COMBI=

Communication for Behavioural Impact

ICTs=

information and communication technologies

SMS=

short message services

Apps=

applications

DANE=

National Statistic Department of Colombia

ZMG=

Center for Medicine and Society

IDS=

the State Health Institute

MoH=

Ministry of Health

KAP=

knowledge, attitudes and practices

FGDs=

focus group discussions

SPSS=

the Statistical Packages for Social Sciences

CWP=

changing the water in pet’s bowl

CLT=

cleaning of laundry tank

VdR=

Villa del Rosario

LP=

Los Patios

Authors’ contributions

All authors contributed to the concept and design of the study. MAC, RCS, TRR and AMG did the interviewer training, and drafted the paper. WVK, JL, and AK provided substantial edits on the paper. All authors read and approved the study and the procedures. MAC prepared the submission to the ethics committee. MAC, AMG, and RCS prepared the field study, contacted women associations, and organized the logistics. AMG managed and facilitated the WhatsApp messages. MAC, RCS and AMG conducted the interviewers and questionnaires. MAC, AMG, and TRR contributed to the statistical analysis. AK, WVK, and JL formatted, revised and corrected the article until its final version. All authors read and commented on the draft paper and approved the final version.

Ethics approval and consent to participate

The participation in this study was voluntary, anonymous and did not represent a risk for the participants or their families. This study did not include human participants who were minors. All participants read and signed an informed consent. Participants were also informed they could withdraw from the study at any time. Participants did not receive any kind of compensation for taking part in the study. The WhatsApp registration was done through codes (names were not included in the mobile application). For data protection, the information collected is anonymized: the questionnaire will not record names or addresses. The originals were kept in a safe place (locker) and were not being shown to anybody else. Study results or publications will present aggregated data only. Researchers confirm that all methods were carried out in accordance with relevant guidelines and regulations. The ethics committee of the Albert-Ludwigs-Universität Freiburg, Germany approved the project (protocol number: 142/19) and the intervention was authorized by the local health authorities in Colombia.

Supplemental material

Supplemental Material

Download Zip (47.6 KB)

Acknowledgements

The authors would like to thank participants in this study and the local programs in Villa del Rosario and Los Patios (Families in action and Family, Woman and Childhood) for their valuable support. We gratefully acknowledge the team of the Centre for Medicine and Society, Master Programme Global Urban Health at Freiburg University, particularly Dr Sonia Diaz for her encouragement and support on this work. We thank Johanna Yañez from IDS Norte de Santander for hosting the research team and approval of this study. The authors would also like to thank Roque Suarez and Marcos Rojas from IDS who supervised the field project in each municipality. The authors would also like to thank Adrian Ostre from Freiburg University for fundamental assistance. MAC, TRR and RCS also thank Minciencias, and particularly MAC and RCS are grateful to the Government of Norte de Santander from Colombia for supporting their doctoral studies.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The authors confirm that the data supporting the findings of this study are available within the article or its supplementary materials [additional file 2,3].

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/20477724.2024.2358263

Additional information

Funding

This work was supported by the Ministry of Science, Technology and Innovation (Minciencias) and the State of Norte de Santander in Colombia under Grant number [753]. The Minciencias and the State of Norte de Santander were not actively involved in the design of the study, the data collection, the analysis, the interpretation of the data or in writing the manuscript.

References

  • Mug K, Reiner RC Jr, Oj B, et al. Past and future spread of the arbovirus vectors Aedes aegypti and Aedes albopictus. Nat Microbiol. 2019;4(5):854–863. doi: 10.1038/s41564-019-0376-y
  • Bhatt S, Gething PW, Brady OJ, et al. The global distribution and burden of dengue. Nature. 2013;496(7446):504–507. doi: 10.1038/nature12060
  • Gould E, Pettersson J, Higgs S, et al. Emerging arboviruses: why today? One Heal. 2017;4:1–13. doi: 10.1016/j.onehlt.2017.06.001
  • Deng SQ, Yang X, Wei Y, et al. A review on dengue vaccine development. Vaccines (Basel). 2020;8(1):63. doi: 10.3390/vaccines8010063
  • Nathan M. Introduction. 16 pathogens and global health. Pathog Glob Health. 2013;106(8):427–427. doi: 10.1179/2047772412Z.00000000095
  • Barrera R. Considerations for disrupting dengue virus transmission; ecology of Aedes aegypti and current (nongenetic) methods of control. Gen Control Malaria Dengue. 2016;6:103–124. doi: 10.1016/B978-0-12-800246-9.00006-5
  • Lin H, Liu T, Song T, et al. Community involvement in dengue outbreak control: an integrated rigorous intervention strategy. PLOS Negl Trop Dis. 2016;10(8):e0004919. doi: 10.1371/journal.pntd.0004919
  • World Health Organization. Global Strategic Framework for Integrated Vector Management. [cited 2023 May 4]. Available from: https://www.who.int/publications/i/item/WHO-CDS-CPE-PVC-2004.10
  • Parks W, Lloyd L. Planning social mobilization and communication for dengue fever prevention and control: a step-by-step guide. UNDP/World Bank/WHO, Special Programme for Research and Training in Tropical Diseases. [cited 2023 Mar 3]. Available from: https://apps.who.int/iris/handle/10665/42922
  • Higgs ES, Goldberg AB, LabriqueSH AB, et al. Understanding the role of mHealth and other media interventions for behavior change to enhance child survival and development in low- and middle-income countries: an evidence review. J Health Commun. 2014;19(sup1):164–189. doi: 10.1080/10810730.2014.929763
  • Black AD, Car J, Pagliari C, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLOS Med. 2011;8(1):e1000387. doi: 10.1371/journal.pmed.1000387
  • Sequi-Dominguez I, Alvarez-Bueno C, Martinez-Vizcaino V, et al. Effectiveness of mobile health interventions promoting physical activity and lifestyle interventions to reduce cardiovascular risk among individuals with metabolic syndrome: systematic review and meta-analysis. J Med Internet Res. 2020 Aug 31;22(8):e17790. doi: 10.2196/17790
  • Schoenthaler A, Leon M, Butler M, et al. Development and evaluation of a tailored mobile health intervention to improve medication adherence in black patients with uncontrolled hypertension and type 2 diabetes: pilot randomized feasibility trial. JMIR mHealth uHealth. 2020;8(9):e17135. doi: 10.2196/17135
  • Dammert AC, Galdo JC, Galdo V. Preventing dengue through mobile phones: evidence from a field experiment in Peru. J Health Econ. 2014;35:147–161. doi: 10.1016/j.jhealeco.2014.02.002
  • Bhattarai AH, Sanjaya GY, Khadka A, et al. The addition of mobile SMS effectively improves dengue prevention practices in community: an implementation study in Nepal. BMC Health Serv Res. 2019;19(1):699. doi: 10.1186/s12913-019-4541-z
  • Ryu S World Health Organization. mHealth: new horizons for health through mobile technologies. Healthc Inform Res. 2012 [cited 2019 Feb 18];3(3):231. Available from: http://whqlibdoc.who.int/publications/2011/9789241564250_eng.pdf
  • Boulos MNK, Giustini DM, Wheeler S. Instagram and Whatsapp in health and healthcare: an overview. Future Internet. 2016;8(3):1–14. doi: 10.3390/fi8030037
  • Carrillo MA, Kroeger A, Cardenas Sanchez R, et al. The use of mobile phones for the prevention and control of arboviral diseases: a scoping review. BMC Public Health. 2021 Jan 9;21(1):110. doi: 10.1186/s12889-020-10126-4
  • Statista. Number of Monthly Active WhatsApp Users Worldwide from April 2013 to December 2020. [cited 2021 sep 5]. Available from: https://www.statista.com/statistics/258749/most-popular-global-mobile-messenger-apps
  • Gagnon-Dufresne MC, Azevedo Dantas M, Abreu Silva K, et al. Social media and the influence of fake news on global health interventions: implications for a study on dengue in Brazil. Int J Environ Res Public Health. 2023 Mar 28;20(7):5299. doi: 10.3390/ijerph20075299
  • Giordano V, Koch H, Godoy-Santos A, et al. WhatsApp messenger as an adjunctive tool for telemedicine: an overview. Interact J Med Res. 2017 21;6(2):e11. doi: 10.2196/ijmr.6214
  • Kusuma Dewi DT, Kusumawati W, Ismarwati I. Effect of health promotion and Whatsapp reminder to self efficacy of adherence consume fe tablets pregnant women. J Heal Technol Assess Midwifery. 2(1):23–32. doi: 10.31101/jhtam.683
  • Aguirre PEA, Lotto M, Strieder AP, et al. The effectiveness of educational mobile messages for assisting in the prevention of early childhood caries: protocol for a randomized controlled trial. JMIR Res Protoc. 2019;8(9):e13656. doi: 10.2196/13656
  • Ocampo CB, Mina NJ, Echavarria MI, et al. VECTOS: an integrated system for monitoring risk factors associated with urban Arbovirus transmission. Glob Health Sci Pract. 2019;7(1):128–137. doi: 10.9745/GHSP-D-18-00300
  • Guagliardo SAJ, Ardila Roldan SC, Santacoloma L, et al. Enhanced vector surveillance to control arbovirus epidemics in Colombia. Rev Panam Salud Publica. 2019;43:e50. doi: 10.26633/RPSP.2019.50
  • Higuera-Mendieta DR, Cortés-Corrales S, Quintero J, et al. KAP Surveys and dengue control in Colombia: disentangling the effect of sociodemographic factors using multiple correspondence analysis. PLOS negl trop dis. 2016;10(9):e0005016. doi: 10.1371/journal.pntd.0005016
  • Departamento Nacional de Estadística (DANE). Boletín técnico Indicadores básicos de TIC en Hogares. 2019 [cited 2021 Sep 5]. Available from: https://www.dane.gov.co/files/investigaciones/boletines/tic/bol_tic_hogares_2019.pdf
  • We are social. The global state of digital en octubre 2022. [cited 2024 Mar 14]. Available from: https://wearesocial.com/es/blog/2022/10/the-global-state-of-digital-in-october-2022/
  • Statista. Most popular social networks based on share of users in Colombia in. 2018 [cited 2021 Sep 5]. Available from: https://www.statista.com/statistics/754482/colombia-penetration-social-networks/
  • Deloitte. Estudio de consumo móvil en Colombia 2020. [cited 2020 Sep 5]. Available from: https://www2.deloitte.com/content/dam/Deloitte/co/Documents/technology-media-telecommunications/Brochure_ConsumoMovil_CO_LATAM_2020.pdf
  • USAID - U.S. Agency for International Development. Colombia. [cited 2024 Feb 10]. Available from: https://www.usaid.gov
  • UNHCR. The UN Refugee Agency. Colombia; [cited 2024 Feb 10]. Available from: https://data.unhcr.org/en
  • Herrero PM. La Ciencia al servicio de la solidaridad INESFLY una alternativa para el control de triatominos vectores de la enfermedad del chagas. Experiencia en el chaco boliviano. Revista de sanidad de las Fuerzas Armadas de España. 2007;63(1):65–74.
  • Oropeza V, Pinal R, Mateo P, et al. The residual efficacy of three formulations of insecticidal paints (Inesfly® technology) for the control of Aedes. In: VIII Congreso Nacional de Entomología Aplicada; Mataró, Spain; 2013.
  • Inesfly Coorporation. Inesfly solutions for protection against insects in public buildings. [cited 2023 Jun 5]. Available from: https://inesfly.com/wp-content/uploads/2021/08/INESFLY-SOLUTIONS-FOR-PUBLIC-BUILDINGS.pdf
  • Carrillo MA, Cardenas R, Yañez J, et al. Risk of dengue, Zika, and chikungunya transmission in the metropolitan area of Cucuta, Colombia: cross-sectional analysis, baseline for a cluster-randomised controlled trial of a novel vector tool for water containers. BMC Public Health. 2023;23(1):1000. doi: 10.1186/s12889-023-15893-4
  • Instituto Departamental de Salud Norte de Santander. Análisis de Situación de Salud con el Modelo de Determinantes Sociales Departamento Norte de Santander. 2017 [cited 2023 Jan 3]. Available from: https://ids.gov.co/wp-content/uploads/2020/02/asis-departamental-2017-nortedesantander.pdf
  • Instituto Nacional de Salud-Ministerio de Salud y Protección Social. SISPRO: in Reports ASIS. Datos RIPS. Ministerio de Protección Social. [cited 2024 Mar 10]. Available from: http://rssvr2.sispro.gov.co/reportesAsis/
  • Ogundahunsi O, Kamau EM, editors. Implementation research toolkit. second ed. Geneva: World Health Organization; Licence: CC BY-NC-SA 3.0 IGO. 2018 [cited 2019 Oct 19]. Available from: https://iris.who.int/bitstream/handle/10665/363749/9789241513685-eng.pdf
  • Gunn JKL, Ernst KC, Center KE, et al. Current strategies and successes in engaging women in vector control: a systematic review. BMJ Glob Health. 2018 Jan 7;3(1):e000366. doi: 10.1136/bmjgh-2017-000366
  • Forsyth JE, Mutuku FM, Kibe L, et al. Source reduction with a purpose: Mosquito ecology and community perspectives offer insights for improving household mosquito management in coastal Kenya. PLoS neglected tropical diseases. PLOS negl trop dis. 2020;14(5):e0008239. doi: 10.1371/journal.pntd.0008239
  • Cerrato J, Cifre E. Gender inequality in household chores and work-family conflict. Front In Psych. 2018;9:1330.doi: 10.3389/fpsyg.2018.01330 Epub 2018/08/21
  • Arunachalam N, Tyagi BK, Samuel M, Krishnamoorthi R, Manavalan R, Tewari SC, et al. Community-based control of Aedes aegypti by adoption of eco-health methods in Chennai City, India. Pathog Glob Health. 2012;106(8):488–496. doi: 10.1179/2047773212Y.0000000056 pmid:23318241.
  • Andersson N, Nava-Aguilera E, Arosteguí J, Morales-Perez A, Suazo-Laguna H, Legorreta-Soberanis J, et al. Evidence based community mobilization for dengue prevention in Nicaragua and Mexico (Camino Verde, the Green Way): cluster randomized controlled trial. BMJ. 2015;351. doi: 10.1136/bmj.h3267 pmid:26156323
  • Ministerio de Salud y Protección Social. Cortalé las alas al dengue. [cited 2022 Jul 5]. Available from: https://www.minsalud.gov.co/Paginas/Hay-que-cortarle-las-alas-al-dengue-ministro-Juan-Pablo-Uribe.aspx
  • Higuera-Mendieta DR, Cortés-Corrales S, Quintero J, et al. KAP surveys and dengue control in Colombia: disentangling the effect of sociodemographic factors using multiple correspondence analysis. PlOS Negl Trop Dis. 2016;10(9):e0005016. doi: 10.1371/journal.pntd.0005016
  • Charmaz K. Grounded Theory: objectivist and constructivist methods. In: Denzin NK, Lincoln Y, editors. The Handbook of Qualitative Research. 2nd ed. California: Sage; 2000. p. 509–535.
  • World Health Organization. Template for conducting focus group discussion. Safe Childbirth Checklist collaboration. Instructions for using focus group discussion. Genev (CH): WHO; n.d.
  • Trude ACB, Martins RC, Martins-Silva T, et al. A WhatsApp-based intervention to improve maternal social support and maternal-child health in southern brazil: the text-message intervention to enhance social support (TIES) Feasibility Study. Inquiry. 2021 Jan-Dec;58:469580211048701. doi: 10.1177/00469580211048701
  • Pereira AAC, Destro JR, Picinin Bernuci M, et al. Effects of a WhatsApp-delivered education intervention to enhance breast cancer knowledge in women: mixed-methods study. JMIR mHealth uHealth. 2020;8(7):e17430. doi: 10.2196/17430
  • Rodriguez-Valero N, Oroz ML, Sanchez DC, et al. Mobile based surveillance platform for detecting Zika virus among Spanish delegates attending the Rio de Janeiro Olympic games. PLOS ONE. 2018;13(8):e0201943. doi: 10.1371/journal.pone.0201943
  • Leal Neto O, Dimech GS, Libel M, et al. Saúde na Copa: the World’s first application of participatory surveillance for a mass gathering at FIFA world cup 2014, Brazil. JMIR Public Health Surveill. 2017;3(2):e26. doi: 10.2196/publichealth.7313
  • Patil RN, Almale BD, Patil M, et al. Attitudes and perceptions of medical undergraduates towards Mobile learning (M-learning). J Clin Diagn Res. 2016;10(10):JC06–10. doi: 10.7860/JCDR/2016/20214.8682
  • Luk TT, Cheung YTD, Chan HC, et al. Mobile Chat messaging for preventing smoking relapse amid the COVID-19 pandemic: a pilot randomized controlled trial. Nicotine Tob Res. 2023 Jan 5;25(2):291–297. doi: 10.1093/ntr/ntac045
  • Palmer JRB, Oltra A, Collantes F, et al. Citizen science provides a reliable and scalable tool to track disease-carrying mosquitoes. Nat Commun. 2017 Oct 24;8(1):916. doi: 10.1038/s41467-017-00914-9
  • Función Pública. Gestor Normativo: Ley 1581 de 2012. Ley 1581 de 2012 - Gestor Normativo - Función Pública (funcionpublica.gov.co). [cited 2024 Feb 13].