Patient experience with an educational mobile health application: A pilot study on usability and feasibility in a Saudi population

Abstract

Evidence-based guidelines recommend that patient knowledge should be considered an essential component in oral anticoagulation treatment (OAT) to improve adherence and subsequent clinical outcomes. Therefore, a mobile health (mHealth) technology-based educational intervention using smartphone features could be a promising approach for patients receiving OAT. However, mobile applications (mApps) developed in this context are rare, and there is still a lack of research examining the perceived usability of these apps from end-users’ perspectives and the feasibility of implementing them in daily clinical practice for chronic OAT end-users. In this study, we developed a novel mApp educational tool about OAT (warfarin) and evaluated the perceived usability of this app among Saudi adult patients. In addition, the impact of the app intervention on anticoagulation control parameters was examined. The Coagulation and Anticoagulant Therapy and Awareness (CATA) mApp was developed by a collaborative effort from a multidisciplinary team using the Android system. The mApp content was processed using an intuitive user-friendly interface containing simple Arabic text. The app was uploaded to the mobile phones of patients who were randomly selected from two medical centers in Riyadh, Saudi Arabia. Usability evaluation was conducted after 6 months post-mApp intervention, using a comprehensive questionnaire that addressed five main dimensions of participants’ perceived usability of the mApp: attractiveness, ease of use, satisfaction, operability, and presentation quality. Thirty-seven participants (M_age = 45 years) completed the usage survey. The mean overall usability score (76.8%) demonstrates that most participants were satisfied with the CATA mApp and found it useful. Of the five user experience domains examined, patients expressed the highest agreement score for the presentation quality domain (79.5%), reflecting the attractiveness of the visual appearance and logic of information demonstration of the app. Additional themes were identified by some users in the free-text questions, indicating that further unique features such as longer audiovisual demonstrations and direct web browser facility should be added to ensure sustained engagement with the app in their daily routines. Regarding international normalized ratio (INR) control parameters, no significant changes were observed. Further feedback from practitioners is warranted to gain constructive insights into the facilitators and barriers for mApp implementation in routine care. Additionally, the current data revealed that there are concerns about increasing INR control via the incorporation of a computing tool that empowers patient data entry and doctor sharing of patient records. These features are projected to ultimately bridge the gap between patient adherence and clinical outcomes.

Keywords:

• mobile application (mApp)
• mobile health (mHealth)
• oral anticoagulants
• warfarin
• usability
• feasibility
• patient education

PUBLIC INTEREST STATEMENT

Medical mobile Apps are a key source of knowledge and awareness, and they are often the only reference for patients, especially in the urban areas of developing countries where both information and expert opinions are limited. In Saudi Arabia, several studies indicated that patients’ knowledge, adherence, and satisfaction of anticoagulant therapy are poor. However, none of these studies involved any intervention to educate patients and improve their compliance. Moreover, there is a lack of research on the experiences and preferences of our national stakeholders, patients, and care providers in using Arabic mobile Health apps, which can often result in limited implementation in daily practice. The current study examined the patient experience with an educational Arabic mobile Health Apps concerning anticoagulant medications which were selected from two medical centers in Riyadh, Saudi Arabia.

1. Introduction

Given the current expansion of multifunctional smartphones, most people in various countries are increasingly using mobile applications (mApp) for work in diverse fields (“Mobile App Usage Statistics,” 2017; Newzoo’s, 2018). This expansion can be attributed to facilitations of the compact size and mobility of smartphones. The medical field is no exception (Liu et al., 2011). Medical mApps are a key source of knowledge and awareness, and they are often the only reference for patients, especially in the urban areas of developing countries where both information and expert opinions are limited (Akter & Ray, 2010).

Recently, the Food and Drug Administration acknowledged the utility of mobile health (mHealth) technologies for various health-care purposes, including improving patient self-monitoring and management, increasing general patient education, and facilitating patient access to commonly used reference information (Yetisen et al., 2014). The emergent attractiveness of mobile solutions for health promotion and health-care education can be ascribed partly to the following: the accessibility of the technology; the level of personalization that such technology enables, such as valuable location-based services (i.e., patients can quickly extract valuable information); and tailored care and education to patients’ needs (Akter & Ray, 2010).

Some studies have examined mHealth interventions for preventing or managing chronic conditions (e.g., diabetes (Lim et al., 2011) and cardiovascular diseases (Burke et al., 2015)). Although initial results suggest benefits in clinical practice (e.g., healthcare cost reduction and supporting health behavior change (Bhavnani et al., 2016), evidence-based literature is still insufficient. Additionally, a significant but inadequately researched area concerning engaging and educating patients and health-care providers about proper medication use while initiating them into using mHealth Apps (Aranda-Jan et al., 2014). Several features, such as the integration of a calendar and the excellent digital audio and video recording quality, make smartphones an ideal vehicle for recording complex drug information that patients can have access to anytime, which can facilitate and warrant treatment compliance (Grindrod et al., 2014; Luxton et al., 2011; Malvey & Slovensky, 2017; Yazdanshenas et al., 2016; Yeung et al., 2017).

It is important to note that in a frequently changing emerging field, conducting usability studies to evaluate acceptance and preferences by end-users should be mandatory (Matthew-Maich et al., 2016). Such pilot studies, which are referred to as usability tests, can potentially clarify experiences regarding ease of use and other specific needs (e.g., visual appearance, audio style, text language, and size) by means of direct patient feedback (Moumane et al., 2016). Patient feedback can shed light on aspects that require modification in the mApp and can allow for end-user customization before implementation in clinical practice to ensure successful adoption of an mHealth solution (Luxton et al., 2011; Matthew-Maich et al., 2016; Moumane et al., 2016).

Oral anticoagulation treatments (OAT), such as warfarin therapy, are usually employed for long-term chronic use and for the prevention and treatment of venous thromboembolic diseases (Briere et al., 2019), which could frequently expose patients to over- or under-coagulation (Yoshida et al., 2018). Therefore, the efficacy and safety of warfarin requires an optimum quality of anticoagulation control, which was demonstrated by a time in therapeutic range (TTR) of more than 70%, while minimizing the risk of serious adverse bleeding events (Sjögren et al., 2015). However, achievement of this target may be limited by the need for regular monitoring services owing to its narrow therapeutic range, the requirement of individualized dosing according to genetic differences, and the possibility of food and drug interactions (Hirsh et al., 2003). Poor patient education in OAT leading to low treatment adherence rates have also been observed to significantly intensify the risk of anticoagulation-related adverse events (Pernod et al., 2008).

Based on these well-established findings, evidence-based guidelines recommended that patient knowledge should be considered an essential component in warfarin therapy to improve adherence and subsequent clinical outcomes (Ansell et al., 2004; Hirsh et al., 2008). An mHealth educational intervention using smartphone features could be a promising approach for patients receiving OAT. Nevertheless, our literature review revealed that the number of working mApps developed with this intention is very limited (Guo et al., 2017; Kasemsap, 2018). Additionally, there is still a lack of research examining the effectiveness of mHealth tools on therapy knowledge or adherence, as well as on the feasibility of implementation of these mApps in daily clinical practice for chronic anticoagulant therapy end-users.

In addition, mHealth technologies have limited implementation in medical settings in Arabic countries (Kasemsap, 2018). At the international level, only two studies have addressed the development and testing of an mApp approach directed to patients receiving OAT. In the United States (Lee et al., 2016), a feasibility study was conducted to evaluate an mHealth app called MASS (Mobile Applications for Seniors to enhance Safe anticoagulation therapy) to promote independence and self-care. MASS was intended specifically for older English- or Spanish-speaking adults. In that project, the MASS app was mainly concerned with two components: (a) education about anticoagulation therapies and safety tips, and (b) medication self-monitoring and reminders. The study revealed a significant increase in anticoagulation knowledge after the 3-month intervention, and participants also reported satisfaction with MASS regarding its ease of use and usefulness. In China (Guo et al., 2017), an mApp (called mAF App), which integrates clinical decision support, education, and patient involvement strategies, was specifically designed for atrial fibrillation patients receiving anticoagulant therapy. The mAF App was tested in a pilot trial with a structured follow-up plan, as compared to usual care for 3 months post-intervention. The mAF App significantly improved knowledge, drug adherence, quality of life, and anticoagulation therapy satisfaction (Guo et al., 2017).

In Saudi Arabia, national surveys in distinct settings demonstrated a high prevalence of poor knowledge of basic warfarin aspects (Elbur et al., 2015; Mayet, 2015, 2016; Shilbayeh et al., 2018), leading to patients’ nonadherence and poor satisfaction scores (Elbur et al., 2015; Shilbayeh et al., 2018). The same was observed in other international populations (Davis et al., 2005; Wang et al., 2013). Given the fact that Saudi Arabia has one of the highest rates of mobile phone usage in the world (Newzoo’s, 2018), and since there have been no formal development and feasibility studies for mApps in any health education programs to date, our aims were as follows: (1) to develop a simple educational mApp about anticoagulants, (2) to evaluate the perceived usability of the newly developed app among adult patients receiving OAT therapy, and (3) to explore the impact of app intervention on anticoagulation control parameters.

2. Methods

2.1. Sample and recruitment

To test the perceived usability of the mApp, a convenience sample of 40 patients, who were visiting two anticoagulant clinics (ACCs) at the Security Forces Hospital and Prince Sultan Military Medical City in Riyadh for follow-up visits, was recruited. This sample size was satisfactory since previous studies with samples up to 20 in usability testing were successful (Faulkner, 2003).

Eligibility criteria for participation included the following: (a) aged >18 years, (b) Arabic speaking, (c) receiving OAT anticoagulant (mainly warfarin) and visiting the clinic regularly, (d) a naïve patient (i.e., OAT was prescribed, but not received yet), and (e) using smartphones or tablets with the Android operating system. We excluded patients (a) aged ≥70 years; (b) previously diagnosed with mental disorders; (c) with visual, auditory, or oral communication deficiencies; and (d) who refused to come back for the second visit after 45 days (or his/her scheduled appointment date).

2.2. Ethical approval

Before employing the Coagulation and Anticoagulant Therapy and Awareness (CATA) mHealth App (described below), the study protocol was reviewed and approved by the Institutional Review Board at Princess Nourah University (PNU), and at each study site (17–0074).

2.3. Procedures and intervention

The CATA mApp was uploaded to patients’ mobile phones via the Internet and using a password. Additionally, a brochure describing the download procedure was provided to patients who faced difficulties (e.g., low battery, lacking Internet access, etc.). After a couple of days, patients were contacted to see if they had any questions or faced technical problems. Participants were also informed that, after 6 months of using the mApp, they would be asked to complete a questionnaire about their perceptions and experience with the app, which would take about 30 minutes. Participants were guaranteed anonymity and were informed of the confidentiality of the results.

2.4. CATA mApp development

The CATA mApp was developed by the collaborative efforts of the pharmacy practice and computer science faculties at PNU. Microsoft Visual Studio was used to develop the application for Android, including an intuitive user-friendly interface and easy-to-read simple Arabic text and images. An audio feature was also incorporated for those who were unable to read. Additionally, the CATA mApp was designed to be computationally powerful and readily available for users everywhere and at any time.

The CATA mApp included three main educational headlines (Multimedia Appendix 1: Supplementary Figure 1): (a) basic knowledge, (b) important tips for patients, and (c) frequently asked questions (FAQs). The basic knowledge headline included two specific subheadings: warfarin basic information (Multimedia Appendix 1: Supplementary Figure 2) and injectable anticoagulant information (Multimedia Appendix 1: Supplementary Figure 3). Important tips for patients included general information summaries on questions that patients may ask before leaving the ACC. The tips were written in short sentences to facilitate patient understanding (Multimedia Appendix 1: Supplementary Figure 4). The FAQs and responses to them are listed in Supplementary Figure 5 (Multimedia Appendix 1).

The CATA mApp had additional features for notifications and alerts including self-reminders for the next appointment, time to measure international normalized ratio (INR), and time to take the warfarin dose (Multimedia Appendix 1: Supplementary Figure 6). All alerts were developed to be comprehensive based on an intensive literature review of OAT variable educational tools (Wofford et al., 2008), basic aspects of warfarin therapy (e.g., action, potential risks, interactions with other drugs or foods, individualized INR targets or therapeutic ranges) (Briggs et al., 2005; Zeolla et al., 2006), and knowledge gaps identified in previous Saudi population studies (Elbur et al., 2015; Mayet, 2015, 2016; Shilbayeh et al., 2018). Participants were encouraged to regularly review all CATA mApp educational issues and set alarms to remind themselves of their ACC appointments and help them adhere to their treatment regimens.

2.5. User survey

Usability evaluation of an app is concerned with how well users can utilize its functions and ultimately achieve its intended purpose (Moumane et al., 2016). User survey is one of the most widespread methods employed in mApps’ usability studies (Jake-Schoffman et al., 2017). Based on a literature review of similar tools, a comprehensive usability questionnaire was developed to assess patients’ perceptions of CATA mApp and the key elements of perceived mApp usability as defined by usability experts (Brooke, 1996; Lee et al., 2014; Lewis, 1995, 2002; Moumane et al., 2016; Nahm et al., 2006).

First, a list of standard software quality items and additional mApp customized items were selected from available questionnaires. The question list was then refined to include what best met the usability aspects that are related to the education of anticoagulant patients (Burke et al., 2015; Lee et al., 2014). Second, content validity was reassessed by examining agreement between the objectives of the scale and items by three external anticoagulant therapy and three usability experts. Third, the questionnaire was translated to Arabic by two professional forward translators. A focus group was assigned to evaluate the conceptual equivalence and the applicability of the translated Arabic version for our specific purposes.

The final questionnaire draft was divided into three sections: participants’ consent, demographic characteristics (e.g., gender, age, education level, frequency of using mApp in healthcare), and participants’ usability experience survey. The usability survey included 27 items that addressed 5 main dimensions of mApp usability: attractiveness (4 items), ease of use (5 items), satisfaction (6 items), operability (8 items), and presentation quality (4 items). All items were rated on a 5-point Likert scale (strongly disagree = 1 to strongly agree = 5), including a neutral midpoint. Responses to all items (except item 6 and item 7) equal to or greater than 4 were considered favorable. Items 6 and 7 on the questionnaire were reverse-coded. A rating of 2 or less on these items was considered favorable, reflecting greater usability score. Average usability scores were set for each item. The total score of each subscale (domain) was calculated by summation of the scores of all items within each construct. The score for the overall scale was calculated by summing the scores of all subscales. Higher scores indicated higher perceived usability of the mApp. Three additional free-text questions were formulated to ask participants what kind of user could most benefit from the CATA mApp, and to obtain further information about what participants liked and disliked about the design of the mApp.

The survey tool was electronically administered to end-user patients through WhatsApp after an average of 6 months of their CATA mApp use.

2.6. Anticoagulation control

For each patient, 4 INR readings (pre- and post-intervention) were obtained to quantify the quality of INR control as a secondary outcome. Subsequently, the quality of INR control was quantified by two indicators: (1) the mean percentage TTR (days) using the Rosendaal method (Rosendaal et al., 1993); and (2) the percentage of visits for which the INR readings were in range (INR stability), according to Rose and colleagues’ methodology (Rose et al., 2009).

TTR is a key outcome, which is a well-authenticated surrogate marker for complication rates and is accepted as a standard quality indicator (Kaatz, 2008).

2.7. Data analyses

Data were coded, entered, and analyzed using the SPSS (version 23.0; SPSS Inc., Chicago, IL). Patients’ responses to each questionnaire item were expressed by mean (M) and standard deviation (SD). The total score for each usability construct was estimated by calculating the M and SD of the responses from each subgroup items. The minimum possible summation score on all the items was 27 and the maximum was 135. The overall usability score, out of 100%, was rated as unsatisfactory (<50%), satisfactory (50–60%), good (61–69%), very good (70–79%), or excellent (80–90%). The internal consistency and reliability of the usability questionnaire was assessed using Cronbach’s alpha (Taber, 2018) for the scale and its 5 dimensions and using the inter-item and item-total mean correlations.

3. Results

3.1. Participant information

Thirty-seven participants completed the usage survey. The response rate, calculated as the total number of surveys submitted post CATA mApp usage divided by the number of completed surveys, was 92.5%. The characteristics of the respondents are illustrated in Table 1. Most participants were Saudi current warfarin users. Participants' mean age was 45 years (SD = 10). Most had completed at least secondary school education or higher. Only 16% of participants indicated that they had either used the mApp often or very often. Nearly all were invited to use the CATA mApp by pharmacists.

Table 1. Participants’ demographics (N = 37)

Variable	n (%)	Variable	n (%)
Age group (years)		Warfarin use
18–24	6 (16.2)	Current users	26 (70.3)
25–34	7 (18.9)	Learned about CATA mApp
35–44	8 (21.6)	From a doctor	4 (10.8)
45–54	5 (13.5)	From a pharmacist	22 (95.5)
55–64	10 (27)	From a friend	6 (16.2)
≥65	1 (2.7)	Other means	5 (13.5)
Sex		Using mApp for healthcare
Male	7 (18.9)	Never	3 (8.1)
Female	30 (81.1)	Rarely	15 (40.5)
Education		Sometimes	13 (35.1)
Illiterate	2 (5.4)	Often	4 (10.8)
Primary education	4 (10.8)	Very often	2 (5.4)
Intermediate education	5 (13.5)	Specific usage areas of mApp
Secondary education	6 (16.2)	Health awareness	29 (78.4)
University education	19 (51.4)	Self-care	9 (24.3)
Higher education	1 (2.7)	Diagnosis	4 (10.8)
Employment		Follow-up and reminder	6 (16.2)
Currently employed	14 (37.8)	Other	2 (5.4)
Hospital
Security Force Hospital	19 (51.4)
Prince Sultan Military Medical City	8 (21.6)

3.2. Reliability and validity of the usability questionnaire

The usability questionnaire had excellent reliability (Cronbach’s alpha = 0.95). Additionally, the discrimination indices (corrected item-total correlation coefficients) of the questionnaire items were greater than 0.4, which are psychometrically acceptable (minimum value of 0.2) and ranged 0.44–0.84. The independent Cronbach’s alpha value for the five subscale domains (attractiveness, ease of use, satisfaction, operability, and presentation quality) was consistently higher than 0.75 (range = 0.76–0.91), demonstrating acceptable to good internal consistency, suitability, and relevance.

3.3. End-user usability testing

Patients’ responses to each individual item on the perceived CATA mApp usability questionnaire are provided below (Table 2). The range of total scores was 64 to 133. The mean total score was 103.7/135 (SD = 16.7; 76.8%). This indicates that most users found the CATA mApp usable (Table 3).

Table 2. Patients’ ratings on the perceived CATA mApp usability questionnaire^a (N = 37)

Item	Mean (SD)
Attractiveness
There is nothing on the CATA mApp that distracts attention from the content.	3.62 (1.01)
The visual layout of the CATA mApp attracts attention.	3.73 (0.93)
The CATA mApp has novel or unique features that make it more interesting.	3.51 (0.96)
A variety of formats for presenting information (e.g., text, images) helps maintain attention.	3.70 (1.07)
Ease of Use
You would think that most people would learn how to use the CATA mApp very quickly.	3.70 (0.84)
You found the CATA mApp very cumbersome to use^b.	3.81 (1.17)
You needed to learn a lot of things before you could get going with the CATA mApp^b.	3.59 (1.18)
Navigating the CATA mApp does not require any special skills or experience.	4.03 (0.76)
Overall, the information in the CATA mApp is accessible without restricted permissions.	4.08 (0.75)
Satisfaction
The purpose of the CATA mApp is clear.	4.14 (0.88)
The CATA mApp appears to contain credible information.	3.97 (0.83)
Overall, the information on the CATA mApp appears to be accurate.	3.84 (0.92)
Overall, the information contained in the CATA mApp is current and up-to-date.	3.70 (0.99)
The CATA mApp contains little or no redundant or irrelevant information.	3.19 (1.24)
The CATA mApp provides adequate coverage of topics presented.	3.97 (0.83)
Operability
The CATA mApp provides opportunities to communicate with its creators for on-line help.	3.70 (0.87)
No matter where you are in the CATA mApp, you can return directly to the home page.	3.84 (1.01)
The design of the CATA mApp uses a navigation system that enables efficient access to any section from any page on the application.	3.86 (0.85)
The CATA mApp features are active and fully functioning.	4.08 (0.72)
Buttons, links, and other navigation mechanisms work the way they should on the CATA mApp.	4.08 (0.59)
There is little or no delay in accessing the content of the CATA mApp.	4 (0.88)
The CATA mApp is optimized for mobile access (i.e., smartphones, tablets, etc.).	3.86 (0.82)
You can control what information you wish to access on the CATA mApp.	3.76 (0.95)
Presentation Quality
The information on the CATA mApp is presented in a clear and well-organized manner.	3.97 (1.11)
The text of the CATA mApp is well-written without grammatical, spelling, or other errors.	3.97 (1.01)
The organization of the CATA mApp is simple and clear.	3.95 (0.99)
Visual (e.g., interface, photographs, and color) content included in the CATA mApp helps to clarify or describe the topics presented.	4 (0.88)

^aAll items were rated on a 5-point Likert scale (strongly disagree = 1 to strongly agree = 5), including a neutral midpoint. A mean value of ≥3.0 was considered favorable.

^bItems 6 and 7 were reverse-coded (1 = strongly agree to 5 = strongly disagree).

Table 3. The perceived CATA mApp usability questionnaire total subscale scores (N = 37)

Subscale (range)	Mean (SD)	Mean percentage converted	Cronbach’s alpha
Attractiveness (4–20)	14.57 (3.3)	72.9	0.84
Ease of use (13–25)	19.2 (3.5)	76.8	0.76
Satisfaction (6–30)	22.8 (4.6)	76	0.89
Operability (20–40)	31.2 (4.8)	78	0.86
Presentation quality (4–20)	15.89 (3.6)	79.5	0.91
Total (64–133)	103.7 (16.7)	76.8	0.95

The mean score for the attractiveness domain was 72.9%, which is considered good. However, it also indicates that additional unique features and formats for presenting information should be added to make it more interesting and to ensure sustained engagement with using the app daily. The mean score for the ease of use domain was 76.8%, indicating that the app has achieved a satisfactory level of simplicity and accessibility. The mean score for the overall satisfaction domain was 76%, which reflects that patients were highly satisfied in adopting new technology with the CATA mApp. However, the close to neutral score in item 15 (mean, 3.19; SD, 1.24) indicates that further in-depth reviewing and editing of redundant or irrelevant information is essential to meet the educational needs of various end-users. The mean score for the operability domain was 78%, indicating the app was excellent regarding its functionality, flexibility, and control features of use. The mean score for the presentation quality domain was 79.5%, reflecting admiration for the visual appearance and logic of information demonstration of the app.

Concerning participants’ perceptions toward the CATA mApp, as observed in the responses to the free-text questions, patients with any disease condition (56.8%), followed by warfarin users (24.3%), patients with thromboembolic disorder (16.2%), or patients’ caregivers (5.4%) were indicated as ideal users for this mApp.

Additionally, participants expressed that the ease of use, short access time, clear and simple information, illustration figures, colorful presentation, useful alert notes, dosage and clinic appointment reminders, and laboratory diary were among the most desirable features of the app. Regarding the disliked app characteristics that were resolved through participants’ text answers, few participants highlighted the lack of regular updating of information (n = 3), little space and time availability for audiovisual (video) demonstrations for drug-related problems (n = 3), or bulk of general medical knowledge (n = 2). Some participants provided positive comments regarding the ease of the overall app use in obtaining answers for queries they were intending to ask during their next ACC visits. Others suggested the addition of a navigation property (direct web browsers) that allows searching for more in-depth information without needing to book an ACC appointment (e.g., blood coagulation risk factors, complications, and prevention measures). One patient reported difficulties in opening some pages when navigating through different sections in the app. Another patient indicated an overlap in audio-messages associated with reading through the various app parts and during navigation.

3.4. INR control

There were slight improvement trends in mean values of TTR percentage and INR stability percentage; however, these were non-statistically significant (Table 4). Nonetheless, by applying the known clinical categorization scale for TTR percentage (<60% [poor control], 60–75% [moderate control], and >75% [optimal control]), and the classification mode of INR stability parameter (i.e., value of ≥50% “safe warfarin management,”), we noticed a global shift in patients’ mean values in both INR quality measures (i.e., TTR and its stability percentage) from poor to moderate control (TTR percentage = 60.58%), which was parallel with the safer warfarin use condition (INR stability = 55.1%). Whether this clinically significant variation was related to the CATA mApp use is an issue that remains to be elucidated in a larger sample study with a longer follow-up period.

Table 4. Effect of CATA mApp intervention on INR quality measures (N = 31)

Measure	Baseline		Post-intervention		P^a
	Mean (SD)	Range	Mean (SD)	Range
TTR %	55.9 (32.6)	0–100	60.58 (36.5)	0–100	.60
INR stability %	51.1 (31.8)	0–100	55.1 (36.1)	0–100	.58

^aWilcoxon signed rank tests for nonparametric or paired samples t-test for normal distributed outcomes.

4. Discussion

Previous studies indicated that the use of mHealth interventions can potentially support the successful management of chronic conditions through several aspects including tailoring care and education considering patient necessities, providing indirect feedback interactions, and improving their communication with their health-care providers (Burke et al., 2015). However, these studies were mostly limited to pilot measuring of users’ satisfaction, which lacks a comprehensive framework of usability testing (Moumane et al., 2016), and evidence-based evaluations demonstrating clinically meaningful changes in physiological parameters of users’ medical conditions (Burke et al., 2015; Matthew-Maich et al., 2016). Additionally, few published trials have been conducted within the context of mApp interventions to improve medication knowledge, adherence, and healthcare for low mHealth-literate populations (Parker et al., 2018; Wali et al., 2016).

Recently, a pilot qualitative analysis revealed that older American adults on warfarin therapy are interested in mHealth technology, specifically for warfarin medication management, which may enhance their self-care (Lee et al., 2014). In Saudi ACCs, several studies indicated that patients’ knowledge, adherence, and satisfaction of anticoagulant therapy are poor (Elbur et al., 2015; Mayet, 2015, 2016; Shilbayeh et al., 2018). However, none of these studies involved any intervention to educate patients and improve their compliance. Overall, there is a lack of published research on mApp use in this therapeutic area worldwide and more data are needed to address the potential usability, implementation, effectiveness, and cost-effect of apps that are being purpose-developed to improve OAT outcomes, as compared to usual care in ACCs or primary care settings (Guo et al., 2017; Lee et al., 2016). Moreover, there is a lack of research on the experiences and preferences of our national stakeholders, patients, and care providers in using such mHealth apps (Atallah et al., 2018), which can often result in limited implementation in daily practice (Matthew-Maich et al., 2016).

In this study, we developed a novel mApp educational tool to improve knowledge and awareness of OAT, specifically warfarin, to enhance safe and effective clinical outcomes. Unlike the two previous apps, the CATA mApp was intended to be employed among Arabic patients of all ages (Lee et al., 2016) and with any indication of OAT (Guo et al., 2017), mainly for chronic thromboembolic complications. The findings concerned with the impact of the app on knowledge improvement specific to OAT are published elsewhere; this article, however, focused on the usability of the app based on users’ opinions, which are helpful for improving its practical application.

To generate solid evidence of the usability of the app, an Arabic version of a usability questionnaire was developed and validated. The questionnaire was reliable and comprehensive in its inclusion of the main usability characteristics of an mApp (Brooke, 1996; Jake-Schoffman et al., 2017; Lee et al., 2014; Lewis, 1995, 2002; Nahm et al., 2006): attractiveness, ease of use, satisfaction, user operability, and presentation quality. The mean scores for all domains were very good. This indicates that patients were highly convinced to adopt the new technology with the CATA mApp and reported it to be accessed effectively in their daily life. The overall usability score reflected a high level of enthusiasm and readiness of the current population that would enable successful mHealth implementation. These results confirmed other research findings, which showed that most patients receiving OAT agreed that educational mApps were easy to use and helpful (Guo et al., 2017; Lee et al., 2016). However, some participants raised specific concerns and offered suggestions concerning how to make the app more interesting. For example, feedback from some patients indicated their desire to have additional interactive functions such as longer video demonstrations and direct web browsers enabling search for regular and updated drug information. These comments helped identify potential aspects that may be considered in future CATA app improvements to sustain its utility.

Many previous studies verified a significant relationship between patients’ warfarin knowledge gained through ACCs’ routine care and their improvement in anticoagulation control (Gadisseur et al., 2003; Khan et al., 2004; Mörsdorf et al., 1999; Sjögren et al., 2015; Tang et al., 2003). However, none of the mApp-intervention studies reported impacts on INR control parameters (Guo et al., 2017; Lee et al., 2016). Our attempt to monitor INR revealed that patients on warfarin post-intervention achieved higher levels of TTR than those reported by other patients in similar clinical settings (Hirsh et al., 2008; Sjögren et al., 2015). However, our failure to demonstrate a significant difference in INR control parameters post-intervention could be partially attributed to a relatively inadequate number of INR values recorded in patients’ medical records pre- and post-intervention during the study period. Therefore, we cannot draw conclusions regarding the impact of knowledge gained through the mApp on the quality of anticoagulation control.

Prior studies concerning patients’ self-testing of anticoagulation demonstrated that a higher frequency of INR monitoring will enable more accurate dose adjustments leading to higher PTT percentage in most patients (Gadisseur et al., 2003; Khan et al., 2004; Mörsdorf et al., 1999). Therefore, future developments of CATA mApp should consider more features that enable patients to regularly and frequently record their INR values in the app. This feature should be enhanced with computational tools to automatically calculate TTR percentage and INR stability parameters. Additionally, a feature that will enable patients to share the data with their ACC health-care providers to update them with their status should be considered. These unique features of mobile devices will further enhance patients’ involvement in their self-management and decision-making, through better agreement with their health-care providers, thereby improving adherence and satisfaction.

4.1. Limitations

This paper was one of the first to examine the acceptability and usability of an mHealth app in a Saudi patient population. However, it had some limitations. First, this was a pilot analysis that involved only a target patient group in two Saudi ACCs. Therefore, the results might not represent the general perception of the entire Saudi population regarding mHealth solutions. Therefore, it is important to consider that some patients will accept the use of mHealth apps, while others might prefer to continue with the standard clinical care provided in ACCs. Second, the usability data were only generated by the survey questionnaire directed to patients mainly receiving warfarin. Future larger scale studies should involve more direct assessment of various ACC practitioners’ preferences pertaining to aspects of app design and implementation (e.g., feasibility, acceptability, and usability in relation to the different end-users), in addition to other health outcomes experienced by clients receiving the interventions.

By combining these results with the current findings, we will gain constructive insights into the facilitators of, and barriers to, agreeable outcomes for mobile app use among OAT users. The estimation of these parameters could be applicable by various approaches: face-to-face interviews, focus groups, or structured questionnaire surveys to be administered post-intervention in a large sample size, including both patients (users) and their caregivers. These strategies will also enable better evaluation of variations in knowledge attainment associated with mApp in other groups (such as male sex, less frequent ACC visitors, new warfarin users, and non-herbal users; Matthew-Maich et al., 2016).

4.2. Future work concerning CATA mApp

The promising findings in this study provide insight into various approaches that could be employed in future research. First, the usability themes highlighted by patient-users indicated that additional mApp functionalities such as clinical data entry and health-care provider feedback would be helpful. Specifically, the practical benefit of such a function, compared to the traditional methods of recording INR, would save time in busy ACCs and allow for quicker responses, particularly for patients living in remote outreach areas.

Second, clinical pharmacists who specialize in OAT will be targeted in our future usability studies. Their opinions about the app can serve as a qualified input to support these services. The recruitment of other relevant experts will be pursued via email invitations that will be sent to several official universities and medical centers in Saudi Arabia. This professional group will be provided with the details to download the CATA mApp and will be instructed to test the app for a period of 7 days before the semi-structured interviews. This targeted group should provide a deeper understanding of the context in which future implementation of the app should occur.

Third, the CATA mApp was only designed for the Android system, which limits its utility. Therefore, further development of the CATA on Apple’s iOS platform is needed since many Saudi patients are using iOS devices (Newzoo’s, 2018); this will help us take advantage of the larger screen offered by iPad by developing an iPad version of this app (Liu et al., 2011).

Fourth, given the need for continuous updates in evidence-based therapeutic guidelines and considering the rapid evolution of mHealth technologies, our future studies will address the enhancement of the mApp to include more interactive features for communication between patients and clinical researchers such as automated data-generating and incorporated standardized scales designed to capture specific health outcomes, such as patient treatment adherence, quality of life, and disease scores. Finally, to expand the future utility in other cultures, the CATA mApp should be bilingual, offering content in both English and Arabic.

5. Conclusions

This study introduced the CATA, a newly developed mApp, which was purposely designed to improve patients’ knowledge and awareness about diverse aspects of OAT, specifically warfarin. The findings highlighted the app was compatible with patients’ educational needs and daily routines. However, the study described additional features desired by current patients considering their use and perceptions, which can be beneficial in facilitating ongoing adaptation modifications and improving design. On the other hand, further feedback from various ACC practitioners’ pertaining to aspects of the app design, implementation, and subsequent health outcomes experienced by patients is warranted to gain constructive insights into the facilitators of, and barriers to, mobile app use among OAT users.

Future studies should integrate more sophisticated smartphone characteristics into the CATA mApp that could further enhance its usability and utility. The introduction of a computing tool to empower patient data entry, and document sharing of patient record/information, could enhance close monitoring of INR and promote effective collaboration between health-care providers and patients to ultimately bridge the gaps in patient adherence and clinical outcomes.

Abbreviations

ACC: anticoagulant clinic

CATA: Coagulation and Anticoagulant Therapy and Awareness

INR: international normalized ratio

M: mean

mApp: mobile applications

mHealth: mobile health

OAT: oral anticoagulation treatment

SD: standard deviation

TTR: time in therapeutic range

Acknowledgements

The authors thank all participants who aided in this study, especially the Information Technology assistants (Hessah Alsaidan, Atheer Altamimi, and Shoq Aalmayouf) for their advice on CATA mApp visual appearance design, and the PNU Pharm D. students (Shahad Alzahrani, Sarah Abutaily, Lujain AlGhwairi, Wejdan AlMadani, and Alhanouf Almoussa) for their input in the development of the electronic usability survey.

Supplementary material

Supplemental data for this article can be accessed here.

Additional information

Funding

This research was funded by the Deanship of Scientific Research at Princess Nourah Bint Abdulrahman University through the Fast-track Research Funding Program.

Notes on contributors

Sireen Abdul Rahim Shilbayeh

Sireen Abdul Rahim Shilbayeh is a professor of clinical pharmacy, Princess Nourah Bint Abdul Rahman University. She is a member of the European Society of Clinical Pharmacy, as well as a member of the International Association of Therapeutic Drug Monitoring and Clinical Toxicology. She is eager to build on a multipurpose and goal-oriented pharmacy career development in various clinical areas including cardiology and nephrology. She published in the areas of psychometric validation of patient-reported outcome measures, pharmacogenetic, population pharmacokinetics, and pharmacy practice. Currently, she is interested in multidisciplinary research projects to enhance the exchange of knowledge and expertise. She can be reached via her email: [email protected].