Developing and evaluating a prototype public health mobile app on the UK NHS Abdominal Aortic Aneurysm Screening Programme

Abstract

Abdominal Aortic Aneurysms (AAA) are asymptomatic with advanced age and male sex being risk factors. Due to their significant mortality rate, the NHS AAA Screening programme was introduced in 2012. However, this is not as well-supported compared to other programmes. When it comes to AAA and its screening, health information is also available from different sources potentially leading to confusion. Based on this, our aim was to develop a prototype mobile application on AAA and its screening, centralising all key information, for the general public. Another aim was to assess the app’s usability and impact (i.e. users’ perceptions about screening attendance and knowledge of AAA). 24 participants completed a pre-app questionnaire followed by app testing and a post-app questionnaire. Ethical approval was granted from the Glasgow School of Art. 75% of participants had never heard of AAA and 92% had never heard of its screening. After app use, the participants’ AAA knowledge significantly increased (Z = −4.318, p < 0.001). App use and opinion of screening attendance were also statistically associated (X¹[1, n = 24] = 6.857, p < 0.05). The app’s usability was rated positively in the USE questionnaire. Research is needed on public health apps regarding their impact on screening uptake and public knowledge.

Keywords:

• mHealth
• app
• screening
• abdominal aortic aneurysm
• communication

Introduction

An aneurysm is defined as a dilation of an artery by at least a 50% widening above its normal diameter (Johnston et al., 1991). Abdominal Aortic Aneurysms (AAA) are often asymptomatic (the patient shows no symptoms until diagnosed by a healthcare professional); a meta-analysis identified that 4.8% of the general population has an asymptomatic AAA (Li, Zhao, Zhang, Duan, & Xin, 2013). Abdominal Aortic Aneurysms occur in 1.3–5% of the male population aged 65–74 in the UK (Wanhainen, 2019). Mortality from spontaneous AAA rupture is significant at 85% (Scott, Bridgewater, & Ashton, 2002). Advanced age is a major risk factor with men being six times more likely to have AAA compared to women. Age 65 is the standard for starting screening considering the prevalence of AAA vs. the risk of rupture (Scott, 2002).

The UK National Health Service (NHS) AAA Screening programme, implemented in 2012, aims to reduce AAA mortality and is projected to decrease deaths by 50% (Cosford, Leng, & Thomas, 2007; Healthcare Improvement Scotland, 2020; Scott et al., 2001). The mortality benefit of the screening programme is maintained up to 13 years and is a long-term cost-effective programme for the NHS with the predicted cost-per-life-year gained at £7600 which is below the guideline threshold of £20 000 per life-year-gained. The cost was £100 greater for those in the screening cohort due to the cost of the screening process and the potential of surgery. Although within the screening cohort there were fewer emergency surgeries and the number of deaths from AAA was reduced (Thompson, Ashton, Gao, Buxton, & Scott, 2012; Thompson, Ashton, Gao, & Scott, 2009). The programme, in the UK, invites men in the year they turn 65 to take part in screening. Ultrasound is used to determine the diameter of an individual’s aorta with the overall process taking <10 min either within a community or a hospital setting. Following the screening, participants are either referred for further surveillance or to the vascular services for intervention or require no further follow-up. In terms of screening, as a whole, certain groups are less likely to attend including individuals with learning and physical disabilities, Gypsy/Traveller communities, and people from a lower socio-economic class (Crilly, Mundie, Bachoo, & Nimmo, 2015; Healthcare Improvement Scotland, 2020; Ross, Scott, & Duncan, 2013). An independent review of screening programmes, commissioned by NHS England, also identified that AAA screening is not as well-supported in regards to research and evaluation of the screening programme with researchers investigating cancer screening programmes having greater access to funding through certain bodies, such as the National Institute for Health Research (Richards, 2019). Although text reminders have been shown to be beneficial for certain cancer screening programmes, these have not been investigated yet for AAA screening (Richards, 2019). It can be argued that increasing awareness of the AAA screening programme’s aims/purpose and providing centralised easy-to-follow electronic information, equally accessible to all potential users, could help increase attendance and in turn lead to early diagnosis of symptomatic AAA, reducing mortality.

The Community Health Index (CHI) system identifies eligible individuals, for the UK NHS AAA Screening programme, and a paper-based invitation letter and information leaflet are issued by Public Health England. These can also be retrieved from the UK government website (Public Health England, 2020). Patient information leaflets lead to either accepting or refusing to take part in the screening programme. However, paper-based patient leaflets have little effect on promoting informed choice, due to the content focussing on the benefits and not providing enough information (Fox, 2006). Additionally, those who engage with leaflets typically have a greater literacy level (Fox, 2006). It has been highlighted that screening programmes should find additional methods, such as web-based decision aids, to relay information on the screening process. Communicating information via alternative ways improves user satisfaction, health behaviour, and health knowledge (Colledge, Car, Donnelly, & Majeed, 2008). Prospective screening participants could also seek information from Internet-based resources, such as NHS web pages. There is a vast array of information available in these, however, not centralised within one website which might be confusing and challenging to accurately interpret especially to the unfamiliar reader.

There has been a surge of research into utilising technology to communicate health information. Owens, Beer, Reyes, and Thomas (2019) noted that more people are using their phones as sources for health information. Recent technological advances have led to the development and use of mobile health (mHealth) applications (apps). mHealth apps that aim to inform patients and the general public about a specific disease were shown to enhance public awareness which can improve public health by promoting healthy living habits (Oluwagbemi, Oluwagbemi, & Ughamadu, 2016). Mahmood, Kedia, Wyant, Ahn, and Bhuyan (2019) noted that among individuals with a chronic medical condition, mHealth apps were associated with increased rates of health-promoting behaviours, such as making health-related decisions and increased patient adherence to taking medication (Pecorelli et al., 2018). Pecorelli et al. (2018) assessed the validity and usability of patient education and self-reporting app for patients undergoing bowel surgery, who had limited health literacy and little or no computer skills/tablet experience. The study found a high level of app usability and 76% of users reported the app increased their motivation to recover from surgery (Pecorelli et al., 2018).

Mobile health is an effective tool for promoting cancer screening and increasing cancer knowledge (Khan et al., 2018; Lee, Koopmeiners, Rhee, Raveis, & Ahluwalia, 2014; Morgan, Laing, McCarthy, McCrate, & Seal, 2015; Rathbone & Prescott, 2017). An app for newly diagnosed patients with early-stage breast cancer increased the participants’ knowledge regarding the risks and benefits of treatment (Morgan et al., 2015). Mobile health apps for cervical screening and colorectal cancer screening have also increased knowledge on the processes and guidelines of screening and prevention approaches (Khan et al., 2018; Lee et al., 2014). Research has shown a positive association between mHealth apps and screening attendance (Lee et al., 2014; Teo, Ng, Lo, Lim, & White, 2019). This may be partly associated with the apps’ ability to reach disadvantaged rural populations (Stotts, Grischkan, & Khungar, 2019). Teo et al. (2019) found that 35% of the population would attend screening earlier than previously intended when using a mobile app.

Multiple benefits of mHealth apps have been identified, such as enhancing patients’ awareness of a disease, increasing adherence to medication, promoting healthy lifestyles and positive behavioural changes, being cost-effective, and revolutionising health information accessibility by reducing geographical and organisational barriers (Mahmood et al., 2019). Although there are mHealth apps for screening programmes, such as breast cancer and prostate cancer, there are currently no such apps for the AAA screening programme. The primary aim of this study was to develop a bespoke mobile app, for the general public, on AAA and the UK NHS AAA screening programme, to centralise key information and consequently collect data on the app’s usability and impact.

Materials and methods

Research questions

The research questions for this study were as follows:

• App’s impact

  •  ^ If, and how, the mobile app influences users' perceptions regarding screening attendance?

  •  ^ If, and how, the mobile app influences users' knowledge of AAA?

• App’s usability

  •  ^ What is the mobile app's usability?

  •  ^ If, and how, the user’s digital literacy influences usability?

Prototype app

The study commenced with the development of the prototype app to centralise all key information (Figure 1). The app contains a mixture of 2D illustrations, 2D animations, and created content. It was built and installed onto an Android device using Unity version 2019.2.9f1, Java JDK, and Android Studio Software Development Kit.

Figure 1. Screenshots from the AAA screening app.

Participant recruitment and user testing

Due to the current COVID-19 worldwide pandemic and social distancing, participant recruitment and app testing were conducted fully remotely and online. Ethical approval, for this study, was granted by the Glasgow School of Art.

The user testing was conducted in three stages as shown in Figure 2.

Figure 2. Process of user testing of the AAA screening app.

Outcomes

Pre-app questionnaire

The pre-app questionnaire had the following sections: general demographic information, digital literacy, AAA knowledge, AAA awareness, and opinion of screening attendance (Supplementary File). The general demographic information gathered data on age, gender and employment. The digital literacy questions gathered information on participants’ experience and use of technology. The scoring system was based on an existing method; each answer was on a graded scale, the scores of which were then totalled to give an overall value of ‘digital literacy’ (Supplementary File) (Lin et al., 2016; Rosen, Whaling, Carrier, Cheever, & Rokkum, 2013). The AAA knowledge questions were five multiple choice questions developed from the content contained within the app. These were informed by other studies investigating the knowledge of patients diagnosed with AAA (Suckow et al., 2016) and a study investigating participant knowledge regarding colorectal screening (Khan et al., 2018). The questions covered the following topics: definition, risk factors, detection methods, and the screening process, four of which were single-choice answers. The fifth question had a possible four correct answers, for each wrong answer chosen a point was subtracted. The maximum score for knowledge was eight. The same AAA knowledge questions were presented in the pre-app and post-app questionnaires. There were two AAA awareness questions that were used to determine if the participants had any prior awareness of AAA and AAA Screening. These were informed and influenced by previous studies investigating the intent to undergo screening (Lee et al., 2014). For both awareness questions, if the participant chose ‘yes’ additional questions appeared asking where they had previously heard of AAA and AAA screening. The attendance question, which was present in the pre-app and post-app questionnaire, was designed to determine if the app influenced the users’ opinion on attending the screening.

Post-app questionnaire

The post-app questionnaire had the following sections: AAA knowledge, opinion of screening attendance, and usability (Supplementary File). An additional attendance question was asked to determine how the app changed the user’s opinion on attendance. To test the app’s usability, the Usefulness, Satisfaction, and Ease of Use (USE) Questionnaire (Lund, 2001) validated questionnaire was used. This consists of 30 statements with a Likert 1–7 scale from strongly disagree to strongly agree, respectively and there is also an option for N/A. The 30 statements each belong to a dimension: usefulness (UU), ease of learning (UL), ease of use (UE), and satisfaction (US). This questionnaire has been used in other studies assessing usability (Filippidis & Tsoukalas, 2009). The final section of the post-app questionnaire consisted of three open-ended questions asking the users’ overall opinion of the application.

Statistical analysis

All data were coded and analysed using SPSS 27.0. Median and inter-quartile ranges (IQR) are presented for the AAA knowledge and digital literacy descriptive statistics. Percentages are presented for the AAA awareness descriptive statistics.

Normality of pre-app knowledge, post-app knowledge, and pre-app and post-app attendance and usability were checked using Shapiro–Wilk’s tests. Non-parametric tests were used as the above did not follow the normal distribution. A paired Wilcoxon test was used to investigate the participants’ knowledge pre-app and post-app. A Chi-Square test was used to investigate the use of the app and participants’ attendance. Due to the small sample size and low expected count, we applied a Yate’s correction followed by a Fisher’s test to ensure the robustness of the results reported (Supplementary File). For the USE questionnaire (Lund, 2001), the scoring was conducted separately for each dimension (UU, UL, UE, and US). The mean and standard deviation is stated, for these, following the analysis of previous studies using this questionnaire (Lund, 2001). If an individual had reported more than 5 ‘N/A’, their data were excluded from the statistical analysis for the USE questionnaire (Gao, Kortum, & Oswald, 2018). Spearman’s Rho Correlation was used to investigate the relationship between the four dimensions of the USE questionnaire (Filippidis & Tsoukalas, 2009; Stoll, Pina, Gary, & Amresh, 2017). Spearman’s Rho Correlation was also used to investigate the relationship between the digital literacy score and the usability score of the participants.

Results

Participant demographics

Of the 30 participants who provided electronic consent, 24 took part in the study. The 24 participants (13 females and 11 males), were aged from 18 to 65. The majority of the participants worked full-time (54%) with 21% working part-time.

Digital literacy

Ninety-six percent of the participants owned a smartphone and 54% of the participants owned three technology devices. Thirty-eight percent of the participants spend 2–3 h per day on their smartphone and 42% often use apps on their smartphone. The median (IQR) for participant digital literacy score was 9 (2.00) (Supplementary File).

Awareness of AAA and NHS AAA screening programme

Ninety-two percent of participants had never heard of the NHS AAA screening programme and 75% had never heard of an AAA. For the six individuals aware of AAA, out of 24 participants, 50% responded with ‘word of mouth’, and the remaining three participants each stated one of the following: ‘job’, ‘family history’, and ‘read about it’ (Figure 3).

Figure 3. Participants’ awareness of AAA and the NHS screening programme prior to the app n = 24.

AAA knowledge

The median (IQR) for the pre-app and post-app knowledge questions was 1 (2.75) and 7 (1.00), respectively (Figure 4).

Figure 4. Mean scores for participants’ knowledge pre-app and post-app.

A Wilcoxon Signed-Rank test showed that after using the app, there was a statistically significant increase in knowledge (Z = −4.318, p < 0.001) (Supplementary File).

AAA screening attendance

Before using the app, 75% of the participants said they would attend the screening or recommend the screening and 25% said they do not know if they would attend or recommend the screening. Post-app use, 100% of the participants said they would attend the screening or recommend the screening and 96% of the participants felt that the app influenced their opinion positively with regards to attending or recommending attendance (Figure 5).

Figure 5. Participants’ opinion on attendance pre-app and post-app use.

A Chi-square test showed the use of the app and opinion of attendance were statistically associated X¹(1, n = 24)= 6.857, p = 0.022. The Chi-square with the Yate’s correction was X¹(1, n = 24)= 4.762, p = 0.029, and the Fisher’s exact p-value was 0.022. The effect size for this finding was moderate with a Cramer’s V, of 0.378.

Usability

The app was rated positively, with the mean scores for each dimension being >5 (out of 7). The average ease of learning score (UL) had the greatest mean 6.76 ± 0.62, ease of use (UE) had a mean score of 6.40 ± 0.97, satisfaction (US) had a mean score of 5.74 ± 1.57 and the average usefulness (UU) had the lowest mean 5.48 ± 1.69 (Figure 6).

Figure 6. Mean scores of the four dimensions of the USE questionnaire.

There were statistically significant correlations between all the categories in pairs except usefulness and ease of learning [r_s (20)= 0.223., p > 0.05] (Table 1).

Table 1. Spearman's rho correlations for the four dimensions of the usability questionnaire.

	UU	UE	UL	US
UU
Correlation coefficient	1.000	0.336	0.223	0.286
Sig. (two-tailed)	.	<0.001	0.079	0.002
N	136	136	63	116
UE
Correlation coefficient	0.336	1.000	0.422	0.333
Sig. (two-tailed)	<0.001	.	<0.001	<0.001
N	136	218	79	140
UL
Correlation coefficient	0.223	0.422	1.000	0.362
Sig. (two-tailed)	0.079	<0.001	.	0.001
N	63	79	79	79
US
Correlation coefficient	0.286	0.333	0.362	1.000
Sig. (two-tailed)	0.002	<0.001	0.001	.
N	116	140	79	140

UU: usefulness; UE: ease of use; UL: ease of learning; US: satisfaction.

Digital literacy and usability

The mean digital literacy score was 8.95 ± 1.79 and the mean usability was 6.09 ± 0.76. Spearman’s correlation revealed no statistically significant correlation between the digital literacy score and usability score [r_s (20) = 0.106., p > 0.05].

Open-ended questions

The post-app questionnaire had three open-ended questions, the responses for the first question ‘what did you like most about the app’, the most common words used were ‘simple (n = 5)’, ‘ease of use (n = 7)’, ‘informative (n = 4)’, and ‘illustrations (n = 7)’. The second question ‘what they disliked most about the app’ responses found ten of the participants stated ‘nothing/N/A’ and other comments included ‘Going back to the menu each time to work through sections’, and ‘Using the 'back' button on an android phone does not take me back to my previous section’. The final question ‘any further comments’, was not compulsory therefore, thirteen participants did not respond. Responses included ‘rolled out to other screening programs’, ‘Narration in the surgery animations would make the tool more accessible’.

Discussion

Previous research has shown that mHealth apps have been successful in increasing the participants’ knowledge and awareness regarding screening programmes (Jongerius, Russo, Mazzocco, & Pravettoni, 2019; Lee, Lee, Gao, & Sadak, 2018). This study aimed to develop the first mobile app (to date) on the AAA and the UK NHS screening programme, providing centralised information, and test its impact and usability.

Awareness

The current study highlighted the need for further general public awareness regarding the NHS AAA screening programme, as only 8% of participants had heard of the screening programmes before using the app. There was a statistically significant association in the opinion of attendance or recommending AAA screening and use of the prototype AAA app. This finding is consistent with previous research showing a positive association between mHealth apps and screening attendance (Lee et al., 2014; Teo et al., 2019). Studies have also discussed the potential of mHealth apps to promote positive health behaviours, such as attending a screening programme (Mahmood et al., 2019; Pecorelli et al., 2018). This is reflected in the finding that 96% of participants stated this app influenced their opinion positively towards screening attendance or recommending it. Studies on the tone of language used within public health apps have suggested that stating negative consequences could lead the user to actively seek screening (Owens et al., 2019). The AAA app stated both negative consequences and positive consequences including the chance of death with a ruptured/rupture AAA. The tone of language used may have also influenced the high percentage of participants willing to attend a screening or recommend it.

Knowledge

In this study, there was a statistically significant increase in knowledge scores, after use of the app, which suggests the app successfully informed the participants about AAA and the NHS AAA screening programme. This finding is similar to existing mobile apps discussing cancer screening (Khan et al., 2018; Lee et al., 2014; Morgan et al., 2015). The statistically significant increase in knowledge could be attributed to the combined use of multimedia, 2D illustrations, animations, text, and videos to present the information on AAA. Research suggests that presenting information in multiple formats increases comprehension and subsequently increases knowledge retention with the varying multimedia supporting the various learning styles of the users (Green et al., 2004). The statistically significant increase in knowledge could also be attributed to the use of a reading level suitable for the general public with easy-to-follow language that was used throughout the app; this was evident by the user feedback ‘informative’, ‘right level for me to understand’, and ‘information was very easy to understand’. Although, one individual suggested that the ‘language is a bit simple’. Participants’ education was not investigated, so it is unclear whether the level of education had an influence on their opinion of the level of language used.

Usability

Analysis of the usability of the application highlighted that the app was rated positively. The participants found the app to be useful, easy to use, easy to learn, and satisfying. These findings are similar to those reported for usability tests of mHealth apps on REACH, which is an app on anxiety in youth, a bowel surgery app, and ScreenMen app (Pecorelli et al., 2018; Stoll et al., 2017; Teo et al., 2019). For this AAA screening app, the results present that ease of learning had the greatest score and usefulness had the lowest score. Further development and evaluation are needed to improve and assess the usefulness of this app, respectively. A focus group interview could be carried out with members of the public to develop the app. Further analysis of the usability results showed that there were positive correlations between all dimensions except for usefulness and ease of learning. This finding does not correspond to the literature, as conceptual models on mHealth technologies within the literature suggest that learn-ability (ease of learning) and efficiency of the app (ease of use) are essential to increase user satisfaction and usability (Stoll et al., 2017). However, one reason for this finding may be that the sample was small (n = 20) and not sufficiently diverse. Further research should be conducted to establish and investigate the association between usefulness and ease of learning.

Digital literacy

In regard to the research question ‘If, and how, the digital literacy influences usability’, a non-statistically significant finding was noted. This does not correspond to the literature. Reduced digital literacy, as a result of a lack of experience and technical skill, can form a barrier to engagement with digital health intervention (O’Connor et al., 2016). A pilot study assessing the usability of a swallowing training mHealth app considered the variables; participants’ education and device usage. The pilot study found the low-potential group (reduced device usage/experience) had a mean usability score in the ‘low marginal acceptability’ threshold. In comparison, the high-potential group (greater device usage/experience) had a higher usability score; ‘acceptable’ threshold (Kim et al., 2020). Although it should be noted this study used an alternative validated usability questionnaire; the System Usability Scale (SUS) and the different findings may have been due to the lack of a validated questionnaire on digital literacy.

Limitations

The relatively low sample size meant that the power of the study was reduced, and caution should be applied when generalising the results, due to the potential over-estimation of the magnitude of association. The participants’ education was not assessed, which might have influenced the results. An observed trend within previous research reports individuals with higher education have higher levels of health literacy and are more likely to use new technologies (Cho, Park, & Lee, 2014). The questionnaires contained bespoke questions for the knowledge section. These could be validated by specialists to ensure they test the correct level of understanding required for patients undergoing the NHS AAA screening programme. Additionally, users completed the post-app questionnaire immediately after using the app, therefore, testing the immediate knowledge effect and not long-term retention. Due to the lack of AAA screening knowledge, it is unlikely that members of the public will search the app stores to find an app on AAA screening. Additionally, as the app is built only for Androids its distribution may be more difficult in comparison to the paper-based information leaflets sent in the post.

Future avenues

To further analyse the usability of the app, user interaction data could be collected and examined; for example, the click-tracking sequence, time spent on each canvas, and overall completion time. This could provide insight into key areas for further app development. The app could also be developed for other operating systems, such as iOS with the aim of increasing accessibility. Future analysis should compare the app with the current paper-based leaflets to determine if there is similar knowledge gained between the two methods, and evaluation of the app should determine whether it offers an improvement compared to the current means of communication. The impact of the app relies on the extent of its reach. Future research would need to consider how the app can be disseminated among the general public, such as advertisement, recommendation of the app from healthcare professionals, or referral of the app on patient letters sent from the screening programme (Teo et al., 2019).

Conclusion

A novel prototype AAA app was developed for this study, which was shown to increase the awareness, knowledge and positively influence the opinion of attendance for the NHS AAA screening programme. Further research is needed to investigate whether the uptake of AAA screening and public awareness and knowledge can be improved, through the use of a public health app, specifically within groups who are less likely to attend the screening.

Acknowledgements

The authors would like to thank all the participants for their contribution to this study. The authors would also like to acknowledge that this research work has been previously discussed in Ella Jones’ MSc thesis entitled ‘A Public Health Mobile App on the UK Abdominal Aortic Aneurysm Screening Programme’ that was submitted in partial fulfillment of the MSc Medical Visualisation and Human Anatomy degree in 2020.

Disclosure statement

The authors report no conflict of interest.