Old age or cognitive decline? Examining the usability of a mobile health app for older Australians

ABSTRACT

There is a growing literature on the role of mobile health applications (mHealth apps) in supporting older adults and the self-management of personal health. The purpose of this pilot study is to assess the usability of a government-funded mobile health app amongst older Australians and to evaluate whether cognitive function and demographic characteristics (i.e. age, gender, education) are associated with usability. A total of 28 older adults living in a regional city in Australia took part in the study. The participants were recruited using purposive sampling. Data collection instruments consisted of validated cognitive tests, task-based usability tests, and a questionnaire. The data was analyzed using non-parametric strategies. The findings of this study demonstrated that a government-funded, mHealth app was usable by older adults. Users were able to perform basic tasks in an effective and efficient manner. The hypothesis that elderly age would be significantly associated with performance on cognitive tests, as well as usability, was not supported. Performance on some cognitive tests was significantly and positively related to usability. Education and gender were not related to usability. The results suggest that traditional stereotypes surrounding aging and cognitive decline need to be reexamined.

KEYWORDS:

• Cognitive function
• older adults
• usability
• mHealth apps

Introduction

As people age, the risk of dying from four main diseases, cardiovascular disease, cancer, diabetes and chronic respiratory disease, increases. Modern lifestyles contribute to the risk of contracting these diseases, which share common risk factors such as smoking, poor diet and physical inactivity.¹ Consequently, scholars and practitioners are increasingly interested in the factors driving the adoption of mobile health applications (mHealth apps) since they help prevent disease, improve quality of life and reduce health care costs.^2–7 Use of mHealth apps has increased considerably, particularly in the light of the COVID-19 outbreak in China,⁸ and around the world, since people were forced to adopt digital technologies in order to deal with the restrictions posed by the pandemic.⁹

The principle of “inclusive design” recognizes that good design has to accommodate the needs of a diverse set of users.¹⁰ There is a growing body of literature examining the usability of mHealth apps for older adults¹¹ and numerous usability frameworks and guidelines exist to assist with the design of senior-friendly apps.¹² The barriers to mHealth adoption with respect to older age groups are broad in scope and include cognitive decline.¹³ Yet there is a scarcity of literature that objectively measures cognitive function in usability studies, with a few exceptions.^14,15 Therefore, gaining insights into the role of cognitive aging in influencing usability remains important.

The aim of this pilot study is to assess the usability of a government-funded mHealth app amongst older Australians and evaluate whether cognitive functioning and demographic characteristics (i.e., age, gender, education) are associated with usability. This study contributes to the literature and to practice in several ways:

1. This study adds to the scarce literature that reports the relationship between cognitive aging and the usability of mHealth apps, using validated tests of cognitive functioning. While some studies measure age-related decline in older adults using objective tests of cognitive decline,^14,15 the focus is often on evaluating specialized medical apps or new technologies for use in clinical settings, which are aimed at older people who are frail, and experiencing physical or mental impairments.^16–18 This study focuses on healthy, older adults and the usability of a mainstream mHealth app in a non-clinical, community setting.

2. Despite a solid body of evidence supporting the idea that mHealth apps for older adults have to be designed with usability in mind,^19–22 relatively few studies have investigated older adults in Australia. For instance, one study focused on tertiary-educated Australians and their experiences with using mHealth apps as a tool to facilitate self-care.²³ This study extends the scope of the literature by focusing on older adults, with diverse educational backgrounds, in regional Australia.

3. Demographic factors are commonly studied in usability studies,^24,25 yet findings are often mixed and inconclusive. By correlating demographic factors (i.e. age, gender, education) with usability, the study contributes to the well-established literature on the role of demographic variables in influencing the usability of mHealth apps. As noted by Buss et al. (2022),¹⁸ inclusivity and equity must be considered when using mHealth apps if the goal is to reach all people that could potentially benefit from a mHealth intervention.

4. On a practical level, user feedback on the mHealth app should have value for app designers and healthcare professionals.

The outline of this paper is as follows: we briefly review the literature on mHealth apps and focus on usability and age-related barriers to usability. This is followed by the formulation of the hypotheses and their justification with prior research. Subsequently, before presenting the research findings, the methodology and sampling procedures are described. Lastly, discussion, implications for practice, together with limitations, are laid down accordingly.

Literature review

The market for mHealth apps has grown rapidly and apps range from simple, health and fitness apps to more complex apps, with diverse functionality, oriented toward the self-management of chronic conditions.² mHealth apps are seen as a mechanism to improve the health and quality of life of older adults, and positive clinical outcomes have been reported.²⁶ App-based interventions result in better lifestyle choices^27,28 assist with the self-monitoring of symptoms⁶ help improve blood pressure and medication adherence^25–29 help reduce social isolation³⁰ assist older adults at risk of falls³¹ and support independent living.³² They also hold promise for patients who are less likely to attend in-person mental health services, particularly since engagement in mHealth does not require the time, resources, or effort that enable travel to “bricks and mortar” clinics.³³ As the recommendation of mHealth apps by medical professionals is expected to grow, growing attention is being paid to the issue of usability for older adults.

Usability is defined by the International Organization for Standardization (ISO) as “the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.”³⁴ Poor usability can reduce the motivation to use apps, particularly if benefits do not manifest themselves quickly and easily. Tracking data shows that users typically spend only 30 seconds learning how to use an app. If an app is not easy to use, people quickly abandon it or simply give up using mobile apps altogether.¹⁰ The concept of inclusive design recognizes that good design must cater for a range of physical and cognitive abilities, and be easy to use for people of all ages.³⁵ In recent years, several systematic reviews of usability requirements for older adults have been undertaken.^29,31,36,37 Scholars have reported usability findings for apps targeting a wide range of health conditions, including diabetes,³⁸ COVID-19 management,³⁹ heart failure and medication reminders.⁴⁰ It is concluded that older adults have unique requirements and aging may affect their physical capabilities and cognitive functions.⁴¹ For instance, reduced dexterity can make it difficult for some people to click small buttons,⁴² and eyesight decline can make it challenging for older adults to read graphs and text on a small screen.⁴³ There are numerous rating tools and check-lists available to help experts assess design parameters and ensure that the apps meet the needs of elderly individuals.^11–13,41 App designers need to consider a wide range of factors that affect usability, including navigation, volume of information, and design elements that cater for visual impairment (i.e. color contrast, size of font, buttons and icons).^11,13,42 The present study adds to the literature on usability by undertaking a controlled usability test of older adults and gaining user feedback on a government-funded mHealth app.

Cognitive function and age-related barriers to usability

Numerous studies show that increasing age is associated with lower levels of cognitive performance.^44,45 Research shows that crystallized cognitive capabilities, such as reading, comprehension, math, science and vocabulary, improves until the age of 60 years, and then declines at around 80 years.⁴⁶ However, when it comes to fluid cognitive capabilities, such as solving a puzzle or any other problem solving strategies, there is steady decline from the age of 20 to 80.⁴⁷ Aging is associated with cognitive impairment in two main domains: episodic memory (i.e. a type of long-term memory that involves recollection of previous experiences together with their contexts in terms of time, place and associated emotions) and psychomotor speed (i.e. processing speed).⁴⁷

There is a lack of consensus in relation to the age at which cognitive decline begins, as well as the degree of impairment associated with aging. Salthouse (2009)⁴⁸ concludes that cognitive decline starts after adult maturity and clearly occurs prior to the age of 60. Studies show that the differences in cognitive impairment between the youngest (60 plus) and oldest groups (80 plus) are substantial, depending on the test used.⁴⁹ It is acknowledged that the extent of cognitive impairment will depend on several factors. Variability in cognitive function is affected by genetics, cognitive reserve (which refers to the ability to withstand pathologic changes to the brain), educational experiences during childhood, socio-economic status, social stressors, baseline intelligence and knowledge of new technology.^45,50 Harrington et al. (2017)⁴⁹ conclude that age-related cognitive decline has been overestimated in the literature, and the effect of age on most aspects of cognition is small.

Cognitive decline is consistently seen as a barrier in the literature on the usability of mHealth apps for older adults.^{11,13,13,37,40,51} Cognitive changes, such as processing speed, may negatively influence interactions with apps.⁵² Due to cognitive decline, older users may need more time to learn how to use mHealth apps,⁸ and according to Wang et al. (2022),¹¹ app designers need to provide reminders to cater for the deterioration in memory and poor concentration. Chronic health conditions can exacerbate cognitive decline²⁴ and affect the ability of older adults to use mHealth apps.⁸ Patel et al. (2022)¹⁸ considered potential inequities to mHealth access in a US study. Using the Cognitive Abilities Screening Instrument to assess cognitive ability, it was found that older age and lower cognitive ability decreased the odds of having access to mHealth.¹⁸ Likewise, a study by Arlati et al. (2021)¹⁵ of the acceptance and usability of virtual reality among older adults (with mild cognitive impairment) found that cognitive deficits led to more errors in using the technology.

Based on the above studies on cognitive aging, the following hypotheses are formulated:

H1:

Older age is positively correlated with mild cognitive impairment.

H2:

Mild cognitive impairment is negatively correlated with the usability of a mobile health app.

Demographics and usability of mHealth apps

A recent literature review²⁵ shows that many factors influence the acceptance of mHealth apps, including demographic characteristics, social attributes, source credibility, legal issues as well as the application design dimension (i.e., functionality, perceived ease of use and usefulness, security, and cost).

Age

Several studies report age-related barriers to the usability of mHealth apps. For example, Peute et al. (2022)⁴¹ and Wildenbos et al. (2018)¹³ concluded that older adults are likely to experience usability issues with mHealth apps due to age-related physical and cognitive decline. Other studies on the acceptance and adoption of apps report similar findings. In a recent study, Heponiemi et al. (2022)⁵³ reported that, when compared to others, older adults were less inclined to use online, health-related services (i.e., receiving test results, renewing prescriptions and scheduling appointments). Patel et al. (2022)¹⁴ found that older adults were less likely than younger people to own fitness trackers.

Based on the afore-mentioned studies and the literature on cognitive aging, the following hypotheses are proposed:

H3a:

Older age is negatively correlated with the usability of a mobile health app.

H3b:

Significant differences exist between younger and older age groups in relation to the usability of a mobile health app.

Gender

Gender is often explored in relation to the adoption of mHealth apps, rather than with usability. Studies report mixed outcomes for the relationship between gender and the adoption of mHealth apps. A recent report concluded that gender plays a limited role in explaining differences in technological use in most countries. In both advanced and emerging economies, men and women generally use technology (i.e. smartphones, the internet and social media) at similar rates.⁵⁴ There is some early scholarly research that regards gender as an important indicator of mHealth acceptance.⁵⁵ A large scale survey by Carroll et al. (2017)⁵⁶ using a U.S. nationally representative sample, found that being male, older, and having less than high school education, were all significantly associated with a reduced likelihood of adopting mHealth apps. However, a recent study found that gender, along with educational level, were non-significant moderators of the intention to use mobile apps.⁵⁷ Yet another study found that gender was not associated with aggregated mHealth use.⁵⁸

Hence, we hypothesize that:

H4:

There are no significant gender differences in relation to the usability of a mobile health app.

Education

Education is consistently associated with the adoption and the usability of mHealth apps. Prior research reported that users of mHealth apps were generally more highly educated (and younger) than non-users.⁵⁸ Morano et al. (2019)⁵⁹ found that users of mHealth tools were significantly more likely to have an educational standard higher than high school education. Other studies have found that certain groups in the US, including less educated and socially disadvantaged groups, are less likely to be able, or willing to, receive health information provided through online channels alone.⁶⁰ Low levels of educational attainment have been linked to dissatisfaction with web-based applications.⁶¹ Hence, we hypothesize that:

H5:

There are significant differences in relation to levels of educational attainment and the usability of a mobile health app.

Materials and methods

The following section presents the aims, research questions, research methods and sampling procedures.

Aim of the study and research questions

The purpose of this exploratory, pilot study is to assess the usability of a government-funded mHealth app amongst older Australians and evaluate whether cognitive functioning and demographic characteristics (i.e., age, gender, education) are associated with usability. Our primary research questions are as follows:

1. Is older age positively correlated with mild cognitive impairment?

2. Is mild cognitive impairment negatively correlated with the usability of a mobile health app?

3. Are demographic factors such as age, gender and education related to the usability of a mobile health app?

4. What feedback can users provide to help improve the design of a mobile health app targeted at older adults?

Selected mHealth app

The selected app comes from an Australian government-funded health service, healthdirect (https://www.healthdirect.gov.au/) and it falls into the category of “disease-related education apps.”² This app was chosen since it is a free app; it is government-funded, is evidence-based and validated according to health literacy standards. A mainstream health app rather than a specialized medical app was chosen since the app was aimed at the targeted demographic, i.e., healthy, independent adults living in Australia. Due to time and cost constraints, only one app was chosen for usability testing.

Participants

Older adults in Townsville, a regional city in Queensland, were recruited for the study through purposive sampling. Recruitment strategies included advertising in a university newsletter, advertising in a newsletter sent to a community group, the University of the Third Age (UTA), as well as recruitment through an event, the Seniors Lifestyle Exposition, organized by the local council, to access people from lower socio-economic groups. The final sample size was 28.

Participant inclusion criteria were as follows: age ≥55 years (in order to capture older adults who were in the workforce, close to retirement age and retirees) and fluency in written and spoken English. Participants were not required to be users of mHealth apps but were expected to have some familiarity with smartphones and apps in general. Participant exclusion criteria were as follows: any illness that would limit the ability to give informed consent. The study was approved by the Human Research Ethics Committee in the authors’ university.

Measures, tools and procedures

The participants were asked to perform a variety of tasks such as cognitive tests and task-based usability tests and complete a questionnaire.

Tests took place at the first author’s office, at a booth in a Seniors Exposition, a local library and in the participant’s home. The settings, although varied, were carefully chosen to be comfortable, well-lit and free of distractions. Each session lasted approximately 20–30 minutes. The researcher introduced the session, obtained informed consent, and collected the demographic data. Then the cognitive tests were administered (the Mini-Cog Test and the Trail Making Test A and B). The user was then directed to the chosen app and was asked to complete three usability tasks. Following this, the participants were asked to complete a questionnaire which captured user feedback of the app. At the end of the session, participants were thanked for their participation and were given shopping vouchers as a token of appreciation.

Cognitive tests

To measure mild cognitive deterioration, the Mini-Cog test and the Trail Making Test A and B were used.

The Mini-Cog test is a brief and validated instrument that can increase the detection of mild cognitive decline and dementia in older adults.⁶² The Mini-Cog test takes only three to five minutes to administer and is suitable for use in community settings. It consists of two components, a three-word recall task that assesses memory and a clock drawing test that assesses cognitive domains such as cognitive function, language, visual-motor skills and executive function.⁵⁰ The scoring of the Mini Cog test involves assigning a score of 0 to 3 points on the word recall task for the correct recall of the three words. The clock drawing test is scored from 0–2, with 0 points for an abnormal clock drawing and 2 points for a normal clock. The 3-item recall and clock drawing scores are then added together. A total score of 0, 1, or 2 out of 5 indicates a higher likelihood of cognitive impairment. A total score of 3, 4, or 5 indicates a lower likelihood of cognitive impairment.⁶³

The Trail Making Tests (TMT) A and B were also used.⁶⁴ TMT is a test for cognitive impairment. It makes good use of working memory, which refers to the ability to momentarily hold information in memory while simultaneously manipulating that information.⁵⁰ TMT-A requires an individual to draw lines sequentially connecting 25 encircled numbers. Task requirements are similar in the case of TMT-B except the person must alternate between numbers and letters (e.g., it is an alphanumerical test). TMT-A and TMT-B test processing speed, sequencing, mental flexibility and visual-motor skills. The tests were scored based on the following criteria – whether participants could complete the tasks, and task completion time which was measured in seconds.

Task-based usability tests

Task-based usability tests were employed in order to observe and record the objective performance of the target users when interacting with the app.⁶⁵ Performance was recorded via usability metrics, such as task duration (measured in seconds) and completion rate, which is common in the literature.³¹ The researchers also recorded the number of cues or prompts needed by the participants to complete each task.

The usability tasks reflected a functional concept of health literacy, and the participants were asked to compete three tasks, as follows:

(1) Find the advice given on abdominal pain.

(2) Find the nearest GP to your location.

(3) Look for the details on Ketorolac medicine.

Questionnaire

Participants were asked to complete a short questionnaire which consisted of three components, the system usability scale, the net promoter score and demographic data.

Using the System Usability Scale (SUS), participants were asked to rate the usability of the mobile app using a 10-item survey, with 5-point Likert scales, that captures the overall usability of a product. Reporting takes the form of a score from 0 to 100, and according to Brooke (1996),⁶⁶ a score of 68 indicating average usability, and higher scores indicate better system usability. The SUS has been validated in multiple studies.

The Net Promoter Score (NPS) was also used. The NPS has a rating from 0–10 and is seen as a valid measure of the customer’s experience with a product or service.⁶⁷ The questionnaire also had open-ended questions capturing the participants’ subjective opinions on the mHealth app, such as the most valuable feature of the app, ways to improve the design of the app, and the usefulness of the app in improving the user’s (health-related) quality of life.

The demographic section of the survey captured information on the gender, age, educational level and profession of the participants.

Data analysis

Data were analyzed using the Statistical Package for the Social Sciences (SPSS), version 25 (IBM Corp). Prior to running the analysis, the data was checked for any missing values. The data was also checked for assumption violations. Since the assumptions of normality and linearity were violated, we decided to analyze the data using non-parametric statistical tests. The open-ended questions in the questionnaire were analyzed with the conventional content analysis method, and according to Hsieh and Shannon (2005),⁶⁸ this occurs when coding categories are derived directly from the text data (and coding refers to the process used to organize large quantities of text into smaller categories)⁶⁹.

Results

Sample demographics

The data was collected from 28 participants. The average age of participants was 73 years (S.D. = 6.80). A total of 11 males (Mean age = 73.83, SD = 4.8) and 17 females (Mean age = 72, SD = 7.9) participated in the study. Most respondents were educated beyond secondary school level and had white-collar jobs. Educational level was broken down into three levels: secondary school/year 12 level (n = 7), post-secondary level, such as having a diploma, advanced diploma or a bachelor’s degree (n = 17), and a post graduate degree (n = 4). Therefore, the sample captured a diverse group in terms of education. Table 1 summarize the sample’s profile.

Table 1. Profile of sample (n = 28).

Characteristics	Number, %
Gender
Male	11 (39%)
Female	17 (61%)
Education
Secondary	7 (25%)
Bachelor’s degree, diploma, advanced diploma	17 (61%)
Post-graduate degree (Masters, Phd)	4 (14%)
Age range
55 to 60	2 (7%)
61 to 70	7 (25%)
71 to 80	16 (57%)
81 to 85	3 (11%)
Occupation (prior to retirement)
Health sector (nursing, dentist, doctor, aged care)	5
Not stated	3
Engineering (software, civil) and geology	3
Management, clerical and administration (consultant, librarian, payroll, book-keeper, telephonist)_	10
Education (teacher, lecturer, childcare)	4
Science (botanist, marine scientist)	2
Personal services (massage therapist)	1

Results of cognitive and usability tests: descriptive statistics

All participants completed the clock drawing test easily and efficiently. In relation to the total score for the Mini-Cog test, the mean value was 4.7, indicating that the respondents had a normal score. For TMT-A and TMT-B, there was considerable variation in “time-to-task” but both tests were completed.

In relation to the usability tests, the three navigation tasks on the mHealth app were successfully completed by the respondents, but there was wide variation in terms of “time-to-task.” The qualitative data show that respondents rated the app as usable and were willing to recommend it to friends, with a mean score of 76 out of 100 for the System Usability Score (SUS) and 7 out of 10 for the Net Promoter Score (NPS).

Table 2 shows the mean, median and standard deviation for all variables.

Table 2. Mean, median and standard deviations: scores for cognitive tests and usability.

	Cognitive tests			Task-based usability tests			Usability scores
Variables	Mini-Cog	TMT-A	TMT-B	TTT1	TTT2	TTT3	SUS	NPS
Mean	4.71	37.36	85.71	94.32	82.46	51.82	76.339	7.32
Median	5.00	37.00	73.50	78.50	73.00	43.50	77.50	8.00
SD	.53	14.09	53.65	71.74	58.10	29.29	14.96	2.65

TMT-A = Trail making test A; TMT-B = Trail making test B; TTT1 = Time to Task 1; TTT1 = Time to Task 2; TTT1 = Time to Task 3; SUS = System usability scale; NPS = Net promoter score. TMT-A, TMT-B and app navigation tasks are scored in seconds; the SUS score is out of 100; the NPS score is out of 10.

Age, mild cognitive impairment and usability

In order to address hypothesis 1 (i.e., older age is positively correlated with mild cognitive impairment), a Spearman’s Rank Order correlation was run. Table 3 summarises the results and the data showed that elderly age was not significantly related to worsening performance on cognitive tests.

Table 3. Intercorrelation matrix for variables, age, cognitive tests and usability (Spearman’s rank order correlation).

Variables	1	2	3	4	5	6	7	8	9
1. Mini-Cog	-
2. Age	−.24	-
3. TMT_A	−.33	.27	-
4. TMT_B	−.42^*	.14	.63	-
5. TTT1	−.01	.18	.37	.25	-
6. TTT2	−.22	.25	.44^*	.54**	.09	-
7. TTT3	−.23	−.24	.12	.29	.25	.09	-
8. SUS	.51	−.24	−.25	−.38^*	.08	−.49**	−.35	-
9. NPS	.22	−.33	−.33	−.18	.20	−.41^*	.10	.70	-

TMT_A = Trail making test A; TMT_B = Trail making test B; TTT1 = Time to Task 1; TTT1 = Time to Task 2; TTT1 = Time to Task 3; SUS = System usability scale; nps = Net promoter score. TMT-A, TMT-B and Navigation tasks are scored in seconds; the SUS score is out of 100; the NPS score is out of 10.

Correlations between cognition and usability that are significant are displayed in bold text.

*p < .05, **p < .01.

The hypothesis examining the relationship between mild cognitive impairment and usability was partially confirmed. Out of the three tests for objective usability performance (i.e., based on direct measures of usability that are specific enough to identify problems), the results were non-significant for task 1 and task 3. There were interesting findings for task 2. In relation to objective usability, participants performing well on TMT A (cognitive test) showed moderate but significant efficiency in navigating task 2, which was finding the nearest GP, (rs(28) = .44, p = .020). A similar result was observed for performance on TMT B (cognitive test), which was significantly correlated with faster navigation of Task 2 (usability) (rs (28) = .54, p = .003). Fast performance on Task 2 was also associated with the participants’ usability scores (rs(28) = −.49, p = .008) and Net Promoter Score (rs(28) = −.41, p = .031). In relation to subjective usability (i.e., based on indirect measures of usability such as the general opinions of respondents captured in a survey), efficient performance on TMT B (cognitive test) was significantly correlated to the System Usability Score (rs (28) = −.38, p = .04).

Age groups, mild cognitive impairment and usability

The participants were allocated to two groups, older and younger using the median split (see Table 4). A Mann Whitney U test was computed to see if the two groups differed in their scores for the cognitive tests, the navigation tasks as well as the SUS and NPS. The results showed that though participants in the younger age group had a higher mean rank on Mini Cog, faster time on TMT A and B, and Tasks 1 and 2 and higher SUS and NPS, the difference was not significant. The analysis indicates that age was a not a significant factor affecting mild cognitive impairment and usability.

Table 4. Cognitive tests and usability: analysis by age category (Mann Whitney).

		Cognitive tests (objective)			Task-based usability tests (objective)			Usability scores (subjective)
Age category		Mini-cog	TMT-A	TMT-B	TTT1	TTT2	TTT3	SUS	NPS
Younger (>75)	Mean Rank	16.07	12.50	11.93	12.25	13.32	15.18	16.93	16.29
Older (>75)	Mean Rank	12.93	16.50	17.07	16.75	15.68	13.82	12.07	12.71

TMT_A = Trail making test A; TMT_B = Trail making test B; TTT1 = Time to Task 1; TTT1 = Time to Task 2; TTT1 = Time to Task 3; SUS = System usability scale; NPS = Net promoter score.

Gender and usability

A Mann Whitney U test was run to assess if there were differences between males and females in relation to the usability scores. The results are depicted in Table 5. The results showed that for navigation task 1 and 2, males took a longer time than females and for navigation task 3, females were slower than males, but the difference was not significant. Males ranked the app slightly higher than females in terms of usability and gave a higher Net Promoter Score. The results therefore confirmed hypothesis 4, that gender was insignificant.

Table 5. Usability: analysis by gender (Mann Whitney).

		Task-based usability tests (objective)			Usability scores (subjective)
Gender		TTT1	TTT2	TTT3	SUS	NPS
Males	Mean Rank	16.95	14.77	14.14	16.41	15.59
Females	Mean Rank	12.91	14.32	14.74	13.26	13.79

TMT_A = Trail making test A; TMT _B = Trail making test B; TTT1 = Time to Task 1; TTT1 = Time to Task 2; TTT1 = Time to Tas (Mannk 3; SUS = System usability scale; NPS = Net promoter score.

Education and usability

In order to test for differences in usability by educational level, participants were categorized into three groups: secondary level, degree level and post-graduate level. A Kruskal-Wallis H test showed that (see Table 6) there was not a statistically significant difference in the usability scores of the three groups. The best scores and best timing were shown by participants with the highest educational qualifications, but the difference was not statistically significant. The hypothesis that levels of educational attainment are significantly related to usability was not confirmed.

Table 6. Usability: analysis by education level (kruskal-wallis).

	Usability tests (objective)			Usability scores (subjective)
	TTT1	TTT2	TTT3	SUS	NPS
Median	78.50	73.00	43.50	77.500	8.00
Chi-Square	5.61	.20	4.67	2.21	3.05
Asymp. Sig.	.060	.904	.097	.331	.217

TMT_A = Trail making test A; TMT _B = Trail making test B; TTT1 = Time to Task 1; TTT1 = Time to Task 2; TTT1 = Time to Task 3; SUS = System usability scale; NPS = Net promoter score.

Users’ feedback on the mHealth app

The majority of the participants (n = 23) responded positively to the app, and the main benefit of the app was access to reliable information. When asked about the most valuable feature of the app, respondents commented on ease of use (n = 11) and access to credible information (n = 10), such as the ability to check symptoms, medicines and health topics. Other comments were that it would be useful for people at home alone (n = 1); good for emergencies (n = 1); it is a self-help tool (n = 1); it saves time (n = 1); it is transportable (n = 1), and has a touch screen which is good for people with arthritic fingers (n = 1). A few respondents (n = 5) gave negative or neutral evaluations of the mHealth app, citing the risk of self-diagnosis and anxiety about medical conditions.

User feedback on how to improve the design of the mHealth app is shown in Figure 1. The participants mentioned visual design elements such as font size, a clearer search box, more visual cues, along with voice recognition and predictive spelling. The large volume of information was seen as a negative factor, along with the technical nature of information. Two people suggested that marketing was needed to increase public awareness of the app and training on how to use a mHealth app would be valued.

Figure 1. Recommendations for improving the app.

Discussion

The purpose of this pilot study is to assess the usability of a government-funded mHealth app amongst older (healthy) Australians and evaluate whether cognitive functioning and demographic characteristics (i.e., age, gender, education) are associated with usability.

The results of the mini cog and TMT tests showed that all of the participants achieved scores that were in a normal range, suggesting the absence of mild cognitive impairment or dementia in the community sample.⁶³ There was wide variation in the results of the TMT-A and TMT-B tests, which may be due to the relative complexity of the TMT-B task and the amount of information processing required. The hypothesis that elderly age would be associated with worsening cognitive performance was not supported. Although there were differences in test scores between the younger (under 75) and the elderly cohort (75+), and differences were more pronounced on the Trial Making Test B (which was a more difficult test than TMT-A), the result was not statistically significant. This finding is not consistent with findings on cognitive aging in the fields of gerontology and neuro-cognitive science.⁴⁸ However, the target group of this study was healthy, autonomous older adults and not adults with preexisting cognitive impairment or chronic disease, and we conclude that older adults can successfully use mHealth apps if they are well-designed.

The findings revealed a wide variation in the time taken to perform the usability tests, and this is not surprising since the tasks varied in their complexity. The findings show that performance on cognitive tests was significantly and positively related to higher usability, although results were restricted to performance on one app navigation task, and not all three tasks. The hypothesis that mild cognitive impairment is related to usability was partially confirmed. This finding is aligned with prior research.^13,14 For instance, Patel et al., (2022) reported higher odds of engagement with mHealth with higher cognitive ability.¹⁴ Furthermore, the finding supports the concept of inclusive design and need for app designers to cater for a range of cognitive abilities.¹⁴

Our study did not find a significant relationship between aging and declining levels of usability, which is not aligned with the literature. Several studies report age-related barriers to app usage and aging is generally seen as a negative factor in technology acceptance studies.^14,29,51 However, age is just one of the traditional risk factors for cognitive decline³² and many adults retain high levels of cognitive function in older age.⁵² Our results are compatible with emerging studies that highlight that age-related cognitive decline has been overestimated in the literature.⁴⁹ Scholars argue that studies overlook complex relationships between cognitive skills and decline.⁵³ For instance, work on “cognitive super-agers” shows that above-average memory capacity can be encountered in advanced age. It is suggested that genetics may play a role: superior performers resist age-related changes or have simply started from a much higher baseline. Scholars are now questioning “ … the belief that a gradual loss of intellectual ability is part of ‘normal’ aging. It turns out that this disarmingly simple assumption is extremely difficult to substantiate or refute.”⁷⁰ Various scholars highlight the ongoing negative narrative about aging, where older people are portrayed as suffering from declining levels of cognitive function, which is a potentially harmful stereotype.⁷¹

Our study did not find a statistically significant relationship between education and usability. It must be noted that participants with a secondary level qualification took slightly longer to complete the trail making tests, and navigation tasks 2 and 3, compared with those with a higher level of education. Their scores for the System Usability Scale (SUS) were also lower that those with a higher level of education. Prior studies highlight that users of mHealth apps are generally more highly educated than non-users.^58,59 In a systematic review undertaken by Jacob et al. (2022),⁴ the authors found that education is a prominent demographic factor which explains willingness to use mHealth tools (often explained by access to, and skills in using, technology, and higher eHealth literacy). Prior research also shows that education is related to better cognitive performance on the Trail Making Rest.⁷² Our finding is surprising, but some scholars propose that the evidence for a digital divide in mobile interventions due to socio-demographics is inconclusive.²⁴ Furthermore, Carroll et al. (2017)²⁷ concluded that while differences in engagement with mHealth persist due to educational attainment (as well as age and gender), these obstacles are becoming less potent.

The hypothesis that gender would be unrelated to usability was confirmed. Females did perform better than males on several tests, but gender was not statistically significant. This finding finds some support in the literature on technology adoption, with scholars concluding that gender is not a significant moderator of intention to use mobile apps,⁵⁷ and gender is not associated with aggregated mobile health use.⁵⁸

There was above average satisfaction with the usability of the app, with the sample providing an average score of 76.3 on the System Usability Scale. This finding is not surprising since the app contains quality medical information that originates from the government, a credible source, and prior research reports positive evaluations of mHealth apps.⁷³

In relation to user feedback, the survey participants recommended several ways to improve the design, such as increasing the button size, and this is congruent with usability guidelines in relation to visual design and data visualizations.^74,75 App designers were advised to incorporate voice recognition into the app, and reduce the amount of technical information; this feedback is aligned with the usability principle of ensuring that the input of text is effortless, and with the principle of reducing cognitive load (i.e., the process of thinking and remembering).⁷⁵ Users also recommended the marketing of the app to promote public awareness and training, and the latter is aligned with prior research that recommends the provision of face-to-face coaching to reduce technology anxiety and resistance to using mHealth apps.⁷⁶

Like all forms of research, this study has its limitations, it is a pilot study and generalizations beyond the sample cannot be made. Additional research on larger and more diverse samples is warranted. Acknowledging these limitations, this study provides useful preliminary information on the usability of a government-funded mHealth app amongst a sample of older, autonomous adults living in regional Australia.

Acknowledgments

This study was supported by a grant from the authors’ university.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Data will be made available from the first author.

Additional information

Funding

The work was supported by the James Cook University [Not applicable].