Exploring the usability, user experience and usefulness of a supportive website for people with dementia and carers

Abstract

Purpose

This study explores the usability, usefulness and user experience of the Forward with Dementia website for people with dementia and family carers, and identifies strategies to improve web design for this population.

Methods

The website was iteratively user-tested by 12 participants (five people with dementia, seven carers) using the Zoom platform. Data collection involved observations, semi-structured interviews and questionnaires. Integrative mixed-method data analysis was used, informed by inductive thematic qualitative analysis.

Results

Users of Version 1 of the website experienced web functionality, navigation and legibility issues. Strategies for desirable web design were identified as simplifying functions, streamlining navigation and decluttering page layouts. Implementation of strategies produced improvements in usability, user experience and usefulness in Version 2, with mean System Usability Scale scores improving from 15 to 84, and mean task completion improving from 55% to 89%. The user journey for people with dementia and carers overlapped, but each group had their own unique needs in the context of web design.

Conclusions

The interplay between a website’s content, functionality, navigation and legibility can profoundly influence user perceptions of a website. Dementia-related websites play an important role in informing audiences of management strategies, service availability and planning for the progression of dementia. Findings of this study may assist in guiding future web development targeting this population.

IMPLICATIONS FOR REHABILITATION

• People with cognitive impairment can provide useful feedback on design and accessibility of websites, and their input should be obtained when developing digital applications for this group.

• This paper provides practical suggestions for website design features to improve function, legibility and navigation of websites for older people and people living with dementia.

Keywords:

• People with dementia
• carers
• digital applications
• supportive websites
• user-centred design
• user-testing

Introduction

Dementia affects 47 million people globally and encompasses a range of neurocognitive conditions characterized by progressive declines in cognitive, functional and physical capabilities [1]. Global dementia prevalence is expected to rise to 75 million by 2030, costing an estimated US$2 trillion in direct medical and social costs, and opportunity costs associated with informal care [2]. Since 1990, the number of disability adjusted life years (DALYs) attributed to dementia have increased at rates several times higher than any other disease group, drawing considerable resources from communities, health care systems and governments [2].

Cognitive difficulties for people with dementia can manifest as a declined ability in frontal and executive functioning, hindering capacity for planning, abstract thinking, problem solving and judgement [3]. Dementia may also lead to a progressive decline in learning, memory and language capabilities [3]. Impaired motor planning and ability, and reduced physiological reserves can exacerbate the incidence of frailty, falls, malnutrition and other physical comorbidities [4]. Furthermore, sensory challenges such as visual-perceptual impairments, and visual or auditory hallucinations can be experienced by people with dementia [3]. Cognitive, motor and sensory challenges may compound each other, severely impacting daily functional activities.

Family carers of people with dementia also experience substantial physical, psychological and social impacts as they face increasingly challenging needs. Carers experience higher rates and severity of adverse mental health outcomes while also reporting poorer levels of physical health, sleep, exercise and social support [5]. Caring for a person with dementia usually takes place over a long time span and requires significantly more hours per week than caring for individuals with other conditions, impacting carers’ social and financial livelihoods [6].

Recently accelerated by COVID-19 social distancing requirements, health and social care providers are increasingly exploring opportunities to provide dementia information, education and intervention via digital applications [7]. These include software ranging from websites to communication programs and utility software that are run on a computer, tablet or mobile interface. Declines in executive functioning, language, motor and visuo-perceptual capabilities experienced by people with dementia [3,4] can be a significant barrier to successful digital health application usage [8,9]. Nonetheless, substantial increases in internet uptake by older people and a greater focus on self-management for people living with chronic disease are expanding the potential for digital dementia applications [10,11]. Emerging digital applications have been used by people with dementia to assist with activities of daily living as well as to facilitate management, planning, communication, learning or other unmet needs [12–14].

The knowledge base for the design of digital applications for people with dementia is emerging. Previously, this population has been largely excluded from research and design processes [15,16]. However, attempts at user-testing with this group showed that valuable data can be collected by using appropriate procedures, tools and conditions [12,16]. User-testing studies often utilize a mixture of participant observations, interviews and questionnaires to determine application effectiveness and provide recommendations on digital interface design [16,17]. Past user-testing studies have led to improved content and terminology [18,19], reduced navigational or functionality issues [18–21], informed typography, layout and aesthetics [22–24], as well as stimulated improvements in accessibility guidance such as the international Web Content Accessibility Guidelines (WCAG) [25] for digital applications.

However, the current evidence base informing digital application design for people with dementia remains small, and predominantly focuses on early stages of application development [16,17], or on the needs of people with dementia as defined by carers [26].

Carers and people with dementia tend to be in older age groups and their familiarity with digital technologies is typically lower than that of younger cohorts [27]. There is however a range of digital uptake and literacy within older people, with many demonstrating high levels of technological competence [28]. Knowledge concerning the design of digital applications for older people, such as implementing large sans serif typeface, simplified scrolling and minimalist interfaces also applies to carers and people with dementia [19].

Carers have been more involved in the development of dementia applications than people with dementia, with some researchers suggesting that the needs of people with dementia might be supplanted by the needs of carers [29]. Carers report needs for knowledge, resources and decision support aids to prepare for the large range of care and decision making they take on as dementia progresses [6]; these needs could be supported through digital applications. A systematic review of 17 randomized controlled trials (RCTs) found that internet-based supportive interventions improved mental health outcomes but required greater personalization to help carers develop coping competence, react to behavioural symptoms, and alleviate carer burden [30].

To answer these and other identified dementia support needs, the Forward with Dementia website (www.forwardwithdementia.org) was developed to provide post-diagnosis information for people with dementia and carers by the international research collaboration COGNISANCE (Co-Designing Dementia Diagnosis And Post Diagnostic Care). The website provides a browsable collection of articles tailored to either people with dementia or carers. In Version 1 (V.1) of the website, an associated “toolkit” allowed users, through creation of a unique URL, to create their own library of articles, and develop a prioritized, online schedule of practical actions as suggested at the end of each article. Both the website and toolkit allow printing, downloading or emailing articles. The design, content and basic functions of the website were co-designed with people with dementia, carers and health and social care professionals [31] and web developers utilized WCAG 2.0 (2018) as guiding principles. This study was conducted during the adaptation and evolution phase of web development, where key errors, omissions and shortcomings were identified and addressed [32].

This study aimed to explore the usability, usefulness and user experience of the Forward with Dementia website for people with dementia and carers. Additionally, this study aimed to identify strategies for designing digital applications to be used by people with dementia and carers.

Materials and methods

Design

We conducted an observational cross-sectional study, employing qualitative and quantitative methods to evaluate website usability, user experience and usefulness. Definitions and methods are provided in Table 1. The study was conducted over two phases. Version 1 of the Forward with Dementia website was tested in phase one. User feedback from V.1 informed major changes to the navigation, functionality, legibility and minor changes to the content to produce V.2, which was tested in phase two. Recruitment was pragmatic, conducted during COVID-19 lockdowns and within tight time frames required by the web developers. Participants were drawn from a population who volunteered to participate in further research through the COGNISANCE project [31]. Attempts were made to sample from different levels of digital literacy, location, age and gender to create diversity within the group. Participants were not required to test both versions of the website. Four participants tested V.1, two of whom also tested V.2. Nine additional participants tested V.2.

Table 1. Ergonomics of human–system interaction.

Construct	Definitions	Data collection methods
Usability	Extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [33].	Participant interviews Participant observations Modified System Usability Scale (SUS) [34]
User experience	A person’s perceptions and responses that result from the use and/or anticipated use of a product, system or service [33].	Participant interviews Participant observations
Usefulness	The degree to which a person believes that using a particular system would enhance their wider life objectives [35,p.320]	Participant interviews

Ethical approval for this study was given by the University of Sydney’s Human Research Ethics Committee (HREC no. 2021/329).

Participants

Five people with early-stage dementia and seven family carers participated in user testing (Table 2). Formal assessment of stage of dementia was not conducted. However, all participants retained the ability to provide informed consent and were competent in the use of all basic computer functions. Additional inclusion criteria for all participants were access to a personal internet connected device, use of the internet more than once a week, not having a visual or physical impairment that prevented the use of internet device, over 18 years old, and able to read English at a sufficient level to navigate an English language website.

Table 2. Participant demographics.

Participant category	Participant identifier	Gender	Age range	Years since diagnosis	Location	Version tested
People with dementia	P1	Female	55–59	10	Metropolitan NSW	V.1 + V.2
	P2	Female	70–74	6	Regional NSW	V.2
	P3	Male	60–64	2	Metropolitan NSW	V.2
	P4	Male	60–64	2	Metropolitan ACT	V.2
	P5	Male	70–74	0.25	Metropolitan VIC	V.2
Carers	C1	Female	70–74	n/a	Regional NSW	V.1
	C2	Female	_	n/a	Regional NSW	V.1
	C3	Female	55–59	n/a	Regional NSW	V.1 + V.2
	C4	Female	55–59	n/a	Metropolitan WA	V.2
	C5	Female	50–54	n/a	Regional NSW	V.2
	C6	Female	45–49	n/a	Metropolitan QLD	V.2
	C7	Female	70–74	n/a	Metropolitan VIC	V.2

Procedure

The website was tested by participants using their own internet browsing device. Participants shared their screen and interacted with researchers via Zoom videoconference in a physical setting of their choice. Sessions were recorded via Zoom and recordings stored in secure password-protected servers. Recordings were transcribed using Otter AI software [36]. NVivo [37] was utilized to assist with the coding and analysis of qualitative data.

Participants were asked to browse the Australian Forward with Dementia website for 60–75 min while sharing their screen via Zoom. Participants were instructed to explore the website at their own pace while researchers observed their interaction with the website and asked questions based on the semi-structured interview guide.

Carers were encouraged to express their moment-by-moment thoughts, intentions, beliefs and expectations as part of the think-aloud protocol (TAP) [38]. For people with dementia, the TAP was not strictly promoted as previous user-testing studies with people with dementia found that TAP hindered participants’ ability to concentrate on the application and did not yield insightful information [12,20]. People with dementia were encouraged to express their thoughts as they browsed the site, but with less regular prompting from researchers.

A list of six common tasks relating to navigation and functions of the website (e.g., scrolling, printing and navigating to and from an external link) that would demonstrate mastery was created (Table 3). Researchers observed whether tasks were independently attempted. If no attempt was made, participants received gradually increasing levels of assistance from researchers, from non-directive prompts, e.g., “Would you find this useful to have a printed copy of this article?” to direct requests, e.g., “Can you scroll to the print button and try printing an article?” until the task was able to be completed [19]. Tasks differed slightly depending on the difference between V.1 and V.2 functionality at the time of user-testing.

Table 3. Tasks tested during user testing.

Tasks
1. Locate information of relevance and scroll down to the bottom of the page.
2. Save/print toolkit^a (V.1) or article (V.2)
3. Email/share toolkit^a (V.1) or article (V.2)
4. Click on an external link and navigate their way back to the forward with dementia site
5. Find and use the search function to search for information within articles, stories and news.
6. Use the breadcrumb^b function (V.2)

^a Toolkit – intended to be a digital portal where users can save articles and actions to do (“to-do’s”) from the main Forward with Dementia website. The toolkit could be re-access through a unique URL that remembers all saved items, bypassing the need for an account or login. This feature was discontinued due to overly complex and unintuitive nature, and poor integration into the main website.

^b Breadcrumb – a clickable navigation trail that aid users in keeping track of their location within the overall website.

Data collected

Participant interviews, questionnaires and observation enabled data triangulation to enhance credibility and address weaknesses of each individual data collection method.

A semi-structured interview guide was used to collect qualitative data during and after user-testing. Examples of questions asked include “What aspects of the website were most helpful/unhelpful to you?” or “How useful was function X in helping you reach your goals”? This facilitated open-ended discussion around the usability, usefulness and user experience of the website.

The System Usability Scale (SUS) [34] consists of 10 short statements about usability of digital interfaces. Following review by JZ and LFL, the SUS was simplified by changing words, terms and question order to be more relevant to participants of the current study. For example, the original statement “I think that I would use this system frequently” was simplified to “I think that I would use this website frequently”. The SUS captures quantitative data on a five-point Likert scale regarding participant perceptions of the usability, user experience and usefulness of the site. The first five questions concern agreement with positive aspects and the remaining five questions concern negative aspects of the participants’ interaction with the website. The SUS was administered at the end of the session. The original and simplified version is available in Appendix 1.

A User-Testing Behaviour Observation Grid (Figure 1) was developed by JZ and LFL based on key variables measured in similar studies [19,21,22]. This grid facilitated collection of both quantitative and qualitative data, and recorded Task Completion Rates (TCRs), observations of user/navigational error, verbal and visual indications of difficulties, and irregular mouse movements [19,21,22]. Observations were recorded during user-testing sessions by using this grid, with observations reviewed by analysing video recordings.

Figure 1. Participant behaviour observation grid and task completion activities.

A 16 × 5 cell table listing eight types of observable reactions participants may demonstrate while undertaking task involving using the website, and six tasks to be attempted during the user testing session. Columns provide space for the researcher to record time, function or section of the website where difficulties were experienced, what the participant was trying to do and a description of the issue.

Data analysis

Audio recordings of semi-structured interviews were transcribed verbatim and content analysis [39] undertaken to identify common topics, ideas or patterns relating to the constructs of usability, usefulness and user experience. Integrative mixed-method interpretation [40,41] was undertaken by JZ, in collaboration with MG and LFL.

Notes taken using the observation grid and participant transcripts were analysed using line-by-line coding. Notes related to usability, user experience or usefulness were assigned a code (e.g., page length, scrolling and labels) and entered into NVivo analysis software. Codes with similar characteristics were compiled into categories (e.g., barriers to information pathways) which ultimately informed themes (e.g., navigational usability), and were then mapped to the three main constructs of usability, user experience, and usefulness. Quotes were those most representative of the entire sample’s experiences within each category/theme. Quotes that highlighted issues experienced by a subsection of participants were also used. Qualitative data captured through participant observation (visual observations of frustration, navigational difficulties, unvocalized issues) have also been recorded within the participant behaviour grids and captured in NVivo to assist with analysis.

Integration through narrative with a weaving approach [42] was used to merge quantitative data with qualitative findings from interviews and participant observation. This entailed the concurrent but independent analysis of our quantitative and qualitative data. An integrated analysis of all data on a theme-by-theme or concept-by-concept basis was used to understand participant experiences of the website.

Results

Interview and participant observation data

From interviews, observation data and the TAP protocol, people with dementia and carers described characteristics they considered important for a usable, user friendly and useful website. Results are presented in Table 4 (website content including information, images, language and terminology), Table 5 (website functionality including search, scroll up/down arrow, breadcrumb, print, email and toolkit functions), Table 6 (website navigation involving movement between and within pages of the website) and Table 7 (website legibility such as visual design and typography). These characteristics emphasized simplification, reduction of number of steps for any action, streamlining of navigation, clarity of presentation and respectful language to support usability.

Table 4. Characteristics relating to user experience and usefulness of website content including information, images, language and terminology (participant interview responses are quoted and italicized).

Construct	Participant identified desirable web design characteristics	Evaluation of Forward with Dementia against desired web design strategies
User experience	Employ simple, clear and direct language. “Keep to the point and don’t get verbose.” (P4)	V.1: small number and frequency of complaints, but some people with dementia were fixated on several words/phrases that were ambiguous, misinterpreted or deemed too abstract. “Practical help to me means ‘resources’ not ‘articles’, that would frustrate me.” (P1) V.2: unclear terminology fixed in this version with virtually no complaints about language clarity thereafter. Users appreciated the short, succinct sentences and minimal use of jargon. “It’s easy to read, nice and short. That’s good.” (P4)
	Cultivate empowering, practical and positive language to enhance user perceptions. “Be helpful and be kind in the tone of your articles.” (C3)	V.1 + V.2: language was generally praised for being realistic, positive and inspiring. “I like ‘Forward with Dementia’… It’s positive and it’s progressive. It’s not being a victim.” (C3) Certain isolated phrases criticized by some users for appearing paternalistic, disempowering or overly negative. “Manage expectations. That’s a really ugly term to me, it’s a very paternalistic term, so I wouldn’t click on that.” (P1)
	Use relatable, representative and warm images to instil a sense of hope and understanding. “It’s important to have the images informative, but positive.” (C5)	V.1 + V.2: images resulted in mixed responses; some found the images profoundly resonating: “The photography, it’s really beautiful… I can immediately relate to that, that’s how I'm feeling about my person who needs my care.” (C5) Others found the images somewhat unrepresentative of the Australian population, creating a sense of frustration or disconnection from the website. The absence of younger people in the images were also particularly noteworthy for some people with dementia. “They’re always depicted as older people… it’s a stereotype thing. And we’re a little bit conscious of the actual pictures.” (P4)
Usefulness	Provide tailored content that is comprehensive, practical, and reassuring or inspiring. “A lot of people who are in the role of carer have just so many questions.” (C5)	V.1 + V.2: dividing the content into sections tailored towards specific informational or resource needs of people with dementia or carers was highly valued. Usefulness of content widely praised due to relatability and relevance, links to external resources, actionable suggestions and validating tenor. Content about dementia management strategies, support services and planning for financial/legal issues were particularly helpful. “I'm looking at every single challenge, every single doubt, everything you come up against, it’s here.” (C5) Few content suggestions were made – more targeted information on younger onset dementia, indigenous communities, and poorly understood phenomena such as sundowning.
	Present sources/team credentials to enhance credibility and perceived usefulness of content. “I’ll look at it and go, is it accredited, is it all above-board?” (C6)	V.1 + V.2: credentials of website unclear for some users due to lack of visible information on organization/universities behind website. "Maybe just a brief description to say what the aim of the website is, who we are?” (P2) Other users derived credibility from the high accuracy and relatability of the content. “From what I’ve seen, the data source that they’re using must be pretty good. So, the rest is probably pretty accurate and pretty good.” (P4)

Table 5. Characteristics of website functionality including search, scroll up/down arrow, breadcrumbs, print, email and toolkit functions (participant interview responses are quoted and italicized).

Construct	Participant identified desirable web design characteristics	Evaluation of Forward with Dementia against desired web design strategies
Usability	Implement functions that don’t require memory, numerous complex steps or additional set-up. “Simplify. If you’re not hand holding and making it simple, they’ll leave the site.” (C3)	V.1: most toolkit functions required memory and incorporated numerous complex steps (navigating between several pages, requiring numerous clicks, reading lengthy instructions). Usability was hindered and guidance was required for most toolkit functions. V.2: saving an article (print + email) was achieved without guidance at higher rates. However, the need for additional set-up on some users’ devices rendered the email function unusable for three people with dementia. V.1 + V.2: search function and within-site navigational “breadcrumbs” were used without issue for all people with dementia and carers
	Incorporate generalized functions found in other familiar digital spaces. “I tend to generalize. That’s how my brain works now.” (P2)	V.1: relatively novel functions were harder to understand, learn and use successfully, such as the toolkit’s unique URL system. “Okay, that’s too confusing. I don’t know what that’s all about.” (P1) V.2: functions such as the search bar and breadcrumbs commonplace in other digital spaces were identified and used successfully at high rates. “Because I use folders a lot on my hard drive, I'm thinking this is what it’s doing.” (P3) Other novel functions like the embedded scroll up/down buttons could be used but needed prompting for users to interact with it.
User experience	Enhance user confidence by employing straightforward functions. “Make it simple, make it manageable. Don’t overwhelm me.” (C3)	The search function in both V.1 and V.2 were praised for its straightforwardness and clarity. “Got to be happy with that, it’s a simple search, nothing hard about it.” (P4) V.1: toolkit function created confusion and frustration due to its complexity, lack of clarity and poor instructions. “I'd be quite uncomfortable now. Because I almost got something useful. But it’s gone into the ether somewhere.” (P1) V.2: website functions were generally well received due to their simplicity and intuitiveness, particularly the print and breadcrumb functions, which built user confidence.
Usefulness	Incorporate functions that simplify finding, saving or sharing information. “When I don’t see what I want, I'll usually try and shortcut to a search.” (P1) “Most of us have trouble reading on screen, so we’d prefer to print it.” (P2) “You connect and share info with people going through the same experiences.” (C5)	V.1: participants had mixed views on the usefulness of the toolkit as a concept to help organize, save and share information. “I think the idea of a toolkit is grand, actually. Yeah, really grand.” (C2) “A virtual toolkit doesn’t help me at all because I’d forget about it.” (P1) V.2: most participants saw the value in the print/email functions in simplifying the saving and sharing process. The search bar successfully simplified finding pieces of information for all participants. The breadcrumbs added value by assisting navigation and orientation. “The functions were well integrated. They’re something that could be useful, absolutely.” (P2) V.1 + V.2: users that tested both versions claimed that removing the V.1 toolkit in favour of the much simpler V.2 print/email function was appropriate. “A printed toolkit is more important than a virtual toolkit. Take me to a print PDF button.” (P1)

Table 6. Characteristics of website navigation between and within pages of the website (participant interview responses are quoted and italicized).

Construct	Participant identified desirable web design characteristics	Evaluation of Forward with Dementia against desired web design strategies
Usability	Aid navigation by employing clear labels and creating shorter pathways to information. “Let me go down the rabbit hole that I want. But make it simple to get there.” (C3)	V.1: users needed to navigate through longer pathways (i.e., number of clicks) to reach information due to long page lengths in V.1 requiring users to scroll extensively. Labels to less applicable aspects of the website were equally prominent as the more relevant ones, leading users down confusing pathways. Numerous interruptions to user pathways included pop-up messages and other distracting elements. V.2: users were able to reach targeted information with fewer clicks and in a shorter duration due to shorter page lengths and better labelled pathways. Numerous navigational options were also spread throughout the page, assisting navigation without cluttering the screen. Some people with dementia still indicated scrolling difficulties. V.1 + V.2: three participants had issues navigating between the website and external links that opened in new browser tabs.
	Ensure clickable targets are clearly identifiable in navigation.	Some users were unsure of what was and wasn’t clickable, slowing down or providing barriers to navigation V.1: 3/4 participants tried to click on images within toolkit instructions, delaying user pathways as they waited for a response. V.2: one person with dementia had issues with clicking on the drop-down menu. Other users clicked on unclickable labels and images, creating longer pathways to information.
User experience	Implement simple and intuitive navigation to keep users focused and engaged. “The frustration I could see starting to happen was if it gets too hard to click through.” (C3)	V.1: users commonly lost focus and became frustrated with navigation due to complexity of website mapping and pathways, compounded by distractions and ambiguity of the toolkit. “Oh, I don’t know how it got there. I don’t know what just happened.” (P1) V.2: navigation in V.2 was viewed as easier, friendlier and more forgiving. “I didn’t feel lost. But if I were, I could navigate out of it. So that’s good.” (P3) Changes implemented to V.2’s navigation from V.1 were positively received “The whole website is a lot simpler. You can get to what you want faster.” (C3)

Table 7. Characteristics around website legibility such as visual design and typography (participant interview responses are quoted and italicized).

Construct	Participant identified desirable web design characteristics	Evaluation of Forward with Dementia against desired web design strategies
Usability	Promote the use of features by enhancing legibility with clear prompts, cues and visual aids. “Anywhere you don’t have to search hard for something, it’s good.” (P4)	V.1: toolkit function was very legible due to prompts, and cues spread throughout the website allowing all users to attempt using it. Once inside the toolkit, legibility was mixed, with some features not accessed until prompted. “For older people, if it’s not there (immediately visible), it’s not there.” (P1) V.2: legibility of features was mixed. Print and email icons were missed by half of participants due to their location at the bottom of pages. Embedded scroll up/down buttons were not noticed by some users due to their small size and poor colour contrast depending on website background. “I've got ‘search’ and ‘languages’ where I would expect to see the print button.” (P1) V.1 + V.2: search bar located at the top right corner of every page was adequately legible, vast majority of participants located it without any hinting.
	Limit the number of clickable options presented at any one time. “Simplify the copy. Give me less choices.” (C3)	V.1: key landing pages overwhelmed users with too many clickable options. Some users were presented with 10+ clickable choices at a time, leading users to click less relevant links. V.2: the number of clickable options displayed to users on one screen were reduced significantly, allowing users to find relevant information and follow the website easier. “There weren’t too many choices, and you could follow step-by-step. I like that.” (C7)
User experience	Participants desire simple and minimalist designs and layouts. “Less is more, I don’t want to be overwhelmed.” (C3)	V.1: website layouts generally perceived to contain too many elements, too much text, and were too long in length. “Once I start tinkering, there’s all of this like whoa, that’s too much on a landing page.” (P1) V.2: most users appreciated the shortened pages, reduced text length and reduced number of web page elements. Nonetheless, some people with dementia still reported issues with page lengths. “I like the way it’s easy to read, easy to follow and without all the distractions.” (P2) V.1 + V.2: large, clear, sans serif font that uses bolding, bullet points, text alignment and sufficient white space create an easier experience. Punctuating paragraphs with frequent sub-headings also appreciated. “It’s good, you’ve included bullet points which I always like to break it up.” (C3)
	Improve legibility and user perceptions through colour use. “The difference we have with our eyes and colours….” (P4)	V.1 + V.2: colour in most areas praised for simplicity ability to illuminate key aspects of the website. Website colour scheme was well received for contributing to an inviting and warm tone. “I know dementia awareness is purple, so that resonates with me, it’s soft.” (P3) However, poor colour contrast in several parts of the website inhibited legibility. Colour gradients were also noted as distracting and inhibited the legibility of text for several participants with dementia. “Not enthusiastic about that, my brain is noticing that rather than the words.” (P2)

In V.1 of the website, content was regarded as useful due to its comprehensiveness of content, practicality and reassuring tone, but it did not meet other desired website characteristics. Participants found navigation was unclear and unintuitive, functionalities too complex, and visual design cluttered.

Website V.2 simplified functionalities. The toolkit was removed due to its overly complex and unintuitive nature, and poor integration into the main website, and replaced with a print and email function in the main website. V.2 navigation was streamlined, with more direct user pathways by shortening pages to reduce the need for scrolling and implementing alternative ways to navigate such as the breadcrumb function. Decluttering of the website by simplifying layout, reducing the number of clickable options, and incorporating adequate white space helped enhance legibility. While improvements across the website were not entirely uniform, the strategies identified for creating a usable, user friendly and useful digital application were more often achieved in V.2. A side-by-side screenshot of V.1 and V.2 of the landing page is shown in Figure 2. This side-by-side comparison illustrates that web design may appear aesthetically pleasing but perform poorly for users without detailed consideration of the needs of people with specific cognitive, motor and sensory issues.

Figure 2. Side-by-side comparison of V.1 (left) and V.2 (right).

Side-by-side comparison of V.1 (left) and V.2 (right) of the Forward with Dementia landing page of the people with dementia page. Changes made to create V.2 after incorporating feedback from participants included reducing page length by 50%, reducing visual clutter and the number of clickable options presented to users at any one time, and simplifying layout/navigation by dividing web sections into separate pages. Furthermore, the most pertinent links have increased in visibility by enhancing link size and positioning them centrally in V.2 of the website. The text in the banner has also been modified from “Dementia. Now what?” to “Dementia. Find answers here” following user feedback about the perceived ambiguity of the text in V.1 of the website.

Task Completion Rates and System Usability Scale

Table 8 shows TCRs. TCRs recorded in V.1 indicated that participants experienced difficulties saving, printing and emailing content. Completion rates of these tasks improved for V.2. For example, there was an improvement of 88.8% in TCR percentage for item-2 relevant to saving/printing, and a smaller improvement of 30.5% and 38.8% in TCR percentage for item-3 relevant to emailing/sharing and item-4 relevant to navigating, respectively. However, people with dementia continued to have difficulties with the email function.

Comparison of mean SUS [43] scores showed improvement on all 10 domains (Figure 3). For the first five questions concerning positive interactions with the website, mean scores changed from $\overline{x}$ = 1.60 to $\overline{x}$ = 4.28, where a higher score indicates higher usability. For the last five questions concerning negative interactions with the website, mean scores changed from $\overline{x}$ = 4.40 to $\overline{x}$ = 1.58, with a lower score indicating fewer usability barriers. Mean SUS scores were calculated from the 10 questions using standard SUS scoring processes [43] and improved from $\overline{x}$ = 15.0 in V.1 to $\overline{x}$ = 83.75 in Version 2, with a score of >68 indicating better than average usability [43].

Figure 3. Simplified System Usability Scale scores.

A horizontal grouped bar chart showing mean scores for a five-point Likert scale, measuring 10 factors that influence usability of a website. Mean scores for Version 1 (V.1) of the website are shown as block shaded bars and mean scores for version 2 (V.2) as diagonally striped bars. Usability significantly improved for V.2.

Table 8. Task Completion Rates.

Tasks	V.1 participants					V.2 participants										Total improvement of TCR from V.1 to V.2
	P1	C1	C2	C3	Total TCR	P1	P2	P3	P4	C3	C4	C5	C6	C7 + P5^a	Total TCR
1. Locate information of relevance and scroll down to the bottom of the page.	✓	✓	✓	✓	100%	✓	✓	✓	✓	✓	✓	✓	✓	✓	100%	0.0%
2. Save/print toolkit (V.1) or article (V.2)	b	b	a	a	0%	✓	✓	a	✓	✓	✓	✓	✓	✓	88.8%	88.8%
3. Email/share toolkit (V.1) or article (V.2)	b	a	a	✓	25%	b	b	b	✓	✓	✓	✓	✓	a	55.5%	30.5%
4. Click on an external link and navigate their way back to the Forward with Dementia site	✓	x	✓	b	50%	✓	✓	✓	✓	✓	✓	✓	✓	b	88.8%	38.8%
5. Find and use the search function.	✓	✓	✓	✓	100%	N/A	✓	✓	✓	✓	✓	N/A	N/A	N/A	100%	0.0%
6. Use the breadcrumb function (V.2)	N/A	N/A	N/A	N/A	N/A	✓	✓	✓	N/A	✓	✓	✓	✓	✓	100%	N/A

Key: (x) Could not complete. (✓) Completed without assistance. (a) Completed with prompting or minimal guidance. (b) Completed with detailed, explicit instruction. (N/A) Not applicable. (P) Person with dementia. (C) Family carer.

^a Participated as a dyad.

Discussion

The purpose of this user testing study was to explore the usability, usefulness and user experience of the Forward with Dementia website for people recently diagnosed with dementia and carers, as well as identify strategies to improve both Forward with Dementia during website development website for these users and develop recommendations for improvement of digital interfaces for this population in Australia.

Usability and user experience of V.1 and V.2 of the Forward with Dementia website

Data from participant observation, interviews and the SUS indicate that the overall usability and user experience of V.1 were subpar. Long user pathways, overly complicated functionalities, and cluttered layouts contributed to negative participant feedback, low SUS scores and low TCRs. Improvements in these areas meant that V.2 was more usable as reflected in improved SUS scores, TCRs and interview feedback. Content usefulness was high for both versions. While only one carer (carer 7) and one person with dementia (participant 3) tested both V.1 and V.2, their feedback emphasized streamlined navigation, simplified functionalities and enhanced legibility.

Experiences of carers compared with people with dementia

Carers and people with dementia had overlapping perceptions of usability, user experience and usefulness, with key exceptions in several areas. Some participants from both groups did not realize there was content below the initial screen or found scrolling difficult. People with dementia experienced and verbalized these difficulties frequently, consistent with previous research [16]. Difficulties with functions that required or assumed additional set up, such as email, were more prominent in people with dementia. They also experienced more direct and intense responses to colour, images and language. For example, people with dementia were more likely to complain about text and background colour contrasts, likely due to oculo-visual changes associated with dementia [44]. People with dementia were also more sensitive to images portraying people with dementia, requesting depiction of wider range of age groups. While the language used was generally perceived to be appropriate by all participants, people with dementia were more ready to suggest friendlier or more understandable phrases. This may reflect different sensitivities to language between people with dementia and the general public [24].

Perceptions of function usefulness slightly differed between groups. The print and email functions were appreciated by most participants. Carers were generally more enthusiastic about the email function compared to people with dementia, who conversely prioritized the ability to print. This may be because carers are more likely to be advocates and share information [45].

Furthermore, people with dementia and carers looked for slightly different content, not just in terms of information, but also in subtextual needs. The Forward with Dementia site was valued by offering content tailored to specific informational and resource needs of each group, such as information and strategies to prevent or manage family carer burnout for carers, or information and resources on how to share the diagnosis for people with dementia.

Desired characteristics of digital applications for people with dementia and carers

This study found that minimalist design that reduced cognitive load better met the needs of people with dementia and carers. This may be achieved through decluttering interfaces, shortening user pathways and simplifying functions. People with dementia and carers wanted websites where it was easy find, save and share information, and that information should be comprehensive, practical and reassuring.

The desired characteristics identified in this study are consistent with the literature. The use of simple, clear and direct language to minimize misinterpretations of content and web elements has been frequently promoted [18,19]. The need to minimize scrolling and reduce pathways to information is well documented for people with dementia and older populations [16,46]. Intuitive but minimalist layouts with shortened page lengths reduces content density [18,21] and decreasing clickable options reduces cognitive load [15,47]. This study identified the importance of colour, colour contrast and typographic elements to enhance legibility, as documented by previous user-testing studies [22,23]. These findings also generally mirror recommendations/guidelines for designing websites for older people and people with other cognitive conditions [19,23,27]. Previous studies have indicated uncertainties about the impact of labels and visual aids in assisting navigation within this population [24,48]. This study found that labels and icons are desirable for assisting navigation when prioritized for salient user pathways and implemented alongside a minimalist layout. The literature is mixed on the use of bullet points for people with dementia [18,21], but bullet points are recommended based on the findings of this study.

This study was consistent with previous research where carers emphasized the need for comprehensive, practical and reassuring content [14]. In addition, we found tailoring content into separate sections for people with dementia and carers was highly useful compared to producing combined content for both groups.

This study illustrates that website components (content, functionality, legibility and navigation) are deeply interconnected from a user perspective. For example, being able to notice and use website functions such as the search bar is influenced by the overall legibility of the web page. The effectiveness of the search function shapes user navigation towards relevant content. As such, individual website components need to be designed in a manner that considers the whole user journey. For people with dementia one navigational barrier might mean they could not access large sections of content. Website components need to be designed in an integrated manner that considers a user’s potential motor, sensory or cognitive limitations. A holistic, user-centred design process will continue to be central in developing successful digital applications for people with dementia and carers. Considerations for website design with this population is summarized in Table 9.

Table 9. Key considerations for designing digital applications for people with dementia and family carers.

	User experience	Usability	Usefulness
Web content (information, images, language and terminology)	Employ simple, direct and empowering language. Minimize use of verbose, abstract, paternalistic or overly negative wording Use relatable, warm and representative images, including images of younger people with dementia	N/A	Splitting content into equal sections tailored towards people with dementia or carers is valuable Provide content that is comprehensive, practical, and reassuring or inspiring Present sources/team credentials to enhance credibility and perceived usefulness of content
Web functionality (search, scroll up/down arrow, breadcrumbs, print, email and toolkit functions)	Enhance user confidence by employing straightforward functions Minimize frustration by designing intuitive web functions and/or provide simple instructions	Implement functions that do not require memory, numerous complex steps or additional set-up Incorporate generalized functions found in other familiar digital spaces such as search bars and breadcrumbs. Novel functions are harder to use successfully	Incorporate functions that simplify finding, saving or sharing information. These tasks are heavily prioritized by users
Navigation (navigating between and within pages of the website)	Implement simple and intuitive navigation to keep users focused and engaged. Minimize distracting or complex elements such as pop-ups	Aid navigation by creating shorter pathways to information. Minimize number of clicks or length of scrolling required to find info Ensure clickable targets are clearly labelled and identifiable	N/A
Legibility (visual design and typography)	Enhance user perceptions by utilizing minimalist designs and layouts. Reduce page length and clutter Use colour to illuminate key aspects of website but be mindful of sensitivities relating to colour contrast which may inhibit legibility	Promote the use of features by enhancing legibility with clear prompts, cues and visual aids Limit the number of clickable options presented at any one time to direct users to relevant information	N/A

Further research is required to confirm findings, explore other design intricacies, and enhance knowledge in this area. Areas of particular interest may include the development of websites that negate the need to scroll, given the ubiquity of scrolling issues across the literature, and found in this study. Digital applications that simplify the process of acquiring, saving and sharing potentially large amounts of information should also be further researched.

Limitations

This study has limitations that may influence the generalizability of findings. Despite accounting for only 6–8% of dementia diagnoses [49], three out of five participants with dementia in this study were diagnosed with younger onset dementia. Participants were recruited from the COGNISANCE database and investigator networks, had previously been involved in dementia research or advocacy, and are not necessarily a “typical” person with dementia or carer.

Prior interactions with researchers and the COGNISANCE project by participants and the videoconference nature of data collection may have increased the risk of social desirability bias [50]. Researchers attempted to negate this by repeatedly encouraging participants to give honest feedback and checking findings against recorded participant observations. The retest effect may have also altered user behaviour for the two participants who tested both V.1 and V.2, potentially inflating perceptions of usability, user experience and usefulness [51]. TCRs were introduced as an objective measure to mitigate these limitations.

The study design was affected by difficulties with recruitment during COVID-19 lockdowns and inflexible timelines for website development, resulting in uneven sample size for testing V.1 and V.2. Additionally, not all functionalities were tested in V.2, leading to missing TCR data. SUS data for carers 1 and 2 in V.1 were not collected. However, data collected from participant observation, interviews and TCRs for carers 1 and 2 were consistent with others from the same phase of the study.

This study did not explore other modalities of website usage such as information delivery via video or audio as an alternative to reading. Future research could consider contrasting user experience of different modalities.

Conclusions

Participant observations, interviews and questionnaire data demonstrated that the content in V.1 of the Forward with Dementia website was useful, but with poor usability and user experience in website functionality, navigation and legibility. Strategies to address these issues were successfully implemented to improve usability in V.2. However, some participants still struggled with certain aspects. Some people with dementia continued to find scrolling difficult, found the colour contrast in certain sections of the website hard to see, and could not use the email and print functions. This study illustrates the interplay between a website’s content, functionality, navigation and legibility can profoundly influence user perceptions of a website and they need to be co-designed in an integrated and user-centred manner.

Health and wellbeing information is increasingly available online. To ensure its accessibility by older people, and people with dementia, it is essential to increase understanding of how they interact with, and derive the most benefit from, using digital interfaces. This study provides key considerations for digital interface design for people with dementia and family carers and may guide future web development targeting this population.

Acknowledgements

The authors acknowledge and thank the contribution of participants to the study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Data are not available due to ethical reasons.

Additional information

Funding

This work was supported by the Australian National Health and Medical Research Council (NHMRC), under Grant GNT1169452, as awarded by the EU Joint Program Neurodegenerative Disease (JPND). Author LFL was supported by the NHMRC Boosting Dementia Research Leadership Development Fellowship.

Appendix 1.

Simplified System Usability Scale (SUS)

Table

Original SUS	Simplified SUS
1. I think that I would like to use this system frequently	1. I think that I would use this website frequently
2. I found the system unnecessarily complex	2. I thought the website was easy to use
3. I thought the system was easy to use	3. I found the various functions in this website were well integrated
4. I think that I would need the support of a technical person to be able to use this system	4. I would imagine that most people would learn to use this website very quickly
5. I found the various functions in this system were well integrated	5. I felt very confident using the website
6. I thought there was too much inconsistency in this system	6. I found this website unnecessarily complex
7. would imagine that most people would learn to use this system very quickly	7. I think that I would need help to be able to use this website
8. I found the system very cumbersome to use	8. I thought there was too much inconsistency in this website
9. I felt very confident using the system	9. I found the website very cumbersome to use
10. I needed to learn a lot of things before I could get going with this system	10. I needed to learn a lot of things before I could get going with this website