Exploring the usability, user experience and usefulness of a supportive website for people with dementia and carers

Figures & data

Table 1. Ergonomics of human–system interaction.

Construct	Definitions	Data collection methods
Usability	Extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [33].	Participant interviews Participant observations Modified System Usability Scale (SUS) [34]
User experience	A person’s perceptions and responses that result from the use and/or anticipated use of a product, system or service [33].	Participant interviews Participant observations
Usefulness	The degree to which a person believes that using a particular system would enhance their wider life objectives [35,p.320]	Participant interviews

Table 2. Participant demographics.

Participant category	Participant identifier	Gender	Age range	Years since diagnosis	Location	Version tested
People with dementia	P1	Female	55–59	10	Metropolitan NSW	V.1 + V.2
	P2	Female	70–74	6	Regional NSW	V.2
	P3	Male	60–64	2	Metropolitan NSW	V.2
	P4	Male	60–64	2	Metropolitan ACT	V.2
	P5	Male	70–74	0.25	Metropolitan VIC	V.2
Carers	C1	Female	70–74	n/a	Regional NSW	V.1
	C2	Female	_	n/a	Regional NSW	V.1
	C3	Female	55–59	n/a	Regional NSW	V.1 + V.2
	C4	Female	55–59	n/a	Metropolitan WA	V.2
	C5	Female	50–54	n/a	Regional NSW	V.2
	C6	Female	45–49	n/a	Metropolitan QLD	V.2
	C7	Female	70–74	n/a	Metropolitan VIC	V.2

Table 3. Tasks tested during user testing.

Tasks
1. Locate information of relevance and scroll down to the bottom of the page.
2. Save/print toolkit^a (V.1) or article (V.2)
3. Email/share toolkit^a (V.1) or article (V.2)
4. Click on an external link and navigate their way back to the forward with dementia site
5. Find and use the search function to search for information within articles, stories and news.
6. Use the breadcrumb^b function (V.2)

^a Toolkit – intended to be a digital portal where users can save articles and actions to do (“to-do’s”) from the main Forward with Dementia website. The toolkit could be re-access through a unique URL that remembers all saved items, bypassing the need for an account or login. This feature was discontinued due to overly complex and unintuitive nature, and poor integration into the main website.

^b Breadcrumb – a clickable navigation trail that aid users in keeping track of their location within the overall website.

Figure 1. Participant behaviour observation grid and task completion activities.

A 16 × 5 cell table listing eight types of observable reactions participants may demonstrate while undertaking task involving using the website, and six tasks to be attempted during the user testing session. Columns provide space for the researcher to record time, function or section of the website where difficulties were experienced, what the participant was trying to do and a description of the issue.

Table 4. Characteristics relating to user experience and usefulness of website content including information, images, language and terminology (participant interview responses are quoted and italicized).

Construct	Participant identified desirable web design characteristics	Evaluation of Forward with Dementia against desired web design strategies
User experience	Employ simple, clear and direct language. “Keep to the point and don’t get verbose.” (P4)	V.1: small number and frequency of complaints, but some people with dementia were fixated on several words/phrases that were ambiguous, misinterpreted or deemed too abstract. “Practical help to me means ‘resources’ not ‘articles’, that would frustrate me.” (P1) V.2: unclear terminology fixed in this version with virtually no complaints about language clarity thereafter. Users appreciated the short, succinct sentences and minimal use of jargon. “It’s easy to read, nice and short. That’s good.” (P4)
	Cultivate empowering, practical and positive language to enhance user perceptions. “Be helpful and be kind in the tone of your articles.” (C3)	V.1 + V.2: language was generally praised for being realistic, positive and inspiring. “I like ‘Forward with Dementia’… It’s positive and it’s progressive. It’s not being a victim.” (C3) Certain isolated phrases criticized by some users for appearing paternalistic, disempowering or overly negative. “Manage expectations. That’s a really ugly term to me, it’s a very paternalistic term, so I wouldn’t click on that.” (P1)
	Use relatable, representative and warm images to instil a sense of hope and understanding. “It’s important to have the images informative, but positive.” (C5)	V.1 + V.2: images resulted in mixed responses; some found the images profoundly resonating: “The photography, it’s really beautiful… I can immediately relate to that, that’s how I'm feeling about my person who needs my care.” (C5) Others found the images somewhat unrepresentative of the Australian population, creating a sense of frustration or disconnection from the website. The absence of younger people in the images were also particularly noteworthy for some people with dementia. “They’re always depicted as older people… it’s a stereotype thing. And we’re a little bit conscious of the actual pictures.” (P4)
Usefulness	Provide tailored content that is comprehensive, practical, and reassuring or inspiring. “A lot of people who are in the role of carer have just so many questions.” (C5)	V.1 + V.2: dividing the content into sections tailored towards specific informational or resource needs of people with dementia or carers was highly valued. Usefulness of content widely praised due to relatability and relevance, links to external resources, actionable suggestions and validating tenor. Content about dementia management strategies, support services and planning for financial/legal issues were particularly helpful. “I'm looking at every single challenge, every single doubt, everything you come up against, it’s here.” (C5) Few content suggestions were made – more targeted information on younger onset dementia, indigenous communities, and poorly understood phenomena such as sundowning.
	Present sources/team credentials to enhance credibility and perceived usefulness of content. “I’ll look at it and go, is it accredited, is it all above-board?” (C6)	V.1 + V.2: credentials of website unclear for some users due to lack of visible information on organization/universities behind website. "Maybe just a brief description to say what the aim of the website is, who we are?” (P2) Other users derived credibility from the high accuracy and relatability of the content. “From what I’ve seen, the data source that they’re using must be pretty good. So, the rest is probably pretty accurate and pretty good.” (P4)

Table 5. Characteristics of website functionality including search, scroll up/down arrow, breadcrumbs, print, email and toolkit functions (participant interview responses are quoted and italicized).

Construct	Participant identified desirable web design characteristics	Evaluation of Forward with Dementia against desired web design strategies
Usability	Implement functions that don’t require memory, numerous complex steps or additional set-up. “Simplify. If you’re not hand holding and making it simple, they’ll leave the site.” (C3)	V.1: most toolkit functions required memory and incorporated numerous complex steps (navigating between several pages, requiring numerous clicks, reading lengthy instructions). Usability was hindered and guidance was required for most toolkit functions. V.2: saving an article (print + email) was achieved without guidance at higher rates. However, the need for additional set-up on some users’ devices rendered the email function unusable for three people with dementia. V.1 + V.2: search function and within-site navigational “breadcrumbs” were used without issue for all people with dementia and carers
	Incorporate generalized functions found in other familiar digital spaces. “I tend to generalize. That’s how my brain works now.” (P2)	V.1: relatively novel functions were harder to understand, learn and use successfully, such as the toolkit’s unique URL system. “Okay, that’s too confusing. I don’t know what that’s all about.” (P1) V.2: functions such as the search bar and breadcrumbs commonplace in other digital spaces were identified and used successfully at high rates. “Because I use folders a lot on my hard drive, I'm thinking this is what it’s doing.” (P3) Other novel functions like the embedded scroll up/down buttons could be used but needed prompting for users to interact with it.
User experience	Enhance user confidence by employing straightforward functions. “Make it simple, make it manageable. Don’t overwhelm me.” (C3)	The search function in both V.1 and V.2 were praised for its straightforwardness and clarity. “Got to be happy with that, it’s a simple search, nothing hard about it.” (P4) V.1: toolkit function created confusion and frustration due to its complexity, lack of clarity and poor instructions. “I'd be quite uncomfortable now. Because I almost got something useful. But it’s gone into the ether somewhere.” (P1) V.2: website functions were generally well received due to their simplicity and intuitiveness, particularly the print and breadcrumb functions, which built user confidence.
Usefulness	Incorporate functions that simplify finding, saving or sharing information. “When I don’t see what I want, I'll usually try and shortcut to a search.” (P1) “Most of us have trouble reading on screen, so we’d prefer to print it.” (P2) “You connect and share info with people going through the same experiences.” (C5)	V.1: participants had mixed views on the usefulness of the toolkit as a concept to help organize, save and share information. “I think the idea of a toolkit is grand, actually. Yeah, really grand.” (C2) “A virtual toolkit doesn’t help me at all because I’d forget about it.” (P1) V.2: most participants saw the value in the print/email functions in simplifying the saving and sharing process. The search bar successfully simplified finding pieces of information for all participants. The breadcrumbs added value by assisting navigation and orientation. “The functions were well integrated. They’re something that could be useful, absolutely.” (P2) V.1 + V.2: users that tested both versions claimed that removing the V.1 toolkit in favour of the much simpler V.2 print/email function was appropriate. “A printed toolkit is more important than a virtual toolkit. Take me to a print PDF button.” (P1)

Table 6. Characteristics of website navigation between and within pages of the website (participant interview responses are quoted and italicized).

Construct	Participant identified desirable web design characteristics	Evaluation of Forward with Dementia against desired web design strategies
Usability	Aid navigation by employing clear labels and creating shorter pathways to information. “Let me go down the rabbit hole that I want. But make it simple to get there.” (C3)	V.1: users needed to navigate through longer pathways (i.e., number of clicks) to reach information due to long page lengths in V.1 requiring users to scroll extensively. Labels to less applicable aspects of the website were equally prominent as the more relevant ones, leading users down confusing pathways. Numerous interruptions to user pathways included pop-up messages and other distracting elements. V.2: users were able to reach targeted information with fewer clicks and in a shorter duration due to shorter page lengths and better labelled pathways. Numerous navigational options were also spread throughout the page, assisting navigation without cluttering the screen. Some people with dementia still indicated scrolling difficulties. V.1 + V.2: three participants had issues navigating between the website and external links that opened in new browser tabs.
	Ensure clickable targets are clearly identifiable in navigation.	Some users were unsure of what was and wasn’t clickable, slowing down or providing barriers to navigation V.1: 3/4 participants tried to click on images within toolkit instructions, delaying user pathways as they waited for a response. V.2: one person with dementia had issues with clicking on the drop-down menu. Other users clicked on unclickable labels and images, creating longer pathways to information.
User experience	Implement simple and intuitive navigation to keep users focused and engaged. “The frustration I could see starting to happen was if it gets too hard to click through.” (C3)	V.1: users commonly lost focus and became frustrated with navigation due to complexity of website mapping and pathways, compounded by distractions and ambiguity of the toolkit. “Oh, I don’t know how it got there. I don’t know what just happened.” (P1) V.2: navigation in V.2 was viewed as easier, friendlier and more forgiving. “I didn’t feel lost. But if I were, I could navigate out of it. So that’s good.” (P3) Changes implemented to V.2’s navigation from V.1 were positively received “The whole website is a lot simpler. You can get to what you want faster.” (C3)

Table 7. Characteristics around website legibility such as visual design and typography (participant interview responses are quoted and italicized).

Construct	Participant identified desirable web design characteristics	Evaluation of Forward with Dementia against desired web design strategies
Usability	Promote the use of features by enhancing legibility with clear prompts, cues and visual aids. “Anywhere you don’t have to search hard for something, it’s good.” (P4)	V.1: toolkit function was very legible due to prompts, and cues spread throughout the website allowing all users to attempt using it. Once inside the toolkit, legibility was mixed, with some features not accessed until prompted. “For older people, if it’s not there (immediately visible), it’s not there.” (P1) V.2: legibility of features was mixed. Print and email icons were missed by half of participants due to their location at the bottom of pages. Embedded scroll up/down buttons were not noticed by some users due to their small size and poor colour contrast depending on website background. “I've got ‘search’ and ‘languages’ where I would expect to see the print button.” (P1) V.1 + V.2: search bar located at the top right corner of every page was adequately legible, vast majority of participants located it without any hinting.
	Limit the number of clickable options presented at any one time. “Simplify the copy. Give me less choices.” (C3)	V.1: key landing pages overwhelmed users with too many clickable options. Some users were presented with 10+ clickable choices at a time, leading users to click less relevant links. V.2: the number of clickable options displayed to users on one screen were reduced significantly, allowing users to find relevant information and follow the website easier. “There weren’t too many choices, and you could follow step-by-step. I like that.” (C7)
User experience	Participants desire simple and minimalist designs and layouts. “Less is more, I don’t want to be overwhelmed.” (C3)	V.1: website layouts generally perceived to contain too many elements, too much text, and were too long in length. “Once I start tinkering, there’s all of this like whoa, that’s too much on a landing page.” (P1) V.2: most users appreciated the shortened pages, reduced text length and reduced number of web page elements. Nonetheless, some people with dementia still reported issues with page lengths. “I like the way it’s easy to read, easy to follow and without all the distractions.” (P2) V.1 + V.2: large, clear, sans serif font that uses bolding, bullet points, text alignment and sufficient white space create an easier experience. Punctuating paragraphs with frequent sub-headings also appreciated. “It’s good, you’ve included bullet points which I always like to break it up.” (C3)
	Improve legibility and user perceptions through colour use. “The difference we have with our eyes and colours….” (P4)	V.1 + V.2: colour in most areas praised for simplicity ability to illuminate key aspects of the website. Website colour scheme was well received for contributing to an inviting and warm tone. “I know dementia awareness is purple, so that resonates with me, it’s soft.” (P3) However, poor colour contrast in several parts of the website inhibited legibility. Colour gradients were also noted as distracting and inhibited the legibility of text for several participants with dementia. “Not enthusiastic about that, my brain is noticing that rather than the words.” (P2)

Figure 2. Side-by-side comparison of V.1 (left) and V.2 (right).

Side-by-side comparison of V.1 (left) and V.2 (right) of the Forward with Dementia landing page of the people with dementia page. Changes made to create V.2 after incorporating feedback from participants included reducing page length by 50%, reducing visual clutter and the number of clickable options presented to users at any one time, and simplifying layout/navigation by dividing web sections into separate pages. Furthermore, the most pertinent links have increased in visibility by enhancing link size and positioning them centrally in V.2 of the website. The text in the banner has also been modified from “Dementia. Now what?” to “Dementia. Find answers here” following user feedback about the perceived ambiguity of the text in V.1 of the website.

Figure 3. Simplified System Usability Scale scores.

A horizontal grouped bar chart showing mean scores for a five-point Likert scale, measuring 10 factors that influence usability of a website. Mean scores for Version 1 (V.1) of the website are shown as block shaded bars and mean scores for version 2 (V.2) as diagonally striped bars. Usability significantly improved for V.2.

Table 8. Task Completion Rates.

Tasks	V.1 participants					V.2 participants										Total improvement of TCR from V.1 to V.2
	P1	C1	C2	C3	Total TCR	P1	P2	P3	P4	C3	C4	C5	C6	C7 + P5^a	Total TCR
1. Locate information of relevance and scroll down to the bottom of the page.	✓	✓	✓	✓	100%	✓	✓	✓	✓	✓	✓	✓	✓	✓	100%	0.0%
2. Save/print toolkit (V.1) or article (V.2)	b	b	a	a	0%	✓	✓	a	✓	✓	✓	✓	✓	✓	88.8%	88.8%
3. Email/share toolkit (V.1) or article (V.2)	b	a	a	✓	25%	b	b	b	✓	✓	✓	✓	✓	a	55.5%	30.5%
4. Click on an external link and navigate their way back to the Forward with Dementia site	✓	x	✓	b	50%	✓	✓	✓	✓	✓	✓	✓	✓	b	88.8%	38.8%
5. Find and use the search function.	✓	✓	✓	✓	100%	N/A	✓	✓	✓	✓	✓	N/A	N/A	N/A	100%	0.0%
6. Use the breadcrumb function (V.2)	N/A	N/A	N/A	N/A	N/A	✓	✓	✓	N/A	✓	✓	✓	✓	✓	100%	N/A

Key: (x) Could not complete. (✓) Completed without assistance. (a) Completed with prompting or minimal guidance. (b) Completed with detailed, explicit instruction. (N/A) Not applicable. (P) Person with dementia. (C) Family carer.

^a Participated as a dyad.

Table 9. Key considerations for designing digital applications for people with dementia and family carers.

	User experience	Usability	Usefulness
Web content (information, images, language and terminology)	Employ simple, direct and empowering language. Minimize use of verbose, abstract, paternalistic or overly negative wording Use relatable, warm and representative images, including images of younger people with dementia	N/A	Splitting content into equal sections tailored towards people with dementia or carers is valuable Provide content that is comprehensive, practical, and reassuring or inspiring Present sources/team credentials to enhance credibility and perceived usefulness of content
Web functionality (search, scroll up/down arrow, breadcrumbs, print, email and toolkit functions)	Enhance user confidence by employing straightforward functions Minimize frustration by designing intuitive web functions and/or provide simple instructions	Implement functions that do not require memory, numerous complex steps or additional set-up Incorporate generalized functions found in other familiar digital spaces such as search bars and breadcrumbs. Novel functions are harder to use successfully	Incorporate functions that simplify finding, saving or sharing information. These tasks are heavily prioritized by users
Navigation (navigating between and within pages of the website)	Implement simple and intuitive navigation to keep users focused and engaged. Minimize distracting or complex elements such as pop-ups	Aid navigation by creating shorter pathways to information. Minimize number of clicks or length of scrolling required to find info Ensure clickable targets are clearly labelled and identifiable	N/A
Legibility (visual design and typography)	Enhance user perceptions by utilizing minimalist designs and layouts. Reduce page length and clutter Use colour to illuminate key aspects of website but be mindful of sensitivities relating to colour contrast which may inhibit legibility	Promote the use of features by enhancing legibility with clear prompts, cues and visual aids Limit the number of clickable options presented at any one time to direct users to relevant information	N/A

Table

Original SUS	Simplified SUS
1. I think that I would like to use this system frequently	1. I think that I would use this website frequently
2. I found the system unnecessarily complex	2. I thought the website was easy to use
3. I thought the system was easy to use	3. I found the various functions in this website were well integrated
4. I think that I would need the support of a technical person to be able to use this system	4. I would imagine that most people would learn to use this website very quickly
5. I found the various functions in this system were well integrated	5. I felt very confident using the website
6. I thought there was too much inconsistency in this system	6. I found this website unnecessarily complex
7. would imagine that most people would learn to use this system very quickly	7. I think that I would need help to be able to use this website
8. I found the system very cumbersome to use	8. I thought there was too much inconsistency in this website
9. I felt very confident using the system	9. I found the website very cumbersome to use
10. I needed to learn a lot of things before I could get going with this system	10. I needed to learn a lot of things before I could get going with this website

International Organisation for Standardisation. Ergonomics of human–system interaction – part 210: human-centred design for interactive systems. 2019. https://www.iso.org/obp/ui/#iso:std:iso:9241:-210:ed-2:v1:en

 Google Scholar

Brooke J. SUS: a quick and dirty usability scale. Usability Eval Ind. 1995;11(30):189.

 Google Scholar

Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quart. 1989;13(3):319–340.

 Web of Science ®Google Scholar

Data availability statement

Data are not available due to ethical reasons.