Initial validation of the powered mobility device autonomy residential screen (PoMoDARS)

Abstract

Background

The Powered Mobility Device Autonomy Residential Screen (PoMoDARS) is a new tool to enable clinicians to screen resident capacity and performance skills for powered mobility device (PMD) use in residential aged care settings. The PoMoDARS is context specific, time efficient and promotes resident autonomy and safety.

Aims

To (i) undertake initial face and content validation of the PoMoDARS, and (ii) use the research findings to make any modifications.

Methods

A mixed-methods study design, underpinned by Classical Test Theory. Eight clinicians completed 20 PoMoDARS screens and provided both quantitative and qualitative feedback on item importance and ease of use within a formal interview.

Results

Initial face and content validity of the PoMoDARS were supported, with small modifications made to item descriptors and instructions.

Conclusions

The PoMoDARS has been developed for use in residential aged care settings to screen resident PMD use. While initial validation has been undertaken, further studies to determine the reliability of the tool and continue the validation process are required.

Significance

Older adults in residential aged care facilities benefit greatly from the autonomy gained through PMD use. The PoMoDARS promotes collaboration between occupational therapists, nurses, and the wider team to support residents and safe PMD use.

Keywords:

• Aged care
• facility
• motorised mobility scooter
• nursing assessment
• occupational therapy
• older adult
• power wheelchair
• risk management
• technology

Introduction

The Aged Care Quality Standards in Australia, and similar guidelines internationally, support older adults living in residential aged care facilities to make choices concerning their care and how they spend their time [1–3]. Many older adults choose to supplement their mobility with a powered mobility device (PMD), either a motorised mobility scooter or a powered wheelchair, before or after settling into a residential aged care facility. Using a PMD can enhance autonomy and provide options to individualise daily activities like taking breakfast early, finding a quiet spot to read, visiting a local café with family, or taking an active role in a group activity. The freedom to express personal choice can provide stability within a changing environment, increase participation, return a sense of self control, enjoyment of independence and achievement, all of which are essential for emotional well-being and quality of life in residential care [4].

Residents in aged care facilities have an average age of 85 years, multiple co-morbidities, and a requirement for supervision in one or more of their activities of daily living [5]. The independence achieved through using a PMD in a residential aged care facility can be life enhancing and is often fiercely protected as the last form of independence a resident can enjoy [6]. Literature on the prevalence and severity of adverse PMD incidents that occur in residential aged care facilities in Australia and internationally, is limited [7]. However, a recent audit investigated the number and characteristics of all PMD incidents that occurred over 12 months in 33 aged care facilities within one provider group in New South Wales, Australia, with a total of 55 incident reports attributed to 30 residents located [8]. Results were combined with the estimated number of facilities in Australia [9] to project a figure of 4,453 potential PMD incidents occurring every year in Australian residential aged care facilities [8]. Whilst the recorded injuries were primarily skin tears, there were incidents involving vehicles, collision with furniture, falls, tipping over, running over feet, and going missing, which raised concern for the potential for serious injuries. For an older adult, injuries such as these can require hospitalisation and rehabilitation over extended periods of time, often longer than for younger people [10,11]. Although the audit did not capture escalation of an incident in this manner, any of the recorded (or projected) annual incidents, had the potential to result in serious injury, fatality, litigation, or loss of income in the case of injured staff, in view of the average size and weight of a PMD. The value placed on PMD use, combined with the potential for injury, are factors that highlight the need to promote safety measures for all residents, staff, and visitors.

Among allied health professionals, occupational therapists have specialist skills in the assessment and training of residents for PMD use. Occupational therapists refer to theoretical frameworks such as the International Classification of Functioning, Disability and Health (ICF) [12] and the Person-Environment-Occupation-Performance model (PEOP) [13] to guide the analysis of individual capacity and to build powered mobility performance skills. Skills assessments, task analysis and the evaluation of the environment for adaptation, are underpinned by these frameworks and are applied by occupational therapists to promote independence in performance of activities of daily living [12,13]. Although several protocols published by international residential settings provided detailed PMD initial assessment, skills monitoring, and steps to discontinuation [6,14–16], no recent papers documenting procedures for managing PMD use in residential aged care settings could be located. A literature search for all available published measures of PMD performance located the following, the Powered Mobility Indoor Driving Assessment (PIDA) [17], the Powered Mobility Community Driving Assessment (PCDA) [18], the Powered Mobility Device Assessment Training Tool (PoMoDATT) [19], the Wheelchair Skills Test (WST) [20], the Obstacle Course Assessment of Wheelchair user Performance (OCAWUP) [21], the Powered Mobility Road Test (PMRT) [22], the Functional Evaluation Rating Scale (FERS) [23], and the Powered Mobility Clinical Driving Assessment (PMCDA) with the co-developed Powered Mobility Screening Tool (PMST) [24]. Review of the eight tools confirmed that none of these assessment or screening tools were suitable without adaptation for use in residential aged care settings. Although the PIDA was designed for use in long-term care facilities [17] considerable time is required to administer the assessment, limiting use in daily practice.

With few permanent occupational therapy positions available within residential aged care in Australia [25], nursing staff oversee PMD use as part of their responsibility for resident safety in everyday activities. Registered nurses use assessment tools to monitor health and well-being, measuring change as it occurs, and providing appropriate referrals as needed. A PMD screening tool could assist nurses as they monitor PMD safety, and where indicated, could prompt referral to an occupational therapist for specialist assessment and/or training. The use of a PMD screening tool by nurses, occupational therapists, or other clinicians within Australia and internationally, in combination with professional guidance in assessment, training, and environmental modification, could provide a multi-pronged approach to improving safe PMD use for all stakeholders.

The Powered Mobility Device Autonomy Residential Screen (PoMoDARS) was developed to address the gap identified for a PMD screening tool in residential aged care, to enable review of PMD capacity and performance skills as aligned with the ICF and PEOP models [12,13]. Scale development should be informed by theory, research, and opinion of users, clinicians, and experts, beginning with determining what needs to be measured, developing a pool of items, and deciding on item format [26,27]. Development of the PoMoDARS was informed by Classical Test Theory [26] and the COnsensus based guidelines for the selection of health Measurement INstruments (COSMIN) [28,29] in addition to findings from the recent incident audit [8]. Expert opinion from PMD specialty experience, theory, and literature review of similar tools were used to generate and assemble the items to be measured and the format for the tool. The items and format were then reviewed by and agreed among the research team and presented for testing in the current study.

This paper outlines the initial validation of the PoMoDARS that was undertaken through testing of the screen with clinicians in the field. The aims of this study were (i) to determine face and content validity of the PoMoDARS based on findings from preliminary testing and review by clinicians, considering item importance, ease of use, clarity, accuracy, and comprehensiveness for use in clinical practice, and (ii) to use the research findings and feedback to modify the PoMoDARS for clinical use as part of the development process.

Materials and methods

Study design

Within the context of a measurement study based on Classical Test Theory and following COSMIN guidelines, a mixed-methods approach [30] was applied to develop and provide initial face and content validity of the PoMoDARS. Face validity is a measure of the usefulness of a test, a subjective measure of whether the test appears to test what is intended [31] and content validity indicates the degree to which the construct of interest is measured by the tool under consideration [31,32]. The study was given ethical clearance by the Human Research Ethics Committees at Federation University. Data collection involved clinicians using the PoMoDARS with a small number of clients, and then participating in a structured interview with closed (numerical) and open (text) responses to gather feedback on the content and format.

Participant selection

A convenience sample was identified for this study through responses to a request for volunteers to test and review the PoMoDARS that was advertised through a national assistive technology Listserv. Inclusion criteria were a professional qualification in nursing, occupational therapy or physiotherapy, a current caseload with suitable clients and willingness to participate. Exclusion criteria included professionals from other disciplines, and not having experience with older people living in residential care who use PMDs. Eight clinicians were selected to participate in the study, six females and two males. Five were registered nurses, two occupational therapists and one physiotherapist, with an average of 14 (SD 9.047) years in practice, ranging from six to 31 years. Participants self-rated their overall experience with residents using powered mobility as minimal (n = 3), moderate (n = 3), advanced (n = 1) and not stated (n = 1). Twenty PoMoDARS screens were completed between the participants, one screen per resident and one to four per participant.

Instruments

The PoMoDARS is an 18-item screening tool divided into two sections; Part 1 Capacity for PMD use and Part 2 Performance and is accompanied by a User Guide. Resident PMD users are scored using items such as general health, cognition, behaviour, severity of prior incidents, and observations made as they perform everyday tasks with their PMD. Residents are scored (1) if autonomous, (2) if experiencing mild difficulty, (3) if experiencing moderate difficulty, and (4) if there are safety concerns. A summed score is then calculated (using the formula provided) providing an indication of the resident’s suitability to safely use a PMD and the amount of supervision they may require. The PoMoDARS screening tool and the PoMoDARS User Guide are available for free download at www.pomodars.com. Basic demographic information was collected from the clinician participants including, professional qualification, gender, years of experience and self-rated PMD experience. A structured feedback interview schedule was developed, using a mixture of questions requiring responses on a 5-point Likert scale and open-ended questions to collect participant feedback on the use of the PoMoDARS [27]. A copy of the Interview Schedule is presented as Appendix 1.

Procedure

Participants were sent a copy of the PoMoDARS and the PoMoDARS User Guide and asked to test the PoMoDARS in their practice. Participants were asked to choose up to six residents who were considering using a PMD, or already had a PMD, and send the completed demographic information (on themselves) and the PoMoDARS screens for their residents, back to the researchers. On completion, an online interview timeslot was scheduled via the Teams or Zoom platforms. Participants were provided with the interview schedule questions ahead of time to enable them to consider their responses. Each interview was video recorded, transcribed verbatim and checked against the video to ensure accuracy.

Data analysis

Quantitative data from the scaled questions were transferred to a raw result excel file from which descriptive statistics were drawn and graphs created to display mean ratings for the importance and the ease of use for each item of the PoMoDARS. Analysis of content validity was conducted using several methods as follows. Content validity index (CVI) calculations were made for the 18 individual PoMoDARS items (I-CVI), such that the number of ratings of 4 and 5 (out of 5) were divided by the total number of ratings awarded by the eight participants. For example, for Item 1 (general health) the seven scores of 4 or 5 were divided by the total of eight participant scores resulting in an I-CVI of 0.88 [33]. To adjust for chance variation, the modified kappa statistic (K*) was applied to the I-CVI calculations using the formula provided [31], where the proportion of agreement expected by chance (Pc) is calculated using N = the number of participants and A = the number of participants awarding a 4 or 5. $$\kappa *=\frac{\text{I-CVI}-p_c}{1-p_c}=p_c=\left[\frac{N!}{A!\left(N-A\right)!}\right]*{.5}^N$$

[31]

To evaluate participant agreement on items for inclusion in the PoMoDARS, an average agreement method (S-CVI/Ave) was used [31,34]. S-CVI/Ave was calculated using the total number of items rated as a 4 or a 5 (133), divided by the total number of ratings (18 items x 8 participants = 144) to obtain the average for the scale [34]. Polit, Beck [35] recommends a S-CVI/Ave of at least 0.90 to achieve ‘excellent’ content validity for a scale as a unit.

Clarity and accuracy of results were evaluated using a content validity ratio (CVR). Krishnaveni and Aravamudhan [36] advocate using the values −1, 0, +1, where −1 indicates that less than half of participants agree, +1 indicates more than half agree, and 0 is half.

Qualitative data from the open-ended questions were analysed using reflexive thematic analysis, where data were coded and themes subsequently generated using an inductive approach to distil information into key themes drawn from the feedback provided by each participant [37,38]. This was initially undertaken by the first author, then the second and third authors reviewed 20% of the data to confirm the coding. Any disagreements were marked, with discussion and rationale provided during a face-to-face online meeting, and an agreement made as to best fit for the data.

Results

The results from the eight participants are presented in relation to their; self-ratings, item importance and ease of use, clarity of descriptors, scoring system and accuracy of the final results, comprehensiveness, participant feedback regarding item inclusion and format, key themes from analysis of participant feedback, and finally, the modifications made to the PoMoDARS following review of the findings.

Participant self-ratings

Participants rated their overall experience in using PMDs with residents as minimal and moderate (as described in the Methods) with only one who rated their experience as advanced. The self-ratings appeared to reflect the amount of specialist knowledge participants felt they had with residents using PMDs. The occupational therapists explained that their specific role was to provide PMD assessment on referral and therefore provided higher ratings. The nurses and physiotherapist were known to have a good understanding of each resident’s everyday PMD skills and safety, referring onward for specialist intervention, and subsequently provided lower ratings of their knowledge.

Item importance

Five-point Likert scales were used to gather participant perception of the importance of each item of the PoMoDARS for PMD use and screening, from ‘not important’ (1) through to ‘very important (5)’. Figure 1 demonstrates the average importance ratings for each item, showing that the eight participants rated 17 items out of 18 (94%) as ‘important’ (4 and above). The lowest average importance scores were 3.75 (SD 1.282) for Item 5 (transfers), followed by 4.25 (SD 0.707) for Item 1 (general health).

Figure 1. PoMoDARS Feedback interview results.

To tally consensus, I-CVI calculations (for ratings of 4 or 5 out of 5) showed that 16 items out of 18 (89%) were rated above the recommended 0.78 (35) to achieve good content validity. Of the two items remaining, Item 5 (transfers) scored 0.63 and Item 8 (substances) scored 0.75. When the Kappa formula was applied to adjust for chance, the results concurred with the I-CVI calculations. The lowest scores were for Item 5, 0.52 and Item 8, 0.72 respectively, rated ‘fair’ and ‘good’ for content validity [33]. These results can be reviewed in Table 1. Consensus for importance of all items together within the overall PoMoDARS screen was tested using an S-CVI calculation, which resulted in a score of 0.92 (above 0.90 is rated as ‘excellent’).

Table 1. PoMoDARS Evaluation Results.

	INDIVIDUAL ITEM IMPORTANCE							EASE OF USE OF ITEMS
ITEM	Mean rating	Standard Deviation	# Ratings 4 /5	% Agreement (4 or 5)	i-CVI 4 or 5	K*c	Evaluation	ITEM NUMBER	Mean rating	Standard Deviation	# Ratings 4 /5	% Agreement (4 or 5)	i-CVI 4 or 5	K*c	Evaluation
1 General Health/Medical	4.25	0.71	7	87.5%	0.88	0.87	excellent	1	4.25	1.04	7	87.5%	0.88	0.87	excellent
2 Visual Function	5.00	0.00	8	100.0%	1.00	1.00	excellent	2	4.75	0.46	8	100.0%	1.00	1.00	excellent
3 Cognitive Function	4.63	0.74	7	87.5%	0.88	0.87	excellent	3	4.25	0.46	8	100.0%	1.00	1.00	excellent
4 Motor Control	4.50	0.76	7	87.5%	0.88	0.87	excellent	4	4.25	1.04	7	87.5%	0.88	0.87	excellent
5 Transfers	3.75	1.28	5	62.5%	0.63	0.52	fair	5	4.25	1.04	7	87.5%	0.88	0.87	excellent
6 Behaviour	4.88	0.35	8	100.0%	1.00	1.00	excellent	6	4.25	1.04	7	87.5%	0.88	0.87	excellent
7 Medication	4.50	0.54	8	100.0%	1.00	1.00	excellent	7	2.88	1.13	3	37.5%	0.38	0.21	poor
8 Substances	4.50	0.93	6	75.0%	0.75	0.72	good	8	4.25	1.17	6	75.0%	0.75	0.72	good
9 PMD Incident History	4.50	0.76	7	87.5%	0.88	0.87	excellent	9	4.12	1.13	6	75.0%	0.75	0.72	good
10 Ability to Go	4.88	0.35	8	100.0%	1.00	1.00	excellent	10	4.63	0.52	8	100.0%	1.00	1.00	excellent
11 Ability to Stop	5.00	0.00	8	100.0%	1.00	1.00	excellent	11	4.62	0.52	8	100.0%	1.00	1.00	excellent
12 Attention & Alertness	5.00	0.00	8	100.0%	1.00	1.00	excellent	12	4.38	0.52	8	100.0%	1.00	1.00	excellent
13 Giving way	4.88	0.35	8	100.0%	1.00	1.00	excellent	13	4.38	0.52	8	100.0%	1.00	1.00	excellent
14 Use of Speed Controls	4.63	1.06	7	87.5%	0.88	0.87	excellent	14	4.25	0.46	8	100.0%	1.00	1.00	excellent
15 Navigating Spaces	5.00	0.00	8	100.0%	1.00	1.00	excellent	15	4.63	0.52	8	100.0%	1.00	1.00	excellent
16 S Posture & Tolerance	4.63	0.52	8	100.0%	1.00	1.00	excellent	16	4.75	0.46	8	100.0%	1.00	1.00	excellent
17 Switching OFF	4.50	0.76	7	87.5%	0.88	0.87	excellent	17	4.50	0.76	7	87.5%	0.88	0.87	excellent
18 Recalling functions	5.00	0.00	8	100.0%	1.00	1.00	excellent	18	4.25	0.71	7	87.5%	0.88	0.87	excellent

Key: #: number of; % Agreement: the % of participants who scored the item with a 4 or 5; i-CVI: individual content validity index score; K*c: modified kappa calculation sourced from Chapter 10, Supplement 1 (31).

Ease of use

Participants were asked how easy each individual item of PoMoDARS was to score, from ‘very difficult’ (1) through to ‘very easy’ (5). Figure 1 shows the average ratings for each item. Participants scored 17 items out of 18 (94%) ‘easy’ (4) or above, however, Item 7 (medication) was scored 2.875 (SD 1.126), with scores ranging from ‘very difficult’ (1) to ‘easy’ (4). Consensus for ease of use for all items together within the overall PoMoDARS screen was tested using an S-CVI calculation, which resulted in a score of 0.90 (‘excellent’).

Clarity of the descriptors

To consolidate feedback on comprehensibility of the PoMoDARS, participants were asked if the item descriptors for part 1 and for part 2 were easy to understand. Results showed that 100% of participants scored the descriptors for both parts as ‘easy’ (4) or ‘very easy’ (5) to understand.

Clarity of the scoring system

Five-point Likert scale scores were used to rate the ease of use of the scoring system for the PoMoDARS. Results showed that seven out of eight participants (88%) rated the PoMoDARS scoring system as ‘easy’ (4) or ‘very easy’ (5) with one participant scoring ‘a little difficult’ (2). Despite a positive perception for ease of use for the scoring system, only two participants completed the scoring for the PoMoDARS correctly, with at least one error made by the other six participants, in simple addition or manually applying the formula.

Clarity and accuracy of final results

After completion and scoring of the PoMoDARS, resident results can be interpreted using an outcomes table as provided in the User Guide. The outcomes table connects the score achieved on the PoMoDARS with a corresponding suggested or anticipated level of supervision the resident will require whilst using the PMD. Participants were asked to rate the ease of use of the outcomes table. Seven out of eight (88%) participants responded with ratings of ‘easy’ (4) or ‘very easy’ (5).

When asked whether the outcomes table was accurate for their residents, six out of eight (75%) participants agreed the outcomes table was accurate. Two participants reported a slight difference which was described as ‘one percentage’ or ‘one category difference’, or ‘2 out of 3’ clients accurate. A content validity ratio (CVR) of +1 indicated a perception of accuracy for the outcome of the PoMoDARS.

Comprehensiveness

To determine comprehensiveness of the PoMoDARS, participants were asked ‘Does this tool cover enough?’ to which seven out of eight (88%) participants agreed. The eighth participant voiced concern that the ability to reverse a PMD (travel backwards) was not itemised within the performance section of the PoMoDARS. Although not itemised, the research team considered that the skill of reversing a PMD was present for observation within the functional task of navigating narrow spaces (Item 15) such as in residential care bedrooms and bathrooms. However, it was agreed that adding ‘reverse’ to the text in Item 15 to highlight the ability to reverse a PMD was a practical modification to guide assessors. To complete the analysis of comprehensiveness of the PoMoDARS, a content validity ratio (CVR) was applied to determine consensus and a + 1 majority was achieved.

Participant feedback regarding items and format of the PoMoDARS

Open text responses were analysed using reflexive thematic analysis [37,38]. The feedback data provided for the 18 items of the PoMoDARS were coded and collated to generate key ideas regarding item selection and format with a view to informing modifications. Following analysis, the feedback for Item 1 of the PoMoDARS (general health) indicated that half of all participants agreed that multiple diagnoses were of concern for PMD use, however, there were greater concerns for the type of diagnosis and whether it was stable or fluctuating or progressive in nature and what impact it would have on PMD use. Feedback for Item 2 (visual function) highlighted some confusion regarding the origin of a visual deficit and whether medical or ophthalmological input was needed, whilst other participants reported being comfortable to rate visual status based on diagnosis and observation of function. Item 3 (cognitive function) showed good agreement on the importance of cognition to PMD use. Some participants liked the inclusion of standardised screens of cognitive function to assist in scoring the item, whilst others acknowledged reluctance, citing potential for cognitive screens to be influenced by individual funding model biases, and affecting access to a PMD. Item 4 (motor function) raised concern among participants who considered use of the hand too narrow a focus, potentially excluding alternatives such as head or foot control. Some participants did not see the relevance or were unsure how to score Item 5 (transfers), citing uncertainty for whether the focus was positioning in the seat or falls risk. Item 6 (behaviour) was generally viewed as important and easy to rate. Feedback for Item 7 (medication) indicated that the lower ratings were due to the time and effort required to locate medication charts, look up effects and discuss with the team. Item 8 (substances) was considered by some participants more difficult to assess as may be concealed or if scored by inexperienced staff. There was uncertainty for whether the item should include past overuse or only current overuse of any substance which may impair ability to control a PMD. Similarly, for Item 9 (incident history) participants indicated that a resident may conceal incidents or injuries, or the records may not be clear or easy to review. In addition, difficulties with the scoring formula related to this item were raised.

In open text responses from the performance section (part 2) of the PoMoDARS, participants highlighted some areas for clarification. For example, terms used in the descriptors to measure the construct such as, mild, moderate, sometimes, mostly, or always, may have been difficult to differentiate in items such as attention and alertness; use of speeds; turning off; and recall for control unit functions. Several participants felt that a 24-hour observation period would improve ease of scoring items such as, going, stopping, attention and alertness, giving way to ambulant pedestrians, use of speeds, and navigating narrow spaces. Some participants mentioned that nurses may not have the opportunity to observe residents outdoors, potentially increasing the challenge to score items such as use of speeds, whilst others mentioned they thought speed was limited in residential facilities anyway. These findings contributed to small but important modifications to the final version of the PoMoDARS.

Key themes from analysis of participant feedback

During the interview, participants were asked which behaviours may indicate a resident is unsafe to continue using a PMD. This question provided a cross check to ensure the items chosen for the PoMoDARS included all safety concerns of the participants. Reflexive thematic analysis showed a strong alignment of themes after coding and interpretation [37,38]. The primary safety concern, and key theme, raised by participants was a perceived risk of danger for the resident to cause injury to self or others. Cognitive impairment was raised most frequently in association with the risk of danger, with multiple references to insight, level of alertness and confusion. Associated with the concern for danger of injury were themes of change or progression of medical condition, with participants emphasising concern for the effect on ability to control the PMD. Ability to control the PMD was in turn linked to issues of behaviour, medication, and potential substance use, with comments relating back to the potential to impair resident alertness for safe use of a PMD. Each of these themes raised by participants as safety concerns were represented within the 18 items of the PoMoDARS with nothing further to include. This result reinforced consensus among participants for the face and content validity of the PoMoDARS.

Modifications to the PoMoDARS

The second aim of this study was to use participant feedback to make modifications to improve item relevance, ease of use, clarity, and accuracy of the PoMoDARS for clinical practice. To achieve this aim, items were adjusted in accordance with specific participant feedback and in alignment with the key themes. These changes were discussed, tested, and agreed among the research team to create the final version of the PoMoDARS. Table 2 provides a summary of the modifications made. The final version of PoMoDARS is presented at www.pomodars.com

Table 2. Research-based modifications to the PoMoDARS.

Item	Resulting grade or issue	Modifications made to PoMoDARS
1. General health	Confusion with measurement terms	Terms ‘good’ and ‘fair’ health replaced to focus on speed of change of health/ medical condition for PMD use
3. Cognitive function	Which to use if there are two very different cognitive screen scores on file? What if cognitive screen is exaggerated for funding?	Note added to User Guide to address which score to use Note added to advise assessor to ensure the score accurately describes current status or to score using the descriptor in preference
4. Motor function	Participants were split as to whether motor control should specify upper limb	Descriptors modified to include motor control with any limb and choices reduced to three: smooth control, altered or uncontrolled
5. Transfers	‘Fair’ for item importance	Descriptors modified to improve clarity for relevance, and choices reduced to three: stable transfers, risk of falls or recurrent falls
6. Behaviour	Minor wording, issue with observation time, issue with possible exaggeration for funding	Descriptors simplified to improve item clarity, observation period added to item heading and caution re exaggeration for funding added to User Guide
7. Medication	‘Poor’ for ease of use	Descriptors simplified to focus on level of alertness needed for safe use of PMD rather than medication variables and options reduced to three
8. Substances	Uncertainty for grading overuse	Descriptors reduced to a choice of two: evidence or no evidence of overuse
9. PMD Incident History	Scoring errors	Observation period for incident history reduced. Clarity of instructions improved. Visual cues added. Auto-score form initiated
14. Use of speed controls	Minor wording	Descriptors modified to improve item clarity
16. Sit posture & tolerance	Minor wording	Descriptors modified to improve item clarity
17. Switching off	Minor wording	Descriptors modified to improve item clarity
18. Recall for control unit functions	Minor wording	Descriptors modified to improve item clarity
What’s missing?	1x query for ‘reverse’	‘Reverse’ included in item 15 heading

Discussion

The PoMoDARS was created to fill a gap to provide clinicians with a simple tool, aligned with the ICF [12] and the PEOP models [13], to screen the capacity and performance of older adults to use a PMD safely in residential aged care. Preliminary face and content validity were established and minor modifications to the screen were made based on the feedback received from participants.

Initial validation of the PoMoDARS

In this study, face validity was examined through the responses of participants concerning the comprehensiveness and perceived accuracy of the PoMoDARS. Complementary calculation methods were used, and this established consensus that resident capacity and performance skills for safe use of a PMD are measured by the PoMoDARS. With a single missing item included and accuracy approved by the majority, preliminary face validity for the PoMoDARS was established, in the residential aged care setting.

The high level of consensus among participants confirmed agreement for the importance and thus inclusion of the items (content) chosen for the PoMoDARS. The analysis of feedback assisted to fine-tune clarity and format of the items for use in clinical practice. For clinical utility, a screening tool must be comprehensible, with all items detailed in simple language, fostering a perception of clarity and ease of use for clinicians [39]. Comprehensibility of the PoMoDARS was established with participant consensus for ease of use and clarity of descriptors and it was improved with simple modifications to several items based on the feedback provided in the open text responses from participants. On completion, all items were considered valid for the process of screening PMD capacity and performance skills, therefore, initial content validity was established.

Creating the final version of the PoMoDARS

The results of content validity calculations were corroborated by rich qualitative data collated and interpreted using reflexive thematic analysis, highlighting small but necessary modifications for the PoMoDARS. Several items required adjustment to clarify descriptors, and the instructions for the scoring system needed simplification and improvement in structure and format. Based on feedback the following changes were made. The descriptors for Item 5 (transfers) were simplified and reduced to three choices: (i) stable transfers including either ambulant or hoist transfers; (ii) instability during transfers presenting increased falls risk or (iii) recurrent falls to indicate unsafe function.

Results showed that medication or substances affecting mental alertness were linked by participants to increased risk during PMD use. Item 7 (medication) was highly rated for importance and lowest for ease of use related to complexity. Feedback highlighted the importance of alertness for PMD use and concern for the effect of sedation, regardless of whether the medication was for pain, mood, or behaviour. Consequently, the scoring choice was streamlined to three options and focus was moved from medication strength to effect on alertness. Similarly, Item 8 (substances) was simplified for assessors to decide whether there was evidence of overuse or not, thus reducing the need to decide between degrees of substance use which may have been too subjective. The User Guide was updated to confirm for assessors that current overuse rather than past overuse was to be rated.

Three further items required simple modifications. The descriptors for Item 1 (general health) were fine-tuned to focus upon fluctuation or speed of change of condition rather than diagnosis, as consistent with our findings and those of Mortenson, Miller [40]. Within the descriptors for Item 3 (cognitive function) observation of the resident was placed before cognitive screen result and a note was placed in the User Guide to prompt assessors to consider which score to use if several cognitive scores are on file. For Item 4 (motor function), responses were divided as to whether control options other than those used with the upper limb were needed. It was decided that inclusion of alternate controls such as head, chin, or foot would be more inclusive and improve the potential to use the PoMoDARS in the wider residential disability community in the future.

Despite a positive perception for ease of use of the scoring system and accuracy of the outcomes table, errors in simple addition and use of the formula were made. Errors were most often related to the scoring for Item 9 (incident history) where the choice between new versus existing PMD users prompted use of a different denominator for calculation of the score. Improvements to Item 9 included narrowing the record period for incidents to six months, simplifying the language in the descriptors, highlighting the scoring instructions for new users with no history and using visual cues such as greyed out scoring boxes. The instructions were also highlighted in the User Guide and construction of an automatic scoring form for the PoMoDARS was initiated. The addition of automatic scoring through use of an Excel formula meant that clinicians did not have to undertake any calculations and after entering the score for each item, the overall result would be immediately and accurately generated. On completion of these modifications, each item of the PoMoDARS was considered relevant to the construct of interest, easy to use with the population and setting of use, no key aspect was missing, and overall comprehensibility was supported.

Limitations and future directions

Insufficient variance due to high agreement among the eight participants on a narrow range of ratings of the PoMoDARS meant intraclass correlation coefficient values could not be calculated to confirm validity. However, use of the complementary content validity calculations and COSMIN framework proved valuable [29,31]. Pressure upon residential aged care settings due to the occurrence of Covid-19 during the data collection period of this study was a limiting factor for recruitment opportunities, therefore, further validation studies with larger samples are recommended. Consistent with Classical Test Theory, research is also required to establish the reliability of the PoMoDARS, including test re-test, interrater and intrarater reliability [31].

Conclusion

Based on Classical Test Theory [26] and drawing on the COSMIN framework [29], this research provides initial face and content validation for the PoMoDARS. The mixed-methods approach adopted in this study combined numerical and qualitative data to clarify perception of item importance, ease of use, clarity, accuracy, and comprehensiveness for the PoMoDARS as a screening tool for PMD use in residential aged care. Consensus was consistently high and items with lower ratings were identified and modified utilising the feedback from participants. The PoMoDARS is ready for additional research to further confirm validity of the tool and to investigate reliability. Use of the PoMoDARS is intended to prompt discussion and multidisciplinary team collaboration around the critical contribution of powered mobility for older adults living in residential care, to acknowledge the importance of choice and control and to create a balance of support and safety measures.

Author contributions

All authors have been involved in the study implementation and have been involved in the decision to prepare and submit the article for submission and accept responsibility and be accountable for the publication.

Ethical approval

This study was given ethical clearance by the Human Research Ethics Committees at Federation University (B21-091).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability Statement

The data that support the findings of this study are available on request from the corresponding author.

Appendix 1

Table A1. Feedback interview schedule.

Relating to Part 1: items 1-9
(Example rating scales are shown in questions 1 A and 1B and removed thereafter)
1A: How important is a ‘general health’ rating for the screening tool & PMD use?
(circle one)
1B: How easy was it to rate ‘general health’ using the screening tool?
If difficult, please describe:
1 C: If a resident has ‘multiple diagnoses’ will it affect PMD use? Yes ☐  No ☐
Please explain your choice:
2 A: How important is a ‘visual function’ rating for the screening tool & PMD use?
2B: How easy is it to rate ‘visual function’ for the screening tool?
If difficult, please describe:
3A: How important is a ‘cognitive function’ rating for the screening tool & PMD use?
3B: How easy is it to rate ‘cognitive function’ for the screening tool?
If difficult, please describe:
4A: How important is a ‘Motor function (Upper Limb)’ rating for the screening tool & PMD use?
4B: How easy is it to rate ‘Motor function (Upper Limb)’ for the screening tool?
If difficult, please describe:
5A: How important is a ‘Transfers’ rating for the screening tool & PMD use?
5B: How easy is it to rate ‘Transfers’ for the screening tool?
If difficult, please describe:
6A: How important is a ‘Behaviour’ rating for the screening tool & PMD use?
(altered behaviour e.g. impulsive/risk-taking/agitation/depression/anxiety)
6B: How easy is it to rate ‘Behaviour’ for the screening tool?
If difficult, please describe:
7A: How important is a ‘Medications’ rating for the screening tool & PMD use?
7B: How easy is it to rate ‘Medications’ for the screening tool?
If difficult, please describe:
7C: In your experience, which medication groups affect ability to control a PMD safely?
(e.g. pain relief, sedation, mood, movement disorder, hypertension, fluid reduction)
LIST:_______
8A: How important is a ‘Over-use of Substances’ rating for the screening tool & PMD use?
8B: How easy is it to rate ‘Over-use of Substances’ for the screening tool?
If difficult, please describe:
8C: Any further comments on the issue of over-use of substances related to PMD use?
9A: How important is a rating for ‘history of incident or injury related to PMD use’ for the screening tool & PMD use? (includes impact to self, to others and/or property in the preceding 12 months)
9B: How easy is it to rate ‘history of incident or injury related to PMD use’ for the screening tool?
If difficult, please describe:
9 C: How did you identify and/or confirm whether your resident had sustained an incident?
9D: How easy is the wording in Part 1 of PoMoDARS to understand?
Please note any wording that needs improvement:
Relating to Part 2, items 10-18
10 A: How important is it to rate ‘ability to GO’ for the screening tool & PMD use?
(i.e. start, travel straight & turns)
10B: How easy is it to rate ‘ability to GO’ for the screening tool?
If difficult, please describe:
11A: How important is it to rate ‘ability to STOP’ for the screening tool & PMD use?
(i.e. on command & for obstacles)
11B: How easy is it to rate ‘ability to STOP’ for the screening tool?
If difficult, please describe:
12A: How important is it to rate ‘attention & alertness’ for the screening tool & PMD use?
12B: How easy is it to rate ‘attention & alertness’ for the screening tool?
If difficult, please describe:
13A: How important is it to rate ‘giving way to ambulant pedestrians’ for the screening tool & PMD use?
13B: How easy is it to rate ‘giving way to ambulant pedestrians’ for the screening tool?
If difficult, please describe:
14 A: How important is it to rate ‘use of speed controls’ for the screening tool & PMD use?
14B: How easy is it to rate ‘use of speed controls’ for the screening tool?
If difficult, please describe:
15A: How important is it to rate ‘navigating narrow spaces’ for the screening tool (& PMD use)?
(i.e. doorways, bed/bathroom, path)
15B: How easy is it to rate ‘navigating narrow spaces’ for the screening tool?
If difficult, please describe:
16 A: How important is it to rate ‘sitting posture & tolerance’ for the screening tool & PMD use?
16B: How easy is it to rate ‘sitting posture & tolerance’ for the screening tool?
If difficult, please describe:
17 A: How important is it to rate ‘switching OFF when not moving’ for screening tool & PMD use?
17B: How easy is it to rate ‘switching OFF when not moving’ for the screening tool?
If difficult, please describe:
18A: How important is it to rate ‘recall for control unit functions’ for the screening tool & PMD use?
(i.e. steering, on/off, speed, tilt)
18B: How easy is it to rate ‘recall for control unit functions’ for the screening tool?
If difficult, please describe:
19: Are the descriptors for Part 2 of PoMoDARS easy to understand?
Please note any wording that needs improvement:
20: How easy was it to use the overall scoring system for PoMoDARS?
What, if anything, would you change?
21: How easy to use was the outcomes table for PoMoDARS?
Did the outcome seem accurate for the resident?
22: How long did it take to conduct the PoMoDARS screen?
23.In your experience, are there deal breaker behaviours that indicate a resident is unsafe to continue PMD use?
24: Does this tool cover enough?
25: Have you used any other PMD screening tool in residential care?
26: Do you have any further comments to make, e.g. your experiences, thoughts on relevance of a tool like this in residential aged care, resident comments, or findings not otherwise recorded?