Engaging the hearts and minds of clinicians in outcome measurement – the UK rehabilitation outcomes collaborative approach

Figures & data

Table I.  Key criteria for tools for inclusion within a national data set.

Criterion	Review question
Validity	Does the tool measure what in purports to measure?• Face validity – in measuring the thing of interest• Content validity – domains of interest are adequately represented• Construct validity – relationships with other measures
Reproducibility	Do repeated applications of the measure produce the same results?• Intra-rater reliability, intra-rater reliability (repeatability)
Scaling properties	Can individual item scores reasonably be added into a single total score?• If not, how should they be grouped?To what extent does the scale provide interval level data?
Feasibility	Is the tool easy to apply in the clinical context?
Responsiveness	Does the tool detect changes when these occur?• And does it remain stable when there is no change?
Interpretability	Do clinicians understand what the output from the tool means?
And …
Engagement	Do clinicians actually want to use the tool?

Table II.  Overview of UKROC data set.

Domain	Content
Demographics	Age, gender, ethnicity, funding authority etc.
	Diagnosis (ICD 10 code). Casemix category (HRG v 4 code)
Process	Response times – referral to admission
	Source of admission, interruption to treatment
	Length of stay
	Discharge destination
Needs (Complexity)	Rehabilitation Complexity Scale
Inputs	Northwick Park Dependency Scales:
	• Nursing Dependency and Care Needs Assessment
	• Therapy Dependency Assessment
Outcomes	Barthel Index
	FIM or UK FIM + FAM (+Neurological Impairment Set)
	Goal attainment scaling (GAS)

HRG, Healthcare Resource Group (The UK casemix classification is currently in version 4); ICD 10, International Classification of Disease version 10; FIM, Functional Independence Measure; UK FIM + FAM, UK Functional Assessment Measure.

Table III.  Common barriers and possible solutions to clinician engagement in outcome measurement.

Barrier	Problems	Possible solutions proposed by the UKROC programme
Time	• Clinicians are increasingly hard pressed for time.• Recording of outcome measures is typically deferred to the end of the day, relying on the clinicians to stay on after hours.• Tools which require multidisciplinary scoring are particularly problematic because of the extra time needed to get clinicians together.	• Tools must be as timely as possible to complete.• If more complex tools are used, the additional benefits should justify the time invested to complete them.• There should be dedicated time for completion.• For measures that require multidisciplinary scoring, time should be allocated during routine multidisciplinary team meetings, to avoid the necessity of extra meetings.
Priority, leadership, and support	• The priority for clinicians will always be clinical care.• There is often a lack of leadership and drive to prioritise outcome measurement.	• Strong leadership is required to ensure that measures are completed.• This is a role for the consultant or other senior team member with string managerial influence. It should not be delegated to junior staff or external personal (such as research staff) – the drive must come from within the team itself.
Admin support and computerization	• Administration support is often lacking so that computer entry is left to the clinical staff.• Computer software is often unfriendly, leading to inaccurate data entry.	• Administration support should be provided to enter the data.• User-friendly data entry tools are required to ensure accurate completion, and staff who enter the data should have adequate training.
Relevance and usefulness	• Outcome measures may seem to have little relevance to the client group or the objectives for rehabilitation.• The tools are seen as just another chore, with no benefit to the team to reward their efforts in completion.	• A range of outcome tools is required to ensure that the rehabilitation objectives for the group are reflected.• Tools should offer added benefits that assist the clinician in their daily practice/clinical decision-making, so that they are perceived to be relevant.
User friendliness	• Complicated, poorly presented tools can be off-putting.• In particular, those with complicated scoring schemes and formulae are often poorly understood by clinicians.	• Tools should be presented in a user-friendly manner – for example in verbal form that the user (e.g. clinicians, or patient for self-report tools) can relate to.• The application of scores and formulae may then be applied when information is computerised.
Training and understanding	• Clinicians are often required to complete outcome measures with little or no training.• They often have little understanding of the use the data are put to, and are afraid that they will be misinterpreted.	• Training in the use of outcome measures should be provided at a level appropriate to the individual’s role.  • Clinicians on the ground require training in the application of the tool.  • Senior staff should have a clear understanding of how to interpret the data that derives from the measure.

Figure 1.  Example of a FAM-Splat for a patient with traumatic brain injury. The FAM-Splat provides graphic presentation of the disability profile in a radar chart. The 30 items are arranged as spokes of the wheel and the levels from 1 (total dependence) to 7 (total independence) run from the centre outwards. Thus a perfect score would be demonstrated as a large circle. The shaded area shows the change between admission and discharge for each item. The dotted line indicates where a goal score set at admission was not achieved.