Inter-rater agreement when linking stroke interventions to the extended international classification of functioning, disability and health core set for stroke

Abstract

Purpose

To uncover the factors that influence inter-rater agreement when extracting stroke interventions from patient records and linking them to the relevant categories in the Extended International Classification of Functioning, Disability and Health Core Set for Stroke.

Method

Using 10 patient files, two linkers independently extracted interventions and linked the target of the intervention to relevant functions in the ICF. The percentage agreement of extracted interventions and the ICF codes was calculated. Non-matching interventions and codes were further analysed to determine the reasons for poor agreement.

Results

A total of 518 interventions were extracted, with 44.01% agreement between the two linkers. Of the non-agree codes and interventions, 43.79% were due to mismatched ICF codes and 56.20% were due to mismatched interventions. Differences were due to linkers (a) extracting interventions from different parts of the patient note (b) differences in interpreting the target of the intervention, and (c) choosing a different code with similar meaning.

Conclusion

Greater reliability when linking interventions to ICF codes can be achieved by; health services using a consistent progress note that uses ICF language, recording the intervention aim, linkers knowing the aims of each discipline’s interventions and using multiple reliability checks and analysis to inform the linking method.

Implications for rehabilitation

• Linking intervention targets to the ICF and to the ICHI is an emerging research field.

• Development of trustworthy inter-rater reliability methods is needed to achieve its potential to demonstrate the equity, quality and effectiveness of interventions.

• Independent linking of patient notes to the ICF can identify factors that impact inter-rater reliability.

• When writing patient notes, health professionals should use a consistent format that identifies the functional target of the intervention using ICF terms.

Keywords:

• Inter-rater reliability
• linked data
• intervention
• stroke rehabilitation
• international classification of functioning
• disability and health

Disclosure statement

The authors report no conflicts of interest.