Global Rating Scale for the Assessment of Paramedic Clinical Competence

Abstract

Objective. The aim of this study was to develop and critically appraise a global rating scale (GRS) for the assessment of individual paramedic clinical competence at the entry-to-practice level. Methods. The development phase of this study involved task analysis by experts, contributions from a focus group, and a modified Delphi process using a national expert panel to establish evidence of content validity. The critical appraisal phase had two raters apply the GRS, developed in the first phase, to a series of sample performances from three groups: novice paramedic students (group 1), paramedic students at the entry-to-practice level (group 2), and experienced paramedics (group 3). Using data from this process, we examined the tool's reliability within each group and tested the discriminative validity hypothesis that higher scores would be associated with higher levels of training and experience. Results. The development phase resulted in a seven-dimension, seven-point adjectival GRS. The two independent blinded raters scored 81 recorded sample performances (n = 25 in group 1, n = 33 in group 2, n = 23 in group 3) using the GRS. For groups 1, 2, and 3, respectively, interrater reliability reached 0.75, 0.88, and 0.94. Intrarater reliability reached 0.94 and the internal consistency ranged from 0.53 to 0.89. Rater differences contributed 0–5.7% of the total variance. The GRS scores assigned to each group increased with level of experience, both using the overall rating (means = 2.3, 4.1, 5.0; p < 0.001) and considering each dimension separately. Applying a modified borderline group method, 54.9% of group 1, 13.4% of group 2, and 2.9% of group 3 were below the cut score. Conclusion. The results of this study provide evidence that the scores generated using this scale can be valid for the purpose of making decisions regarding paramedic clinical competence. Key words: educational measurement; clinical competence; licensure; certification; paramedics; global rating scale; rating scales

Appendix 1. Case Summary

Overview

This case involved a paramedic (i.e., the candidate) working alone and responding to the side of a roadway for a patient with a decreased level of consciousness who was in the rear of a transfer company ambulance.* According to transfer company staff, the patient’s condition began with severe shortness of breath secondary to congestive heart failure that progressed into lethal arrhythmia and eventually cardiac arrest. The transfer company staff are “on scene” (i.e., in the rear of the transfer company vehicle with the patient) arguing over who is responsible for the current predicament.

Call Information

Call for a 75-year-old male/female with shortness of breath.

Case Details

The patient [manikin] presented initially as responding only to painful stimuli with moans, was diaphoretic and tachypneic. His presenting rhythm was ventricular tachycardia. Presenting vital signs were: blood pressure: 68/48 mmHg, heart rate: 190 beats/ min (ventricular tachycardia), respiratory rate: 30 breaths/min shallow and regular (crackles throughout), and blood sugar of 8.8 mmol/L. The patient had a history of Alzheimer's disease, coronary artery disease, two previous myocardial infarctions, congestive heart failure, cerebrovascular accident (no lasting deficits), hypertension, diabetes, and high cholesterol level. The medication list included Aricept, metoprolol, digoxin, lisinopril, Glucophage, and atorvastatin, and the patient was allergic to morphine.

Appendix 2. Global Rating Scale for the Assessment of Paramedic Clinical Competence

Notes

*In Canada, some unregulated private companies may provide transfer services to patients. Generally, these unregulated transfer companies may not be held to the same standard as fully regulated ambulance services, including staff qualifications. Further, unregulated transfer companies are not authorized to transport patients directly to emergency departments.