Establishment of performance standards and a cut-score for the Canadian Forces Firefighter Physical Fitness Maintenance Evaluation (FF PFME)

Abstract

The Bookmark method for setting cut-scores was used to re-set the cut-score for the Canadian Forces Firefighter Physical Fitness Maintenance Evaluation (FF PFME). The time required to complete 10 tasks that together simulate a first-response firefighting emergency was accepted as a measure of work capacity. A panel of 25 Canadian Forces firefighter supervisors set cut-scores in three rounds. Each round involved independent evaluation of nine video work samples, where the times systematically increased from 400 seconds to 560 seconds. Results for Round 1 were discussed before moving to Round 2 and results for Round 2 were discussed before moving to Round 3. Accounting for the variability among panel members at the end of Round 3, a cut-score of 481 seconds (mean Round 3 plus 2 SEM) was recommended. Firefighters who complete the FF PFME in 481 seconds or less have the physical capacity to complete first-response firefighting work.

Abstract

Practitioner Summary: Task-simulation tests are commonly used to evaluate physical fitness for duty in emergency response occupations such as firefighting. This paper describes the novel use of the Bookmark method where a panel of expert judges approved performance standards and set a cut-score for physical fitness of Canadian Forces firefighters.

Keywords::

• firefighting
• task simulation test
• standard setting
• Bookmark method

Notes

This work was supported by the Canadian Forces Fire Marshall, Department of National Defence, Canada.