Further Evaluation of the Validity and Reliability of the Performance Diagnostic Checklist-Human Services

ABSTRACT

The Performance Diagnostic Checklist-Human Services (PDC-HS) is primarily an informant-based assessment tool designed to help consultants identify variables contributing to problematic employee performance and formatted specifically for use in human service settings. In an evaluation of the tool’s validity and reliability, a previous study used a series of video vignettes to simulate a consultant using the PDC-HS to troubleshoot performance problems; participants scored each video twice, providing measures of validity, test–retest reliability, and interrater reliability. Although the results suggested that the PDC-HS was valid and reliable, the vignettes may have lacked external validity. In the current study, we replicated and extended previous research to include video-based scenarios from actual consultant interviews using the PDC-HS. Twenty-one participants scored three videos, each with a different set and number of domains indicated for intervention. Approximately one month later, participants scored the same videos to assess test–retest reliability. Results suggest that participants were largely able to identify the problematic PDC-HS domains and that the tool was generally reliable, although scores were somewhat lower than those reported by in previous research. Results also suggest that the tool is most effectively implemented by a consultant with at least some background in behavior analysis.

KEYWORDS:

• Interrater reliability
• performance Diagnostic Checklist-Human Services (PDC-HS)
• test re-test reliability
• validity

Acknowledgments

The authors would like to thank Benjamin Fagan for his assistance with data collection.

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplementary material

Supplemental data for this article can be accessed on the publisher’s website.