Inter-rater reliability of the Play Assessment for Group Settings

Abstract

Play Assessment for Group Settings (PAGS) was originally designed to measure the play ability of 2- to 8-year-old children in their natural day-care context and for the use of different professionals. The main purpose of this study was to examine the inter-rater reliability of the PAGS. Twelve raters from different professional backgrounds scored a total of 78 children, both videotaped co-scored cases and individual live cases. The many-faceted Rasch model (MFR) was used to examine the goodness-of-fit of the rater severity estimates. Of the 12 raters, 91.7% were demonstrated to fit the MFR model expectations. Overall the results of this study indicated that the raters scored the PAGS in a reliable way, and that they behaved like independent experts. The separation index (2.09) of the raters indicated that the inter-rater scorings were comparable with each other for practical purposes. Further research on the different response processes of children in different diagnostic groups is needed.

Key words::

• Play ability assessment
• test development
• inter-rater reliability
• many faceted Rasch model
• MFR

Acknowledgements

The authors would like to warmly thank all the raters, children, and their parents who committed to this study. They would also like to thank Michael Freeman for checking the language of this manuscript. This study was supported by grants from the Finnish Association of Occupational Therapists.