Interrater Reliability and Content Validity of a Minimal Data Set to Measure Outcomes of Students Receiving School-Based Occupational Therapy and Physical Therapy

Abstract

Background and Purpose: The 1997 Amendments of the Individuals with Disabilities Education Act increased the emphasis on accountability for outcomes of students with disabilities in the United States. The purpose of this study was to determine the content validity and estimate the interrater reliability of the School Outcomes Measure, a new minimal data set to measure groups of students' performance in the areas addressed by school-based therapy. Method: Forty-five occupational therapists and physical therapists working in Oklahoma's public schools participated in the content validity study. Eight occupational therapist-physical therapist pairs rated 17 students for the interrater reliability study. Therapists were recruited and data were collected by mail for both studies. Results: Most validity study participants thought the tool was comprehensive and clear, and that they could complete the tool twice a year for their students. The weighted kappa statistic indicated that the therapists' agreement was substantial (.73) to almost perfect (.96) for 25 of the 28 student functional status items. Discussion: The results support the content validity and interrater reliability of the items. This study was an initial step in development of the tool and further studies are needed in the validation process.

Key Words:

• Content validity
• interrater reliability
• minimal data set
• outcomes
• school-based therapy

Additional information

Notes on contributors

Irene R. McEwen

Michael Lepp, PT, is in private practice, Associate Instructor of the Upledger Institute, and Clinical Instructor, Case Western Reserve University School of Dentistry, Cleveland, OH.

Marie McKenzie, MS, OTR, is in private practice. She was Coordinator of Occupational Therapy at Scottish Rite Hospital at the time of this study.

Heather Miller, MS, OTR/L, is currently employed by Madison County Educational Service Center, providing therapy services to the county's pre-schools. This research was completed in partial fulfillment of the requirements of an MS degree at Ohio State University.

Linda Viscardis, BA, is Team Leader and Community Development Worker, Peterborough Family Enrichment Centre. She is Co-Founder of P.R.O.S.P.E.C.T.S., a support and advocacy group for families who have children with special needs.

Debra Stewart, BSc, OT(C), is Occupational Therapist, Erinoak, Serving Young People with Physical Disabilities, Mississauga, ON, and Clinical Lecturer, School of Rehabilitation Science, McMaster University.

Janette McDougall, MA, was Project Coordinator

Kip Husted, BEd, was a parent representative on the study team.

Aviva Yochman, MA, OTR, was a student in the School of Occupational Therapy, Faculty of Medicine, The Hebrew University of Jerusalem, Israel at the time of this study.

Imelda Burgman, MA OTR/L BCP, completed her MA in Movement Sciences at Teachers College, Columbia University, and was an Occupational Therapist at The Shield Institute, Bronx, NY. She is currently Lecturer, Habilitation, School of Community Health, Faculty of Health Studies, Charles Sturt University, Albury, N.S.W., Australia.

Barbara J. Steva, OTWL, BCP, is Occupational Therapist, University of New England Community Occupational Therapy Clinic, Saco, ME. She was Director of Occupational Therapy and Therapeutic Recreation, Franciscan Children's Hospital and Rehabilitation Center, Boston, MA, when this work was completed.