Issues in developing valid assessments of speech pathology students' performance in the workplace

References

• Alexander, H.. 1996. Physiotherapy student clinical education: the influence of subjective judgements on observational assessment. Assessment and Evaluation in Higher Education. 21, 357–366.
   Google Scholar
• AMERICAN EDUCATIONAL RESEARCH ASSOCIATION (AERA), 1999, Validity. Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.
   Google Scholar
• Anderson, J., 1988, The Supervisory Process in Speech–Language Pathology and Audiology Boston, MA: College Hill.
   Google Scholar
• Barnard, J. J., 1999, Item analysis in test construction. In Masters G. N., Keeves J. P. (eds), Advances in Measurement in Educational Research and Assessment Oxford: Elsevier195–206.
   Google Scholar
• Barrie, S. C.. 2006. Understanding what we mean by the generic attributes of graduates. Higher Education. 51, 215–241.
   Web of Science ®Google Scholar
• Benner, P., 1984, From Novice to Expert: Excellence and Power in Clinical Nursing Practice Englewood Cliffs, NJ: Prentice-Hall.
   Google Scholar
• Benner, P. A., Tanner, C. A., Chesla, C. A., 1996, Expertise in Nursing Practice: Caring, Clinical Judgment, and Ethics New York, NY: Springer.
   Google Scholar
• Biggs, J. B., Collis, K. F., 1982, Evaluating the Quality of Learning: The SOLO Taxonomy (Structure of the Observed Learning Outcome) New York, NY: Academic Press.
   Google Scholar
• Bitzer, E., 1999, Assessing Learning in the MPhil (Higher Education Studies): paving new ways with cornerstones. Paper presented at the Higher Education Research and Development Society of Australasia Annual International Conference, Melbourne, Australia.
   Google Scholar
• Bond, T. G., Fox, C. M., 2007, Applying the Rasch Model: Fundamental Measurement in the Human Sciences Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Boud, D.. 2000. Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education. 22, 151–167.
   Google Scholar
• Boud, D., Falchikov, N.. 2006. Aligning assessment with long-term learning. Assessment and Evaluation in Higher Education. 31, 399–413.
   Google Scholar
• Brasseur, J.. 1989. The supervisory process: a continuum perspective. Language, Speech and Hearing Services in Schools. 20, 274–295.
   Google Scholar
• Brookhart, S. M.. 2001. Successful students' formative and summative uses of assessment information. Assessment in Education. 8, 153–168.
   Google Scholar
• Bruner, J., 1983, Child's Talk: Learning to Use Language New York, NY: W.W. Norton.
   Google Scholar
• Carraccio, C., Wolfsthal, S. D., Englander, R., Ferentz, K., Martin, C.. 2002. Shifting paradigms: drom Flexner to competencies. Academic Medicine. 77, 361–367.
   PubMed Web of Science ®Google Scholar
• Carter, R.. 1985. A taxonomy of objectives for professional education. Studies in Higher Education. 10, 135–149.
   Web of Science ®Google Scholar
• Chapman, J., 1998, Agonising about assessment. In Fish D., Coles C. (eds), Developing Professional Judgement in Health Care: Learning through the Critical Appreciation of Practice Oxford: Butterworth-Heinemann157–181.
   Google Scholar
• Clark, L. A., Watson, D.. 1995. Constructing validity: basic issues in objective scale development. Psychological Assessment. 7, 309–319.
   Web of Science ®Google Scholar
• Cox, K.. 2000. Examining and recording clinical performance: a critique and some recommendations. Education for Health. 13, 45–52.
   PubMedGoogle Scholar
• Cross, V., Hicks, C., Barwell, F.. 2001. Exploring the gap between evidence and judgement: using video vignettes for practice-based assessment of physiotherapy undergraduates. Assessment and Evaluation in Higher Education. 26, 189–212.
   Google Scholar
• Dawson, V.. 1995. C-BOS for students: developing professional competencies in speech pathology students (part 1). Australian Communication Quarterly. Summer, 6–7.
   Google Scholar
• Dawson, V., 2003, The competence of the Australian new graduate speech pathologist. Paper presented at the Speech Pathology Australia National Conference, Hobart, Australia, 4–8 May 2003.
   Google Scholar
• Dreyfus, H. L., Dreyfus, S. E., 1996, The relationship of theory and practice in the acquisition of skill. In Benner P. A., Tanner C. A., Chesla C. A. (eds), Expertise in Nursing Practice: Caring, Clinical Judgment, and Ethics New York, NY: Springer29–47.
   Google Scholar
• Duke, M.. 1996. Clinical evaluation — difficulties experienced by sessional clinical teachers of nursing: a qualitative study. Journal of Advanced Nursing. 23, 408–414.
   PubMedGoogle Scholar
• Embretson, S. E., 1999, Issues in the measurement of cognitive abilities. In Embretson S. E., Hershberger S. L. (eds), The New Rules of Measurement: What Every Psychologist and Educator Should Know Mahwah, NJ: Lawrence Erlbaum Associates1–16.
   Google Scholar
• Embretson, S. E., Reise, S. P., 2000, Item Response Theory for Psychologists Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Epstein, R. M., Hundert, E. M.. 2002. Defining and assessing professional competence. Journal of the American Medical Association. 287, 226–235.
   PubMed Web of Science ®Google Scholar
• Fay, C. H., Latham, G. P.. 1982. Effects of training and rating scales on rating errors. Personnel Psychology. 35, 105–117.
   Web of Science ®Google Scholar
• Ferguson, A.. 2006. Competency-based occupational standards: influences on Australian speech pathology education. Folia Phoniatrica et Logopaedica. 58, 23–31.
   PubMed Web of Science ®Google Scholar
• Ferguson, A., Lincoln, M., McAllister, L., McAllister, S., 2008, COMPASS^® Directions: Leading the Integration of a Competency-based Assessment Tool in Speech Pathology Learning and Teaching. Report prepared for Australian Learning & Teaching Council Ltd, Sydney, an initiative of the Australian Government Department of Education, Employment and Workplace Relations available at: http://www.altc.edu.au/.
   Google Scholar
• Friedman, M., Mennin, S.. 1991. Rethinking critical issues in performance assessment. Academic Medicine. 66, 390–395.
   PubMed Web of Science ®Google Scholar
• Gomez-Mejia, G.. 1988. Evaluating employee performance: does the appraisal instrument make a difference?. Journal of Organizational Behavior Management. 9, 155–172.
   Google Scholar
• Govaerts, J. J. B., Van der Vleuten, C., Schuwirth, L. W. T.. 2002. Optimising the reproducibility of a performance-based test in midwifery education. Advances in Health Sciences Education. 7, 133–145.
   PubMed Web of Science ®Google Scholar
• Govaerts, M. J. B., Van der Vleuten, C. P. M., Schuwirth, L. W. T., Muijtjens, A. M. M.. 2007. Broadening perspectives on clinical performance assessment: Rethinking the nature of in-training assessment. Advances in Health Sciences Education. 12, 239–260.
   PubMed Web of Science ®Google Scholar
• Hager, P.. 2000. Know-how and workplace practical judgement. Journal of Philosophy of Education. 34, 281–296.
   Web of Science ®Google Scholar
• Hager, P., Athanasou, J., Gonczi, A., 1994, Assessment Technical Manual Canberra, ACT: Australian Government Publishing Service.
   Google Scholar
• Hager, P., Gillis, S., 1995, Assessment at Higher Levels. In Hall W. C. (eds), Key aspects of competency-based assessment Adelaide, SA: NCVER, 59–71.
   Google Scholar
• Harris, R., Guthrie, H., Hobart, B., Lundberg, D., 1995, Competency-based Education and Training: Between a Rock and a Whirlpool Sydney: Macmillan Education Australia.
   Google Scholar
• Hays, R. B., Davies, H. A., Beard, J. D., Caldon, L. F. M., Farmer, E. A., Finucane, P. M., McCrorie, P., Newble, D., Schuwirth, L. W. T., Sibbald, G. R.. 2002. Selecting performance assessment methods for experienced physicians. Medical Education. 36, 910–917.
   PubMed Web of Science ®Google Scholar
• Higgs, J., Bithell, C., 2001, Professional expertise. In Higgs J., Titchen A. (eds), Practice Knowledge and Expertise in the Health Professions Oxford: Butterworth-Heinemann, 59–68.
   Google Scholar
• Hunt, D. D.. 1992. Functional and dysfunctional characteristics of the prevailing model of clinical evaluation systems in North American medical schools. Academic Medicine. 67, 254–259.
   PubMed Web of Science ®Google Scholar
• Ilott, I., Murphy, R.. 1997. Feelings and failing in professional training: the assessor's dilemma. Assessment and Evaluation in Higher Education. 22, 307–316.
   Google Scholar
• Johnson, C. J., Shewan, C. M.. 1988. A new perspective in evaluating clinical effectiveness: the UWO clinical grading system. Journal of Speech and Hearing Disorders. 53, 328–340.
   Google Scholar
• Jones, A.. 2000. The place of judgment in competency-based assessment. Journal of Vocational Education and Training. 51, 145–160.
   Google Scholar
• Keen, A. J. A., Klein, S., Alexander, D. A.. 2003. Assessing the communication skills of doctors in training: reliability and sources of error. Advances in Health Sciences Education. 8, 5–16.
   PubMed Web of Science ®Google Scholar
• Kingstrom, P. O., Bass, A. R.. 1981. A critical analysis of studies comparing Behaviorally Anchored Rating Scales (BARS) and other rating formats. Personnel Psychology. 34, 263–289.
   Web of Science ®Google Scholar
• Landon, B. E., Normand, S.-L. T., Blumenthal, D., Daley, J.. 2003. Physician Clinical performance assessment. Journal of the American Medical Association. 9, 1183–1189.
   Google Scholar
• Landy, F. J., Farr, J. L.. 1980. Performance rating. Psychological Bulletin. 87, 72–107.
   Web of Science ®Google Scholar
• Leach, L., Neutze, G., Zepke, N.. 2001. Assessment and empowerment: Some critical questions. Assessment and Evaluation in Higher Education. 26, 293–305.
   Google Scholar
• Linacre, J. M.. 1994. Sample size and item calibration stability. Rasch Measurement Transactions [e-journal]. 7 4: 328.
   Google Scholar
• Linacre, J. M.. 2002. Optimizing rating scale category effectiveness. Journal of Applied Measurement. 3, 85–106.
   PubMedGoogle Scholar
• Lincoln, M., Ferguson, A., McAllister, L., McAllister, S., 2008, Benchmarking Clinical Learning in Speech Pathology to Support Assessment, Discipline Standards, Teaching Innovation and Student Learning. Report prepared for Australian Learning & Teaching Council Ltd, Sydney, an initiative of the Australian Government Department of Education, Employment and Workplace Relations available at: http://www.altc.edu.au/.
   Google Scholar
• Linn, R. L., Baker, E. L., Dunbar, S. B.. 1991. Complex, performance-based assessment: expectations and validation criteria. Educational Researcher. 20, 15–21.
   Google Scholar
• McAllister, S., 2005, Competency Based Assessment of Speech Pathology Students' Performance in the Workplace Sydney: Faculty of Health Sciences, The University of Sydney available at: http://hdl.handle.net/2123/1130.
   Google Scholar
• McAllister, S.. 2008. Introduction to the use of Rasch analysis to assess patient performance. International Journal of Therapy and Rehabilitation. 15, 482–490.
   Google Scholar
• McAllister, S., Lincoln, M., Ferguson, A., McAllister, L., 2006, COMPASS^®: Competency Assessment in Speech Pathology Melbourne: Speech Pathology Association of Australia.
   Google Scholar
• McAllister, S., Lincoln, M., Ferguson, A., McAllister, L., 2008, Engaging speech pathology students in design and validation of competency-based assessment in the workplace. Paper presented at the ATN Assessment Conference, Adelaide, Australia.
   Google Scholar
• McAllister, S. M., Lincoln, M., Ferguson, A., McAllister, L., 2004, User evaluation of the content and processes of an assessment of student performance in workplace settings. Paper presented at the IALP World Congress, 2004, Brisbane, Australia.
   Google Scholar
• McGaghie, W. C., 1993, Evaluating competence for professional practice. In Curry L., Wergin J. F. (eds), Educating Professionals San Francisco, CA: Jossey-Bass, 229–261.
   Google Scholar
• Michell, J.. 1997. Quantitative science and the definition of measurement in psychology. British Journal of Psychology. 88, 355–383.
   Web of Science ®Google Scholar
• Moss, P.. 1992. Shifting conceptions of validity in educational measurement: implications for performance assessment. Review of Educational Research. 62, 229–258.
   Web of Science ®Google Scholar
• Newble, D., Norman, G., Van der Vleuten, C., 2000, Assessing clinical reasoning. In Higgs J., Jones M. (eds), Clinical Reasoning in the Health Professions Oxford: Butterworth-Heinemann, 156–165.
   Google Scholar
• Norcini, J. J.. 2005. Current perspectives in assessment: the assessment of performance at work. Medical Education. 39, 880–889.
   PubMed Web of Science ®Google Scholar
• Norman, G., Van der Vleuten, C., De Graaff, E.. 1991. Pitfalls in the pursuit of objectivity: issues of reliability. Medical Education. 25, 110–118.
   Google Scholar
• Peters, G., Fraser, J., Cowie, F., Loader, J., Rutter, S., Scott, R., Davie, B.. 2001. Competency-based assessment in a perioperative nursing graduate diploma. ACORN Journal. 15, 11–14.
   Google Scholar
• Rethans, J.-J., Norcini, J., Baron-Maldonado, M., Blackmore, D., Jolly, B. C., Laduca, T., Lew, S. R., Page, G. G., Southgate, L. J.. 2002. The relationship between competence and performance: Implications for assessing practice performance. Medical Education. 36, 901–909.
   PubMed Web of Science ®Google Scholar
• Robertson, S., Rosenthal, J., Dawson, V., 1997, Using assessment to promote student learning. In McAllister L., Lincoln M., McLeod S. (eds), Facilitating Learning in Clinical Settings Cheltenham: Stanley Thornes, 154–184.
   Google Scholar
• Rose M., Best D. (eds), 2005, Transforming Practice through Clinical Education, Professional Supervision and Mentoring Edinburgh: Elsevier Churchill Livingstone.
   Google Scholar
• Royal College of Speech & Language Therapists (RCSLT), 2007, Newly Qualified Practitioners Competency Framework: Speech and Language Therapy Competency Framework to Guide Transition to certified RCSLT Membership London: Royal College of Speech & Language Therapists available at: http://www.rcslt.org/resources/newlyqualifiedpractitioners) (accessed on 11 October 2008).
   Google Scholar
• Schuwirth, L. W. T., Van der Vleuten, C. P. M.. 2006. A plea for new psychometric models in educational assessment. Medical Education. 40, 296–300.
   PubMed Web of Science ®Google Scholar
• Shavelson, R. J., Gao, X., Baxter, G., 1993, Sampling Variability in Performance Assessments. CSE Technical Report Number 361 Santa Barbara, CA: National Center for Research on Evaluation, Standards, and Student Testing, University of California.
   Google Scholar
• Speech Pathology Association of Australia (SPAA), 2001, Competency-based Occupational Standards for Speech Pathologists (Entry Level) Melbourne: Speech Pathology Association of Australia.
   Google Scholar
• Spike, N., Alexander, H., Elliott, S., Hazlett, C., Kilminster, S., Prideaux, D., Roberts, T.. 2000. In-training assessment—its potential in enhancing clinical teaching. Medical Education. 34, 858–861.
   Google Scholar
• Wass, V., Van der Vleuten, C., Shatzer, J., Jones, J.. 2001. Assessment of clinical competence. Lancet. 357, 945–949.
   PubMed Web of Science ®Google Scholar
• Wilkinson, T. J., Frampton, C. M.. 2003. Assessing performance in final year medical students. Can a postgraduate measure be used in an undergraduate setting? Medical Education. 37, 233–240.
   Google Scholar
• Woehr, D. J., Huffcutt, A. I.. 1994. Rater training for performance appraisal: a quantitative review. Journal of Occupational and Organizational Psychology. 67, 189–205.
   Web of Science ®Google Scholar
• Wolf, A., 1995, Competence-based Assessment Philadelphia, PA: Open University Press.
   Google Scholar
• Wolfe, E. W., Gitomer, D. H.. 2001. The influence of changes in assessment design on the psychometric quality of scores. Applied Measurement in Education. 14, 91–107.
   Web of Science ®Google Scholar
• Wright, B. D., 1999, Fundamental measurement for psychology. In Embretson S. E., Hershberger S. L. (eds), The New Rules of Measurement: What Every Psychologist and Educator Should Know Mahwah, NJ: Lawrence Erlbaum Associates, 65–106.
   Google Scholar
• Zhu, W.. 1996. Should total scores from a rating scale be used directly?. Research Quarterly for Exercise and Sports. 67, 363–372.
   PubMed Web of Science ®Google Scholar