Teaching to the test: a mixed methods study of instructional change from large-scale testing in Canadian schools

Abstract

Large-scale provincial assessment is an integral part of schooling across Canada. The effect that these assessments have on classroom teaching remains an under-examined aspect of provincial policies. The study upon which this paper is based sets out to quantify the type and level of effect assessments have on teaching professionals using a survey instrument that was distributed to teachers in all provinces. The subjects and grades tested vary across the provinces, as does the manner of assessment, yet this national study did identify both commonalities and contrasts in how teachers react to testing data. The theoretical framework is based upon the concept of reactivity which posits that external evaluation changes the behaviours of the actors within audited systems. Of particular interest in the educational context were the changes in instructional practice that correlate with ‘teaching to the test’ or strategies employed specifically to improve test scores, whereas overall learning might not be improved. These practices are widespread across Canada, across grade levels and subject areas. Two highly significant factors were: (a) the presence of high-stakes exit examinations; and (b) the amount of pressure teachers perceived from the testing process. Teaching to the test is highly evident and in some cases strongly correlated with factors controlled by provincial assessment policy.

Keywords:

• Teaching to the test
• provincial assessment
• high-stakes testing
• reactivity
• assessment policy
• instructional improvement

Notes

1. Copp, D. T. (2015). Teacher-based reactivity to provincial large-scale assessment in Canada. (Boekenplan: Maastricht). p. 14. Accessible online at: http://digitalarchive.maastrichtuniversity.nl/fedora/get/guid:b082d77c-f478-4933-b757-4b327a2c16c8/ASSET1.

2. Campbell, 1957, p. 299: ‘Whenever the measurement process is not a part of the normal environment it is probably reactive’. Campbell showed that a flaw in social sciences research was the fact that any external evaluation of which the subject is aware has an impact on the very things that are being measured.

3. The model was created in part from an examination of the Saskatchewan Teachers’ Federation Code of Professional Competence found at: https://www.stf.sk.ca/portal.jsp?Sy3uQUnbK9L2RmSZs02CjVy0w7ZkI/ks6g2u00gzAtsk=F#https://www.stf.sk.ca/portal.jsp?Sy3uQUnbK9L2RmSZs02CjV/LfyjbyjsxsmWnKbLMKWFA=F

4. For example, the New Brunswick Assessment Programme has stated goals that include administering and reporting on provincial testing as well as coordinating the administration of international assessments. The argument is that these are not unrelated activities and that international results inform provincial policy choices.

5. ‘Extended responses are more able to tap a broader range of skills and objectives, and they give better curricular signals than do multiple choice items. On the other hand, multiple choice items are more reliably scored, at a lower cost than written or performance items. They also make it easier to equate tests over time’. (Nagy, 2000, p. 268).

6. To my knowledge, there has been an examination of administrators’ reactions, but in Ontario only (Volante, Cherubini, & Drake, 2008) and also case studies have been examined in American urban school districts by Lachat and Smith (2009).

7. See footnote 3.

8. ‘The more important that any quantitative social indicator becomes in social decision-making, the more likely it will be to distort and corrupt the social process it is intended to monitor … [thus] attaching serious personal and educational consequences to performance on tests for schools, administrators, teachers, and students, may have distorting and corrupting effects’. Amrein & Berliner, 2002, p. 5.

9. CMEC information retrieved 9 August 2014 from: http://www.cmec.ca/.

10. WNCP information retrieved 9 August 2014 from: https://www.wncp.ca/english/wncphome.aspx.

11. APEF information retrieved 9 August 2014 from: http://www.ednet.ns.ca/files/reports/essential_grad_learnings.pdf.

12. These data are available for all provinces and nationally from Statistics Canada, retrieved 9 August 2014 from: http://www.statcan.gc.ca/pub/81-582-x/2013001/tbl/tblc2.1-eng.htm.

http://www.statcan.gc.ca/pub/81-582-x/2013001/tbl/tblc2.2-eng.htm.

http://www.statcan.gc.ca/pub/81-582-x/2013001/tbl/tblc2.3-eng.htm.

13. Age and sex data from the Statistics Canada. Retrieved 13 April 2103 from http://publications.gc.ca/collections/collection_2007/statcan/81-582-X/81-582-XIE2007001.pdf.

14. The complete study with more complete data analysis can be found at: http://digitalarchive.maastrichtuniversity.nl/fedora/get/guid:b082d77c-f478-4933-b757-4b327a2c16c8/ASSET1.