Improving mathematics diagnostic tests using item analysis

References

• Adams, W. K., Perkins, K. K., Podolefsky, N. S., Dubson, M., Finkelstein, N. D., & Wieman, C. E. (2006). New instrument for measuring student beliefs about physics and learning physics: The Colorado learning attitudes about science survey. Physical Review Special Topics – Physics Education Research, 2(1), 010101. https://doi.org/10.1103/PhysRevSTPER.2.010101
   Google Scholar
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (Eds.). (2014). Standards for educational and psychological testing. American Educational Research Association.
   Google Scholar
• Barbas, H., & Schramm, T. (2018). The hamburg online math test MINTFIT for prospective students of STEM degree programmes. MSOR Connections, 16(3), 43. https://doi.org/10.21100/msor.v16i3.645
   Google Scholar
• Bloom, B. S., Englehard, M. D., Furst, E. J., & Hill, W. H. (1956). Taxonomy of educational objectives: The classification of educational goals: Handbook I, cognitive domain. David McKay Co Inc..
   Google Scholar
• Carlson, M., Madison, B., & West, R. (2010). The Calculus Concept Readiness (CCR) instrument: Assessing student readiness for calculus. http://arxiv.org/abs/1010.2719.
   Google Scholar
• Carlson, M., Oehrtman, M., & Engelke, N. (2010). The precalculus concept assessment: A tool for assessing students' reasoning abilities and understandings. Cognition and Instruction, 28(2), 113–145. https://doi.org/10.1080/07370001003676587
   Web of Science ®Google Scholar
• Carr, M., Fidalgo, C., Bigotte de Almeida, M., Branco, J., Santos, V., Murphy, E., & NíFhloinn, E. (2015). Mathematics diagnostic testing in engineering: an international comparison between Ireland and Portugal. European Journal of Engineering Education, 40(5), 546–556. https://doi.org/10.1080/03043797.2014.967182
   Web of Science ®Google Scholar
• Caspar, A., Hungerbühler, N., & Kalisch, M. (2013). Basisprüfung und vorgängige Leistungen 2011/12 (Interne Auswertung) (Tech. Rep.). ETH Zurich.
   Google Scholar
• Chongchitnan, S. (2019). Electronic preparatory test for mathematics undergraduates: Implementation, results and correlations. MSOR Connections, 17(3), 5–13. https://doi.org/10.21100/msor.v17i3
   Google Scholar
• Code, W., Merchant, S., Maciejewski, W., Thomas, M., & Lo, J. (2016). The mathematics attitudes and perceptions survey: An instrument to assess expert-like views and dispositions among undergraduate mathematics students. International Journal of Mathematical Education in Science and Technology, 47(6), 917–937. https://doi.org/10.1080/0020739X.2015.1133854
   Web of Science ®Google Scholar
• Darlington, E. (2014). Contrasts in mathematical challenges in A-level mathematics and further mathematics, and undergraduate mathematics examinations. Teaching Mathematics and Its Applications, 33(4), 213–229. https://doi.org/10.1093/teamat/hru021
   Google Scholar
• De Ayala, R. J. (2008). The theory and practice of item response theory. In R. J. de Ayala, (Ed). Methodology in the social sciences ser. Guilford Publications.
   Google Scholar
• DeMars, C. (2010). Item response theory. Oxford University Press. https://doi.org/10.1093/ACPROF:OSO/9780195377033.001.0001
   Google Scholar
• Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. L. Erlbaum Associates.
   Google Scholar
• Epstein, J. (2007). Development and validation of the calculus concept inventory. In Proceedings of the ninth international conference on mathematics education in a global community (Vol. 9, pp. 165–170).
   Google Scholar
• Epstein, J. (2013). The calculus concept inventory–measurement of the effect of teaching methodology in mathematics. Notices of the American Mathematical Society, 60(08), 1018. https://doi.org/10.1090/noti1033
   Google Scholar
• Foster, C. (2018). Developing mathematical fluency: comparing exercises and rich tasks. Educational Studies in Mathematics, 97(2), 121–141. https://doi.org/10.1007/s10649-017-9788-x
   Web of Science ®Google Scholar
• Gleason, J., Bagley, S., Thomas, M., Rice, L., & White, D. (2019). The calculus concept inventory: A psychometric analysis and implications for use. International Journal of Mathematical Education in Science and Technology, 50(6), 825–838. https://doi.org/10.1080/0020739X.2018.1538466
   Google Scholar
• Heck, A., & Van Gastel, L. (2006). Mathematics on the threshold. International Journal of Mathematical Education in Science and Technology, 37(8), 925–945. https://doi.org/10.1080/00207390600819003
   Google Scholar
• Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inventory. The Physics Teacher, 30(3), 141–158. https://doi.org/10.1119/1.2343497
   Google Scholar
• Hodds, M. (2021). The early impact of the COVID-19 pandemic on mathematical competencies on entry into a UK university. Teaching Mathematics and Its Applications: An International Journal of the IMA, 40(4), 254–262. https://doi.org/10.1093/teamat/hrab021
   Web of Science ®Google Scholar
• Hodds, M., Shao, J., & Lawson, D. (2022). Changes in student entry competencies 2001–2017. International Journal of Mathematical Education in Science and Technology, 53(7), 1859–1874. https://doi.org/10.1080/0020739X.2020.1849836
   Web of Science ®Google Scholar
• Hsu, E., & Bressoud, D. (2015). Placement and student performance in calculus I. In D. Bressoud, V. Mesa, & C. Rasmussen (Eds.), Insights and recommendations from the MAA national study of college calculus. MAA Press. https://www.maa.org/sites/default/files/pdf/pubs/books/members/insights.pdf#page=71.
   Google Scholar
• Hyland, D., & O'Shea, A. (2022). The nature and prevalence of diagnostic testing in mathematics at tertiary-level in Ireland. Teaching Mathematics and Its Applications: An International Journal of the IMA, 41(1), 32–50. https://doi.org/10.1093/teamat/hrab006
   Web of Science ®Google Scholar
• Jodoin, M. G., Keller, L. A., & Swaminathan, H. (2003). A comparison of linear, fixed common item, and concurrent parameter estimation equating procedures in capturing academic growth. The Journal of Experimental Education, 71(3), 229–250. https://doi.org/10.1080/00220970309602064
   Web of Science ®Google Scholar
• Kinnear, G., Bennett, M., Binnie, R., Bolt, R., & Zheng, Y. (2020). Reliable application of the MATH taxonomy sheds light on assessment practices [hrz017]. Teaching Mathematics and Its Applications: An International Journal of the IMA, 39(4),281–295. https://doi.org/10.1093/teamat/hrz017
   Web of Science ®Google Scholar
• Kinnear, G., Wood, A. K., & Gratwick, R. (2022). Designing and evaluating an online course to support transition to university mathematics. International Journal of Mathematical Education in Science and Technology, 53(1), 11–34. https://doi.org/10.1080/0020739X.2021.1962554
   Web of Science ®Google Scholar
• Krathwohl, D. R. (2002). A revision of bloom's taxonomy: An overview. Theory Into Practice, 41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2
   Web of Science ®Google Scholar
• Lawson, D. (2003). Diagnostic testing for mathematics (Tech. Rep.). LTSN MathsTEAM Project. ISBN: 0704423731. https://www.heacademy.ac.uk/system/files/diagnostic_test.pdf.
   Google Scholar
• Lithner, J. (2008). A research framework for creative and imitative reasoning. Educational Studies in Mathematics, 67(3), 255–276. https://doi.org/10.1007/s10649-007-9104-2
   Google Scholar
• Ludecke, D., Ben-Shachar, M. S., Patil, I., & Makowski, D. (2020). Extracting, computing and exploring the parameters of statistical models using R. Journal of Open Source Software, 5(53), 2445. https://doi.org/10.21105/joss
   Google Scholar
• Maciejewski, W., & Star, J. R. (2016). Developing flexible procedural knowledge in undergraduate calculus. Research in Mathematics Education, 18(3), 299–316. https://doi.org/10.1080/14794802.2016.1148626
   Web of Science ®Google Scholar
• Mazur, E. (1997). Peer instruction: a user's manual. Prentice Hall.
   Google Scholar
• Mejia-Ramos, J. P., Lew, K., & Weber, K. (2017). Developing and validating proof comprehension tests in undergraduate mathematics. Research in Mathematics Education, 19(2), 130–146. https://doi.org/10.1080/14794802.2017.1325776
   Web of Science ®Google Scholar
• Melhuish, K. (2019). The group theory concept assessment: A tool for measuring conceptual understanding in introductory group theory. International Journal of Research in Undergraduate Mathematics Education, 5(3), 359–393. https://doi.org/10.1007/s40753-019-00093-6
   Google Scholar
• O'Shea, A., Breen, S., & Jaworski, B. (2016). The development of a function concept inventory. International Journal of Research in Undergraduate Mathematics Education, 2(3), 279–296. https://doi.org/10.1007/s40753-016-0030-5
   Google Scholar
• Pointon, A., & Sangwin, C. J. (2003). An analysis of undergraduate core material in the light of hand-held computer algebra systems. International Journal of Mathematical Education in Science and Technology, 34(5), 671–686. https://doi.org/10.1080/0020739031000148930
   Google Scholar
• Rach, S., & Ufer, S. (2020). Which prior mathematical knowledge is necessary for study success in the university study entrance phase? results on a new model of knowledge levels based on a reanalysis of data from existing studies. International Journal of Research in Undergraduate Mathematics Education, 6(3), 375–403. https://doi.org/10.1007/s40753-020-00112-x
   Google Scholar
• Rittle-Johnson, B., Schneider, M., & Star, J. R. (2015). Not a one-way street: Bidirectional relations between procedural and conceptual knowledge of mathematics. Educational Psychology Review, 27(4), 587–597. https://doi.org/10.1007/s10648-015-9302-x
   Web of Science ®Google Scholar
• Sangwin, C. J. (2013). Computer aided assessment of mathematics. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199660353.003.0003
   Google Scholar
• Skemp, R. R. (1976). Relational understanding and instrumental understanding. MathematicsTeaching, 77(1), 20–26. https://eric.ed.gov/?id=EJ154208.
   Google Scholar
• Smith, G., Wood, L., Coupland, M., Stephenson, B., Crawford, K., & Ball, G. (1996). Constructing mathematical examinations to assess a range of knowledge and skills. International Journal of Mathematical Education in Science and Technology, 27(1), 65–77. https://doi.org/10.1080/0020739960270109
   Google Scholar
• SQA (2018). Higher mathematics course specification. https://www.sqa.org.uk/files_ccc/HigherCourseSpecMathematics.pdf.
   Google Scholar
• Tallman, M. A., Carlson, M. P., Bressoud, D. M., Pearson, M., & Org, P. (2016). A characterization of calculus I final exams in U.S. colleges and universities. International Journal of Research in Undergraduate Mathematics Education, 2(1), 105–133. https://doi.org/10.1007/s40753-015-0023-9
   Google Scholar
• Todd, K. (2003). In MathsTEAM (Ed.), Diagnostic testing for mathematics (pp. 16–17). LTSN MathsTEAM Project.
   Google Scholar
• Von Korff, J., Archibeque, B., Gomez, K. A., Heckendorf, T., McKagan, S. B., Sayre, E. C., Schenk, E. W., Shepherd, C., & Sorell, L. (2016). Secondary analysis of teaching methods in introductory physics: A 50 k-student study. American Journal of Physics, 84(12), 969–974. https://doi.org/10.1119/1.4964354
   Web of Science ®Google Scholar
• Wakefield, N., Champion, J., Bolkema, J., & Dailey, D. (2018). Diagnostic effects of an early mastery activity in college algebra and precalculus. International Journal of Research in Undergraduate Mathematics Education, 4(3), 376–392. https://doi.org/10.1007/s40753-018-0075-8.
   Google Scholar
• Xie, B., Li, M., Davidson, M. J., & Ko, A. J. (2019). An item response theory evaluation of a language-independent CS1 knowledge assessment. In SIGCSE 2019 – Proceedings of the 50th ACM technical symposium on computer science education (pp. 699–705). Association for Computing Machinery. https://doi.org/10.1145/3287324.3287370
   Google Scholar