Does test-taking motivation predict test results in a high-stakes testing context?

References

• Abdelfattah, F. (2010). The relationship between motivation and achievement in low-stakes examinations. Social Behavior and Personality: An International Journal, 38(2), 159–168. https://doi.org/https://doi.org/10.2224/sbp.2010.38.2.159
   Web of Science ®Google Scholar
• Ackerman, P. L., & Kanfer, R. (2009). Test length and cognitive fatigue: An empirical examination of effects on performance and test-taker reactions. Journal of Experimental Psychology: Applied, 15(2), 163–181. https://doi.org/https://doi.org/10.1037/a0015719
   PubMed Web of Science ®Google Scholar
• Alkharusi, H., Aldhafri, S., Alnabhani, H., & Alkalbani, M. (2013). The impact of students’ perceptions of assessment tasks on self-efficacy and perception of task value: A path analysis. Social Behavior and Personality: An International Journal, 41(10), 1681–1692. https://doi.org/https://doi.org/10.2224/sbp.2013.41.10.1681
   Web of Science ®Google Scholar
• Asseburg, R., & Frey, A. (2013). Too hard, too easy, or just right? The relationship between effort or boredom and ability-difficulty fit. Psychological Test and Assessment Modeling, 55(1), 92–104.
   Google Scholar
• Barry, C. L., & Finney, S. J. (2016). Modeling change in effort across a low-stakes testing session: A latent growth curve modeling approach. Applied Measurement in Education, 29(1), 46–64. https://doi.org/https://doi.org/10.1080/08957347.2015.1102914
   Web of Science ®Google Scholar
• Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246. https://doi.org/https://doi.org/10.1037/0033-2909.107.2.238
   PubMed Web of Science ®Google Scholar
• Brand, C. (1996). The g factor: General intelligence and its implications. John Wiley & Sons.
   Google Scholar
• Brigham, C. (1923). A study of American intelligence. Princeton University Press.
   Google Scholar
• Brookhart, S. M. (1997). A theoretical framework for the role of classroom assessment in motivating student effort and achievement. Applied Measurement in Education, 10(2), 161–180. https://doi.org/https://doi.org/10.1207/s15324818ame1002_4
   Web of Science ®Google Scholar
• Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). SAGE Publications.
   Google Scholar
• Cheng, L., Klinger, D., Fox, J., Doe, C., Jin, Y., & Wu, J. (2014). Motivation and test anxiety in test performance across three testing contexts: The CAEL, CET, and GEPT. Tesol Quarterly, 48(2), 300–330. https://doi.org/https://doi.org/10.1002/tesq.105
   Web of Science ®Google Scholar
• Cole, J. S., Bergin, D. A., & Whittaker, T. A. (2008). Predicting student achievement for low stakes tests with effort and task value. Contemporary Educational Psychology, 33(4), 609–624. https://doi.org/https://doi.org/10.1016/j.cedpsych.2007.10.002
   Web of Science ®Google Scholar
• Coluccia, E., & Louse, G. (2004). Gender differences in spatial orientation: A review. Journal of Environmental Psychology, 24(3), 329–340. https://doi.org/https://doi.org/10.1016/j.jenvp.2004.08.006
   Web of Science ®Google Scholar
• Cronbach, L. J. (1990). Essentials of psychological testing. Harper & Row.
   Google Scholar
• Deary, I. J., Strand, S., Smith, P., & Fernandes, C. (2007). Intelligence and educational achievement. Intelligence, 35(1), 13–21. https://doi.org/https://doi.org/10.1016/j.intell.2006.02.001
   Web of Science ®Google Scholar
• DeMars, C. E., Bashkov, B. M., & Socha, A. B. (2013). The role of gender in test-taking motivation under low-stakes conditions. Research & Practice in Assessment, 8, 69–82.
   Google Scholar
• Demetriou, A., Kazi, S., Spanoudis, G., & Makris, N. (2019). Predicting school performance from cognitive ability, self-representation, and personality from primary school to senior high school. Intelligence, 76, Article 101381. https://doi.org/https://doi.org/10.1016/j.intell.2019.101381
   Google Scholar
• Dimitrov, D. M. (2012). Statistical methods for validation of assessment scale data in counseling and related fields. John Wiley & Sons.
   Google Scholar
• Duckworth, A. L., Quinn, P. D., Lynam, D. R., Loeber, R., & Stouthamer-Loeber, M. (2011). Role of test motivation in intelligence testing. Proceedings of the National Academy of Sciences, 108(19), 7716–7720. https://doi.org/https://doi.org/10.1073/pnas.1018601108
   PubMed Web of Science ®Google Scholar
• Dueber, D. M. (2017). Bifactor Indices Calculator: A Microsoft Excel-based tool to calculate various indices relevant to bifactor CFA models. https://doi.org/https://doi.org/10.13023/edp.tool.01
   Google Scholar
• Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109–132. https://doi.org/https://doi.org/10.1146/annurev.psych.53.100901.135153
   PubMed Web of Science ®Google Scholar
• Eklöf, H. (2010). Skill and will: Test-taking motivation and assessment quality. Assessment in Education: Principles, Policy & Practice, 17(4), 345–356. https://doi.org/https://doi.org/10.1080/0969594X.2010.516569
   Google Scholar
• Gagné, F., & St Père, F. (2001). When IQ is controlled, does motivation still predict achievement? Intelligence, 30(1), 71–100. https://doi.org/https://doi.org/10.1016/S0160-2896(01)00068-X
   Web of Science ®Google Scholar
• Goldstein, H. (1997). Methods in school effectiveness research. School Effectiveness and School Improvement, 8(4), 369–395. https://doi.org/https://doi.org/10.1080/0924345970080401
   Web of Science ®Google Scholar
• Guo, H., Rios, J. A., Haberman, S., Liu, O. L., Wang, J., & Paek, I. (2016). A new procedure for detection of students’ rapid guessing responses using response time. Applied Measurement in Education, 29(3), 173–183. https://doi.org/https://doi.org/10.1080/08957347.2016.1171766
   Web of Science ®Google Scholar
• Haladyna, T. M., & Downing, S. M. (2004). Construct-irrelevant variance in high-stakes testing. Educational Measurement: Issues and Practice, 23(1), 17–27. https://doi.org/https://doi.org/10.1111/j.1745-3992.2004.tb00149.x
   Google Scholar
• Hewson, C., & Charlton, J. P. (2019). An investigation of the validity of course-based online assessment methods: The role of computer-related attitudes and assessment mode preferences. Journal of Computer Assisted Learning, 35(1), 51–60. https://doi.org/https://doi.org/10.1111/jcal.12310
   Web of Science ®Google Scholar
• Hyde, J. S., Fennema, E., & Lamon, S. J. (1990). Gender differences in mathematics performance: A meta-analysis. Psychological Bulletin, 107(2), 139–155. https://doi.org/https://doi.org/10.1037/0033-2909.107.2.139
   PubMed Web of Science ®Google Scholar
• Hyde, J. S., & Linn, M. C. (1988). Gender differences in verbal ability: A meta-analysis. Psychological Bulletin, 104(1), 53–69. https://doi.org/https://doi.org/10.1037/0033-2909.104.1.53
   Web of Science ®Google Scholar
• Immekus, J. C., & McGee, D. (2016). The measurement invariance of the Student Opinion Scale across English and non-English language learner students within the context of low-and high-stakes assessments. Frontiers in Psychology, 7, Article 1352. https://doi.org/https://doi.org/10.3389/fpsyg.2016.01352
   Google Scholar
• Innove. (2020, March 31). State examinations. https://www.innove.ee/en/examinations-and-tests/state-examinations
   Google Scholar
• Jensen, A. R. (1998). The g factor: The science of mental ability. Praeger.
   Google Scholar
• Jöreskog, K. G., & Sörbom, D. (1989). LISREL 7: A guide to the program and applications. SPSS.
   Google Scholar
• Knekta, E. (2017). Are all pupils equally motivated to do their best on all tests? Differences in reported test-taking motivation within and between tests with different stakes. Scandinavian Journal of Educational Research, 61(1), 95–111. https://doi.org/https://doi.org/10.1080/00313831.2015.1119723
   Web of Science ®Google Scholar
• Knekta, E., & Eklöf, H. (2015). Modeling the test-taking motivation construct through investigation of psychometric properties of an expectancy-value-based questionnaire. Journal of Psychoeducational Assessment, 33(7), 662–673. https://doi.org/https://doi.org/10.1177/0734282914551956
   Web of Science ®Google Scholar
• Kong, X. J., Wise, S. L., & Bhola, D. S. (2007). Setting the response time threshold parameter to differentiate solution behavior from rapid-guessing behavior. Educational and Psychological Measurement, 67(4), 606–619. https://doi.org/https://doi.org/10.1177/0013164406294779
   Web of Science ®Google Scholar
• Linn, M. C., & Hyde, J. S. (1989). Gender, mathematics, and science. Educational Researcher, 18(8), 17–27. https://doi.org/https://doi.org/10.3102/0013189X018008017
   Google Scholar
• Liu, O. L., Bridgeman, B., & Adler, R. M. (2012). Measuring learning outcomes in higher education: Motivation matters. Educational Researcher, 41(9), 352–362. https://doi.org/https://doi.org/10.3102/0013189X12459679
   Web of Science ®Google Scholar
• Liu, O. L., Mao, L., Frankel, L., & Xu, J. (2016). Assessing critical thinking in higher education: The HEIghten™ approach and preliminary validity evidence. Assessment & Evaluation in Higher Education, 41(5), 677–694. https://doi.org/https://doi.org/10.1080/02602938.2016.1168358
   Web of Science ®Google Scholar
• Luyten, L., Lowyck, J., & Tuerlinckx, F. (2001). Task perception as a mediating variable: A contribution to the validation of instructional knowledge. British Journal of Educational Psychology, 71(2), 203–223. https://doi.org/https://doi.org/10.1348/000709901158488
   PubMedGoogle Scholar
• Must, O. (2013). Akadeemiline test ja selle roll kõrgkooli astumisel [Academic test and its role in admitting to university]. In K. Täht, J. Harro, O. Must, & A. Realo (Eds.), Kõrgkool ja psühholoogia (pp. 61–77). University of Tartu.
   Google Scholar
• Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide (7th ed.).
   Google Scholar
• Núñez-Peña, M. I., Suárez-Pellicioni, M., & Bono, R. (2016). Gender differences in test anxiety and their impact on higher education students’ academic achievement. Procedia – Social and Behavioral Sciences, 228, 154–160. https://doi.org/https://doi.org/10.1016/j.sbspro.2016.07.023
   Google Scholar
• Pekkarinen, T. (2015). Gender differences in behaviour under competitive pressure: Evidence on omission patterns in university entrance examinations. Journal of Economic Behavior & Organization, 115, 94–110. https://doi.org/https://doi.org/10.1016/j.jebo.2014.08.007
   Web of Science ®Google Scholar
• Peng, Y., Hong, E., & Mason, E. (2014). Motivational and cognitive test-taking strategies and their influence on test performance in mathematics. Educational Research and Evaluation, 20(5), 366–385. https://doi.org/https://doi.org/10.1080/13803611.2014.966115
   Google Scholar
• Penk, C., Pöhlmann, C., & Roppelt, A. (2014). The role of test-taking motivation for students’ performance in low-stakes assessments: An investigation of school-track-specific differences. Large-scale Assessments in Education, 2(1), Article 5. https://doi.org/https://doi.org/10.1186/s40536-014-0005-4
   Google Scholar
• Penk, C., & Richter, D. (2017). Change in test-taking motivation and its relationship to test performance in low-stakes assessments. Educational Assessment, Evaluation and Accountability, 29(1), 55–79. https://doi.org/https://doi.org/10.1007/s11092-016-9248-7
   Web of Science ®Google Scholar
• Penk, C., & Schipolowski, S. (2015). Is it all about value? Bringing back the expectancy component to the assessment of test-taking motivation. Learning and Individual Differences, 42, 27–35. https://doi.org/https://doi.org/10.1016/j.lindif.2015.08.002
   Web of Science ®Google Scholar
• Quinn, H. O. C. (2014). Bifactor models, explained common variance (ECV), and the usefulness of scores from unidimensional item response theory analyses [Unpublished master’s thesis]. University of North Carolina.
   Google Scholar
• Reeve, C. L., & Lam, H. (2007). Consideration of g as a common antecedent for cognitive ability test performance, test motivation, and perceived fairness. Intelligence, 35(4), 347–358. https://doi.org/https://doi.org/10.1016/j.intell.2006.08.006
   Web of Science ®Google Scholar
• Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. https://doi.org/https://doi.org/10.1080/00273171.2012.715555
   PubMed Web of Science ®Google Scholar
• Rios, J. (2021). Improving test-taking effort in low-stakes group-based educational testing: A meta-analysis of interventions. Applied Measurement in Education. Advance online publication. https://doi.org/https://doi.org/10.1080/08957347.2021.1890741
   Google Scholar
• Rios, J. A., Liu, O. L., & Bridgeman, B. (2014). Identifying low-effort examinees on student learning outcomes assessment: A comparison of two approaches. New Directions for Institutional Research, 2014(161), 69–82. https://doi.org/https://doi.org/10.1002/ir.20068
   Google Scholar
• Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016a). Applying bifactor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98(3), 223–237. https://doi.org/https://doi.org/10.1080/00223891.2015.1089249
   PubMed Web of Science ®Google Scholar
• Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016b). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137–150. https://doi.org/https://doi.org/10.1037/met0000045
   PubMed Web of Science ®Google Scholar
• Rosenzweig, E. Q., Wigfield, A., & Eccles, J. S. (2019). Expectancy-value theory and its relevance for student motivation and learning. In K. A. Renninger & S. E. Hidi (Eds.), The Cambridge handbook of motivation and learning (pp. 617–644). Cambridge University Press.
   Google Scholar
• Roth, B., Becker, N., Romeyke, S., Schäfer, S., Domnick, F., & Spinath, F. M. (2015). Intelligence and school grades: A meta-analysis. Intelligence, 53, 118–137. https://doi.org/https://doi.org/10.1016/j.intell.2015.09.002
   Web of Science ®Google Scholar
• Şahin, F. (2017). Exploring validity of computer-based test scores with examinees’ response behaviors and response times [Unpublished doctoral dissertation]. The State University of New York at Albany.
   Google Scholar
• Schnipke, D. L., & Scrams, D. J. (1997). Modeling item response times with a two-state mixture model: A new method of measuring speededness. Journal of Educational Measurement, 34(3), 213–232. https://doi.org/https://doi.org/10.1111/j.1745-3984.1997.tb00516.x
   Web of Science ®Google Scholar
• Schunk, D. H., Meece, J. R., & Pintrich, P. R. (2014). Motivation in education: Theory, research, and applications (4th ed.). Pearson Higher Ed.
   Google Scholar
• Silm, G., Must, O., & Täht, K. (2013). Test-taking effort as a predictor of performance in low-stakes tests. TRAMES: A Journal of the Humanities & Social Sciences, 17(4), 433–448.
   Web of Science ®Google Scholar
• Silm, G., Must, O., & Täht, K. (2019). Predicting performance in a low-stakes test using self-reported and time-based measures of effort. TRAMES: A Journal of the Humanities & Social Sciences, 23(3), 353–376. https://doi.org/https://doi.org/10.3176/tr.2019.3.06
   Web of Science ®Google Scholar
• Silm, G., Pedaste, M., & Täht, K. (2020). The relationship between performance and test-taking effort when measured with self-report or time-based instruments: A meta-analytic review. Educational Research Review, 31, Article 100335. https://doi.org/https://doi.org/10.1016/j.edurev.2020.100335
   Google Scholar
• Spearman, C. (1927). The abilities of man. Macmillan.
   Google Scholar
• Steinmayr, R., & Spinath, B. (2008). Sex differences in school achievement: What are the roles of personality and achievement motivation? European Journal of Personality, 22(3), 185–209. https://doi.org/https://doi.org/10.1002/per.676
   Web of Science ®Google Scholar
• Stenlund, T., Eklöf, H., & Lyrén, P.-E. (2017). Group differences in test-taking behaviour: An example from a high-stakes testing program. Assessment in Education: Principles, Policy & Practice, 24(1), 4–20. https://doi.org/https://doi.org/10.1080/0969594X.2016.1142935
   Web of Science ®Google Scholar
• Stucky, B. D., & Edelen, M. O. (2015). Using hierarchical IRT models to create unidimensional measures from multidimensional data. In S. P. Reise & D. A. Revicki (Eds.), Handbook of item response theory modelling: Applications to typical performance assessment (pp. 183–206). Routledge.
   Google Scholar
• Stucky, B. D., Thissen, D., & Orlando Edelen, M. (2013). Using logistic approximations of marginal trace lines to develop short assessments. Applied Psychological Measurement, 37(1), 41–57. https://doi.org/https://doi.org/10.1177/0146621612462759
   Web of Science ®Google Scholar
• Sundre, D. L. (1999, April 19–23). Does examinee motivation moderate the relationship between test consequences and test performance? [Paper presentation]. Annual Meeting of the American Educational Research Association, Montreal, Quebec, Canada.
   Google Scholar
• Sundre, D. L., & Kitsantas, A. (2004). An exploration of the psychology of the examinee: Can examinee self-regulation and test-taking motivation predict consequential and non-consequential test performance? Contemporary Educational Psychology, 29(1), 6–26. https://doi.org/https://doi.org/10.1016/S0361-476X(02)00063-2
   Web of Science ®Google Scholar
• Sundre, D. L., & Moore, D. L. (2002). The Student Opinion Scale: A measure of examinee motivation. Assessment Update, 14(1), 8–9.
   Google Scholar
• Swerdzewski, P. J., Harmes, J. C., & Finney, S. J. (2011). Two approaches for identifying low-motivated students in a low-stakes assessment context. Applied Measurement in Education, 24(2), 162–188. https://doi.org/https://doi.org/10.1080/08957347.2011.555217
   Web of Science ®Google Scholar
• Voyer, D., & Voyer, S. D. (2014). Gender differences in scholastic achievement: A meta-analysis. Psychological Bulletin, 140(4), 1174–1204. https://doi.org/https://doi.org/10.1037/a0036620
   PubMed Web of Science ®Google Scholar
• Wedman, J. (2017). Theory and validity evidence for a large-scale test for selection to higher education [Unpublished doctoral dissertation]. Umeå universitet.
   Google Scholar
• Weirich, S., Hecht, M., Penk, C., Roppelt, A., & Böhme, K. (2017). Item position effects are moderated by changes in test-taking effort. Applied Psychological Measurement, 41(2), 115–129. https://doi.org/https://doi.org/10.1177/0146621616676791
   PubMed Web of Science ®Google Scholar
• Wigfield, A., & Eccles, J. S. (2000). Expectancy–value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68–81. https://doi.org/https://doi.org/10.1006/ceps.1999.1015
   PubMed Web of Science ®Google Scholar
• Wise, S. L. (2015). Effort analysis: Individual score validation of achievement test data. Applied Measurement in Education, 28(3), 237–252. https://doi.org/https://doi.org/10.1080/08957347.2015.1042155
   Web of Science ®Google Scholar
• Wise, S. L. (2017). Rapid-guessing behavior: Its identification, interpretation, and implications. Educational Measurement: Issues and Practice, 36(4), 52–61. https://doi.org/https://doi.org/10.1111/emip.12165
   Web of Science ®Google Scholar
• Wise, S. L., & DeMars, C. E. (2005). Low examinee effort in low-stakes assessment: Problems and potential solutions. Educational Assessment, 10(1), 1–17. https://doi.org/https://doi.org/10.1207/s15326977ea1001_1
   Google Scholar
• Wise, S. L., & Kong, X. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18(2), 163–183. https://doi.org/https://doi.org/10.1207/s15324818ame1802_2
   Web of Science ®Google Scholar
• Wise, S. L., & Ma, L. (2012, April 14–16). Setting response time thresholds for a CAT item pool: The normative threshold method [Paper presentation]. Annual meeting of the National Council on Measurement in Education, Vancouver, British Columbia, Canada.
   Google Scholar
• Wise, S. L., Pastor, D. A., & Kong, X. J. (2009). Correlates of rapid-guessing behavior in low-stakes testing: Implications for test development and measurement practice. Applied Measurement in Education, 22(2), 185–205. https://doi.org/https://doi.org/10.1080/08957340902754650
   Web of Science ®Google Scholar
• Wise, V. L., Wise, S. L., & Bhola, D. S. (2006). The generalizability of motivation filtering in improving test score validity. Educational Assessment, 11(1), 65–83. https://doi.org/10.1207/s15326977ea1101_3
   Google Scholar
• Wolf, L. F., & Smith, J. K. (1995). The consequence of consequence: Motivation, anxiety, and test performance. Applied Measurement in Education, 8(3), 227–242. https://doi.org/https://doi.org/10.1207/s15324818ame0803_3
   Web of Science ®Google Scholar
• Wolf, L. F., Smith, J. K., & Birnbaum, M. E. (1995). Consequence of performance, test, motivation, and mentally taxing items. Applied Measurement in Education, 8(4), 341–351. https://doi.org/https://doi.org/10.1207/s15324818ame0804_4
   Web of Science ®Google Scholar