Do Students Rapidly Guess Repeatedly over Time? A Longitudinal Analysis of Student Test Disengagement, Background, and Attitudes

References

• Ackerman, P. L., & Kanfer, R. (2009). Test length and cognitive fatigue: An empirical examination of effects on performance and test-taker reactions. Journal of Experimental Psychology: Applied, 15(2), 163. doi:10.1037/a0016995
   PubMed Web of Science ®Google Scholar
• Bailey, D. H., Watts, T. W., Littlefield, A. K., & Geary, D. C. (2014). State and trait effects on individual differences in children’s mathematical development. Psychological Science, 25(11), 2017–2026. doi:10.1177/0956797613518350
   PubMed Web of Science ®Google Scholar
• Bandura, A. (1986). The explanatory and predictive scope of self-efficacy theory. Journal of Social and Clinical Psychology, 4(3), 359–373. doi:10.1521/jscp.1986.4.3.359
   Web of Science ®Google Scholar
• Bandura, A. (1993). Perceived self-efficacy in cognitive development and functioning. Educational Psychologist, 28(2), 117–148. doi:10.1207/s15326985ep2802_3
   Web of Science ®Google Scholar
• Barry, C. L., & Finney, S. J. (2016). Modeling change in effort across a low-stakes testing session: A latent growth curve modeling approach. Applied Measurement in Education, 29(1), 46–64. doi:10.1080/08957347.2015.1102914
   Web of Science ®Google Scholar
• Barry, C. L., Horst, S. J., Finney, S. J., Brown, A. R., & Kopp, J. P. (2010). Do examinees have similar test-taking effort? A high-stakes question for low-stakes testing. International Journal of Testing, 10(4), 342–363. doi:10.1080/15305058.2010.508569
   Google Scholar
• Bentler, P. M. (2007). On tests and indices for evaluating structural models. Personality and Individual Differences, 42(5), 825–829. doi:10.1016/j.paid.2006.09.024
   Web of Science ®Google Scholar
• Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child Development, 78(1), 246–263. doi:10.1111/cdev.2007.78.issue-1
   PubMed Web of Science ®Google Scholar
• Briesch, A. M., & Chafouleas, S. M. (2009). Review and analysis of literature on self-management interventions to promote appropriate classroom behaviors (1988–2008). School Psychology Quarterly, 24(2), 106. doi:10.1037/a0016159
   Web of Science ®Google Scholar
• Clements, D. H., Sarama, J., Spitler, M. E., Lange, A. A., & Wolfe, C. B. (2011). Mathematics learned by young children in an intervention based on learning trajectories: A large-scale cluster randomized trial. Journal for Research in Mathematics Education, 42(2), 127–166. doi:10.5951/jresematheduc.42.2.0127
   Web of Science ®Google Scholar
• Clements, D. H., Sarama, J., Wolfe, C. B., & Spitler, M. E. (2013). Longitudinal evaluation of a scale-up model for teaching mathematics with trajectories and technologies: Persistence of effects in the third year. American Educational Research Journal, 50(4), 812–850. doi:10.3102/0002831212469270
   Web of Science ®Google Scholar
• Cole, D. A. (2006). Coping with longitudinal data in research on developmental psychopathology. International Journal of Behavioral Development, 30(1), 20–25. doi:10.1177/0165025406059969
   Web of Science ®Google Scholar
• Cole, D. A., Martin, N. C., & Steiger, J. H. (2005). Empirical and conceptual problems with longitudinal trait-state models: Introducing a trait-state-occasion model. Psychological Methods, 10(1), 3–20. doi:10.1037/1082-989X.10.1.3
   PubMed Web of Science ®Google Scholar
• DeMars, C. E. (2007). Changes in rapid-guessing behavior over a series of assessments. Educational Assessment, 12(1), 23–45. doi:10.1080/10627190709336946
   Google Scholar
• DeMars, C. E., & Wise, S. L. (2010). Can differential rapid-guessing behavior lead to differential item functioning? International Journal of Testing, 10(3), 207–229. doi:10.1080/15305058.2010.496347
   Google Scholar
• Dweck, C. (2006). Mindset: The new psychology of success. New York: Random House.
   Google Scholar
• Dweck, C. S., Walton, G. M., & Cohen, G. (2011). Academic tenacity. White Paper Prepared for the Gates Foundation. Seattle, WA
   Google Scholar
• Fan, X., & Sivo, S. A. (2005). Sensitivity of fit indexes to misspecified structural or measurement model components: Rationale of two-index strategy revisited. Structural Equation Modeling, 12(3), 343–367. doi:10.1207/s15328007sem1203_1
   Web of Science ®Google Scholar
• Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & Beechum, N. O. (2012). Teaching adolescents to become learners: The role of noncognitive factors in shaping school performance-a critical literature review. Chicago, IL: Consortium on Chicago School Research.
   Google Scholar
• Goldhammer, F., Martens, T., Christoph, G., & Lüdtke, O. (2016). Test-taking engagement in PIAAC. OECD Education Working Papers, No. 133. OECD Publishing.
   Google Scholar
• Guo, H., Rios, J. A., Haberman, S., Liu, O. L., Wang, J., & Paek, I. (2016). A new procedure for detection of students’ rapid guessing responses using response time. Applied Measurement in Education, 29(3), 173–183. doi:10.1080/08957347.2016.1171766
   Web of Science ®Google Scholar
• Heckman, J. J., Stixrud, J., & Urzua, S. (2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24(3), 411–482. doi:10.1086/504455
   Web of Science ®Google Scholar
• Jensen, N., Rice, A., & Soland, J. (2018). The influence of rapidly guessed item responses on teacher value-added estimates: Implications for policy and practice. Educational Evaluation and Policy Analysis, 20(1), 90–98.
   Google Scholar
• Kenny, D. A., & Zautra, A. (1995). The trait-state-error model for multiwave data. Journal of Consulting and Clinical Psychology, 63(1), 52. doi:10.1037/0022-006X.63.1.52
   PubMed Web of Science ®Google Scholar
• Kong, X. J., Wise, S. L., & Bhola, D. S. (2007). Setting the response time threshold parameter to differentiate solution behavior from rapid-guessing behavior. Educational and Psychological Measurement, 67, 606–619. doi:10.1177/0013164406294779
   Web of Science ®Google Scholar
• Kong, X. J., Wise, S. L., Harmes, J. C., & Yang, S. (2006). Motivational effects of praise in response-time based feedback: A follow-up study of the effort-monitoring CBT. In annual meeting of the National Council on Measurement in Education, San Francisco.
   Google Scholar
• Lee, Y.-H., & Jia, Y. (2014). Using response time to investigate students’ test-taking behaviors in a NAEP computer-based study. Large-Scale Assessments in Education, 2(1), 8. doi:10.1186/s40536-014-0008-1
   Google Scholar
• Muthén, L. K., & Muthén, B. O. (1998–2017). Mplus user’s guide (Eighth ed.). Los Angeles, CA: Muthén & Muthén.
   Google Scholar
• Pintrich, P. R. (1999). The role of motivation in promoting and sustaining self-regulated learning. International Journal of Educational Research, 31(6), 459–470. doi:10.1016/S0883-0355(99)00015-4
   Google Scholar
• Prenoveau, J. M. (2016). Specifying and interpreting latent state–Trait models with autoregression: An illustration. Structural Equation Modeling: A Multidisciplinary Journal, 23(5), 731–749. doi:10.1080/10705511.2016.1186550
   Web of Science ®Google Scholar
• Prenoveau, J. M., Craske, M. G., Zinbarg, R. E., Mineka, S., Rose, R. D., & Griffith, J. W. (2011). Are anxiety and depression just as stable as personality during late adolescence? Results from a three-year longitudinal latent variable study. Journal of Abnormal Psychology, 120(4), 832. doi:10.1037/a0023939
   PubMed Web of Science ®Google Scholar
• Rios, J. A., Guo, H., Mao, L., & Liu, O. L. (2016). Evaluating the impact of careless responding on aggregated-scores: To filter unmotivated examinees or not? International Journal of Testing, 17(1), 74–104.
   Web of Science ®Google Scholar
• Rios, J. A., Liu, O. L., & Bridgeman, B. (2014). Identifying low-effort examinees on student learning outcomes assessment: A comparison of two approaches. New Directions for Institutional Research, (2014(161), 69–82. doi:10.1002/ir.20068
   Google Scholar
• Schnipke, D. L., & Scrams, D. J. (2002). Exploring issues of examinee behavior: Insights gained from response-time analyses. In Mills, C. N., Potenza, M.T., Fremer, J. J., & Ward, W. C. (Eds.), Computer-Based Testing: Building the Foundation for Future Assessments. Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Schunk, D. H. (2005). Commentary on self-regulation in school contexts. Learning and Instruction, 15(2), 173–177. doi:10.1016/j.learninstruc.2005.04.013
   Web of Science ®Google Scholar
• Setzer, J. C., Wise, S. L., van Den Heuvel, J. R., & Ling, G. (2013). An investigation of examinee test-taking effort on a large-scale assessment. Applied Measurement in Education, 26(1), 34–49. doi:10.1080/08957347.2013.739453
   Web of Science ®Google Scholar
• Soland, J. (2018). Are achievement gap estimates biased by differential student test effort? Putting an important policy metric to the test. Teachers College Record, 120, 12.
   Web of Science ®Google Scholar
• Soland, J., & Kuhfeld, M. (In Press). Is social-emotional learning stable across school years? Implications for practice, policy, and evaluation. Journal of Research on Educational Effectiveness.
   Google Scholar
• Soland, J., Jensen, N., Keys, T. D., Bi, S. Z., & Wolk, E. (2019). Are test and academic disengagement related? Implications for measurement and practice. Educational Assessment, 24(2), 119–134.
   Web of Science ®Google Scholar
• Steiger, J. H. (2000). Point estimation, hypothesis testing, and interval estimation using the RMSEA: Some comments and a reply to hayduk and glaser. Structural Equation Modeling, 7(2), 149–162. doi:10.1207/S15328007SEM0702_1
   Web of Science ®Google Scholar
• Steyer, R., Ferring, D., & Schmitt, M. J. (1992). States and traits in psychological assessment. European Journal of Psychological Assessment, 8, 79–98.
   Google Scholar
• Swerdzewski, P. J., Harmes, J. C., & Finney, S. J. (2009). Skipping the test: Using empirical evidence to inform policy related to students who avoid taking low-stakes assessments in college. The Journal of General Education, 167–195. doi:10.1353/jge.0.0043
   Google Scholar
• Swerdzewski, P. J., Harmes, J. C., & Finney, S. J. (2011). Two approaches for identifying low-motivated students in a low-stakes assessment context. Applied Measurement in Education, 24(2), 162–188. doi:10.1080/08957347.2011.555217
   Web of Science ®Google Scholar
• Wise, S. L. (2006). An investigation of the differential effort received by items on a low-stakes computer-based test. Applied Measurement in Education, 19(2), 95–114. doi:10.1207/s15324818ame1902_2
   Web of Science ®Google Scholar
• Wise, S. L. (2015). Effort analysis: Individual score validation of achievement test data. Applied Measurement in Education, 28(3), 237–252. doi:10.1080/08957347.2015.1042155
   Web of Science ®Google Scholar
• Wise, S. L. (2017). Rapid‐guessing behavior: Its identification, interpretation, and implications. Educational Measurement: Issues and Practice, 36(4), 52–61. doi:10.1111/emip.2017.36.issue-4
   Web of Science ®Google Scholar
• Wise, S. L., & DeMars, C. E. (2006). An application of item response time: The effort-moderated IRT model. Journal of Educational Measurement, 43(1), 19–38. doi:10.1111/jedm.2006.43.issue-1
   Web of Science ®Google Scholar
• Wise, S. L., & Kingsbury, G. G. (2016). Modeling student test‐taking motivation in the context of an adaptive achievement test. Journal of Educational Measurement, 53(1), 86–105. doi:10.1111/jedm.2016.53.issue-1
   Web of Science ®Google Scholar
• Wise, S. L., & Kong, X. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18(2), 163–183. doi:10.1207/s15324818ame1802_2
   Web of Science ®Google Scholar
• Wise, S. L., & Ma, L. (2012). Setting response time thresholds for a CAT item pool: The normative threshold method. In annual meeting of the National Council on Measurement in Education, Vancouver, Canada.
   Google Scholar
• Wise, S. L., & Smith, L. F. (2016). The validity of assessment when students don’t give good effort. In Handbook of human and social conditions in assessment, 204–220.
   Google Scholar