Confidence ratings are better predictors of future performance than delayed judgments of learning

References

• Arnold, K. M., & McDermott, K. B. (2013). Test-potentiated learning: Distinguishing between direct and indirect effects of tests. Journal of Experimental Psychology. Learning, Memory, and Cognition, 39(3), 940–945. https://doi.org/10.1037/a0029199
   PubMed Web of Science ®Google Scholar
• Benjamin, A. S., & Bjork, R. A. (1996). Retrieval fluency as a metacognitive index. In L. M. Reder (Ed.), Implicit memory and metacognition (pp. 309–338). Erlbaum.
   Google Scholar
• Busey, T. A., Tunnicliff, J., Loftus, G. R., & Loftus, E. F. (2000). Accounts of the confidence-accuracy relation in recognition memory. Psychonomic Bulletin & Review, 7(1), 26–48. https://doi.org/10.3758/BF03210724
   PubMed Web of Science ®Google Scholar
• Cumming, G. (2012). Understanding the new statistics. Routledge.
   Google Scholar
• Diaz, M., & Benjamin, A. S. (2011). The effects of proactive interference (PI) and release from PI on judgments of learning. Memory & Cognition, 39(2), 196–203. https://doi.org/10.3758/s13421-010-0010-y
   PubMed Web of Science ®Google Scholar
• Dougherty, M. R., Robey, A. M., & Buttaccio, D. (2018). Do metacognitive judgments alter memory performance beyond the benefits of retrieval practice? A comment on and replication attempt of dougherty, scheck, Nelson, and narens (2005). Memory & Cognition, 46(4), 558–565. https://doi.org/10.3758/s13421-018-0791-y
   PubMed Web of Science ®Google Scholar
• Dougherty, M. R., Scheck, P., Nelson, T. O., & Narens, L. (2005). Using the past to predict the future. Memory and Cognition, 33(6), 1096–1115. http://doi.org/10.3758/BF03193216
   PubMed Web of Science ®Google Scholar
• Dunlosky, J., & Metcalfe, J. (2009). Metacognition. SAGE.
   Google Scholar
• Dunlosky, J., & Tauber, S. K. (2014). Understanding people's metacognitive judgments: An isomechanism framework and its implications for applied and theoretical research. In T. Perfect, & D. S. Lindsay (Eds.), Handbook of applied memory (pp. 444–463). Sage.
   Google Scholar
• England, B. D., Ortegren, F. R., & Serra, M. J. (2017). Framing affects scale usage for judgments of learning, not confidence in memory. Journal of Experimental Psychology. Learning, Memory, and Cognition, 43(12), 1898–1908. https://doi.org/10.1037/xlm0000420
   PubMed Web of Science ®Google Scholar
• Estes, W. K., Hopkins, B. L., & Crothers, E. J. (1960). All-or-none and conservation effects in the learning and retention of paired associates. Journal of Experimental Psychology, 60(6), 329–339. http://doi.org/10.1037/h0043400
   PubMedGoogle Scholar
• Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. SAGE.
   Google Scholar
• Finn, B. (2008). Framing effects on metacognitive monitoring and control. Memory & Cognition, 36(4), 813–821. https://doi.org/10.3758/MC.36.4.813
   PubMed Web of Science ®Google Scholar
• Franklin, K. (2013, January 3) “The Best Predictor of Future Behavior Is … Past Behavior”: Does the popular maxim hold water? Psychology Today. https://www.psychologytoday.com/us/blog/witness/201301/the-best-predictor-future-behavior-is-past-behavior
   Google Scholar
• Glenberg, A. M., Sanocki, T., Epstein, W., & Morris, C. (1987). Enhancing calibration of comprehension. Journal of Experimental Psychology. General, 116(2), 119–136. https://doi.org/10.1037/0096-3445.116.2.119
   Web of Science ®Google Scholar
• Higham, P. A., & Higham, D. P. (2019). New improved gamma: Enhancing the accuracy of Goodman-Kruskal's gamma using ROC curves. Behavior Research Methods, 51, 108–125. https://doi.org/10.3758/s13428-018-1125-5
   PubMed Web of Science ®Google Scholar
• Kelley, C. M., & Lindsay, D. S. (1993). Remembering mistaken for knowing: Ease of retrieval as a basis for confidence in answers to general knowledge questions. Journal of Memory and Language, 32(1), 1–24. https://doi.org/10.1006/jmla.1993.1001
   Web of Science ®Google Scholar
• Kimball, D., & Metcalfe, J. (2003). Delaying judgments of learning affects memory, not metamemory. Memory and Cognition, 31(6), 918–929. https://doi.org/10.3758/BF03196445
   PubMed Web of Science ®Google Scholar
• Koriat, A. (1997). Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning. Journal of Experimental Psychology: General, 126(4), 349–370. https://doi.org/10.1037/0096-3445.126.4.349
   Web of Science ®Google Scholar
• Koriat, A. (2012). The self-consistency model of subjective confidence. Psychological Review, 119(1), 80–113. http://doi.org/10.1037/a0025648
   PubMed Web of Science ®Google Scholar
• Koriat, A. (2015). When two heads are better than one and when they can be worse: The amplification hypothesis. Journal of Experimental Psychology: General, 144(5), 934–950. http://doi.org/10.1037/xge0000092
   PubMed Web of Science ®Google Scholar
• Koriat, A., Bjork, R. A., Sheffer, L., & Bar, S. K. (2004). Predicting one’s own forgetting: The role of experience-based and theory-based processes. Journal of Experimental Psychology: General, 133(4), 643–656. http://doi.org/10.1037/0096-3445.133.4.643
   PubMed Web of Science ®Google Scholar
• Koriat, A., & Goldsmith, M. (1996). Monitoring and control processes in the strategic regulation of memory accuracy. Psychological Review, 103(3), 490–517. https://doi.org/10.1037/0033-295X.103.3.490
   PubMed Web of Science ®Google Scholar
• Lovelace, E. A. (1984). Metamemory: Monitoring future recallability during study. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10(4), 756–766. https://doi.org/10.1037/0278-7393.10.4.756
   Web of Science ®Google Scholar
• Luna, K., Martín-Luengo, B., & Albuquerque, P. B. (2018). Do delayed judgements of learning reduce metamemory illusions? A meta-analysis. Quarterly Journal of Experimental Psychology, 71(7), 1626–1636. https://doi.org/10.1080/17470218.2017.1343362
   Web of Science ®Google Scholar
• Lyon, T. D., & Flavell, J. H. (1993). Young children’s understanding of forgetting over time. Child Development, 64(3), 789–800. https://doi.org/10.2307/1131218
   PubMed Web of Science ®Google Scholar
• McDaniel, M. A., & Bugg, J. M. (2008). Instability in memory phenomena: A common puzzle and a unifying explanation. Psychonomic Bulletin & Review, 15(2), 237–255. https://doi.org/10.3758/PBR.15.2.237
   PubMed Web of Science ®Google Scholar
• Metcalfe, J., & Finn, B. (2008). Evidence that judgments of learning are causally related to study choice. Psychonomic Bulletin & Review, 15(1), 174–179. https://doi.org/10.3758/PBR.15.1.174
   PubMed Web of Science ®Google Scholar
• Murayama, K., Sakaki, M., Yan, V. X., & Smith, G. M. (2014). Type I error inflation in the traditional by-participant analysis to metamemory accuracy: A generalized mixed-effects model perspective. Journal of Experimental Psychology. Learning, Memory, and Cognition, 40(5), 1287–1306. https://doi.org/10.1037/a0036914
   PubMed Web of Science ®Google Scholar
• Nelson, T. O. (1984). A comparison of current measures of the accuracy of feeling-of-knowing predictions. Psychological Bulletin, 95(1), 109–133. http://doi.org/10.1037/0033-2909.95.1.109
   PubMed Web of Science ®Google Scholar
• Nelson, T. O., & Dunlosky, J. (1991). When people’s judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: The ‘delayed-JOL effect’. Psychological Science, 2(4), 267–270. https://doi.org/10.1111/j.1467-9280.1991.tb00147.x
   Web of Science ®Google Scholar
• Nelson, T. O., Narens, L., & Dunlosky, J. (2004). A revised methodology for research on metamemory: Pre-judgment recall and monitoring (PRAM). Psychological Methods, 9(1), 53–69. https://doi.org/10.1037/1082-989X.9.1.53
   PubMed Web of Science ®Google Scholar
• Pierce, B. H., & Smith, S. M. (2001). The postdiction superiority effect in metacomprehension of text. Memory & Cognition, 29(1), 62–67. https://doi.org/10.3758/BF03195741
   PubMed Web of Science ®Google Scholar
• Putnam, A. L., Ozubko, J. D., MacLeod, C. M., & Roediger, H. L., III. (2014). The production effect in paired-associate learning: Benefits for item and associative information. Memory & Cognition, 42(3), 409–420. https://doi.org/10.3758/s13421-013-0374-x
   PubMed Web of Science ®Google Scholar
• Putnam, A. L., & Roediger, H. L., III. (2013). Does response mode affect amount recalled or the magnitude of the testing effect? Memory & Cognition, 41(1), 36–48. https://doi.org/10.3758/s13421-012-0245-x
   PubMed Web of Science ®Google Scholar
• Rawson, K. A., Dunlosky, J., & Mc Donald, S. L. (2002). Influences of metamemory on performance predictions for text. The Quarterly Journal of Experimental Psychology, 55(2), 505–524. https://doi.org/10.1080/02724980143000352
   Web of Science ®Google Scholar
• Rhodes, M., & Castel, A. D. (2008). Memory predictions are influence by perceptual information: Evidence for metacognitive illusions. Journal of Experimental Psychology: General, 137(4), 612–625. http://doi.org/10.1037/a0013684
   Web of Science ®Google Scholar
• Rhodes, M. G. (2016). Judgments of learning. In J. Dunlosky, & S. K. Tauber (Eds.), The Oxford Handbook of metamemory (pp. 1–32). Oxford University Press.
   Google Scholar
• Roediger, H. L., III, Putnam, A. L., & Smith, M. A. (2011). Ten benefits of testing and their applications to educational practice. In J. P. Mestre & B. H. Ross (Eds.), The psychology of learning and motivation: Cognition in education, Vol. 55, (pp. 1–36). Elsevier Academic Press. https://doi.org/10.1016/B978-0-12-387691-1.00001-6
   Google Scholar
• Serra, M. J., & England, B. D. (2012). Magnitude and accuracy differences between judgements of remembering and forgetting. Quarterly Journal of Experimental Psychology, 65(11), 2231–2257. https://doi.org/10.1080/17470218.2012.685081
   PubMed Web of Science ®Google Scholar
• Smith, M. A., Roediger, H. L., & Karpicke, J. D. (2013). Covert retrieval practice benefits retention as much as overt retrieval practice. Journal of Experimental Psychology. Learning, Memory, and Cognition, 39(6), 1712–1725. https://doi.org/10.1037/a0033569
   PubMed Web of Science ®Google Scholar
• Spellman, B. A., & Bjork, R. A. (1992). When predictions create reality: Judgments of learning may alter what they are intended to assess. Psychological Science, 3(5), 315–316. https://doi.org/10.1111/j.1467-9280.1992.tb00680.x
   Web of Science ®Google Scholar
• Susser, J. A., & Mulligan, N. W. (2019). Exploring the intrinsic-extrinsic distinction in prospective metamemory. Journal of Memory and Language, 104, 43–55. https://doi.org/10.1016/j.jml.2018.09.003
   Web of Science ®Google Scholar
• Tauber, S. K., & Rhodes, M. G. (2010). Does the amount of material to be remembered influence judgements of learning (JOLS)? Memory, 18(3), 351–362. https://doi.org/10.1080/09658211003662755
   PubMed Web of Science ®Google Scholar
• Tauber, S. K., & Rhodes, M. G. (2012). Measuring memory monitoring with judgements of retention (JORs). The Quarterly Journal of Experimental Psychology, 65(7), 1376–1396. https://doi.org/10.1080/17470218.2012.656665
   PubMed Web of Science ®Google Scholar
• Thomas, R. C., Finn, B., & Jacoby, L. L. (2016). Prior experience shapes metacognitive judgments at the category level: The role of testing and category difficulty. Metacognition and Learning, 11(3), 257–274. https://doi.org/10.1007/s11409-015-9144-4
   Web of Science ®Google Scholar
• Tulving, E. (1964). Intratrial and intertrial retention: Notes towards a theory of free recall verbal learning. Psychological Review, 71(3), 219–237. http://doi.org/10.1037/h0043186
   PubMed Web of Science ®Google Scholar
• Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1-48. https://www.jstatsoft.org/v36/i03 https://doi.org/10.18637/jss.v036.i03
   Web of Science ®Google Scholar