Building Better Theories: Prediction Intervals as a Tool for Theory Testing and Improvement

References

• Adner, R., Pólos, L., Ryall, M., & Sorenson, O. (2009). The case for formal theory. Academy of Management Review, 34(2), 201–208. https://doi.org/10.5465/amr.2009.36982613
   Web of Science ®Google Scholar
• Aramovich, N. P., & Larson, J. R. Jr. (2013). Strategic demonstration of problem solutions by groups: The effects of member preferences, confidence, and learning goals. Organizational Behavior and Human Decision Processes, 122(1), 36–52. https://doi.org/10.1016/j.obhdp.2013.04.001
   Web of Science ®Google Scholar
• Boero, R., & Squazzoni, F. (2005). Does empirical embeddedness matter? Methodological issues on agent-based models for analytical social science. Journal of Artificial Societies and Social Simulation, 8(4), 6. http://jasss.soc.surrey.ac.uk/8/4/6.html
   Web of Science ®Google Scholar
• Brodbeck, F. C., Kerschreiter, R., Mojzisch, A., Frey, D., & Schulz-Hardt, S. (2002). The dissemination of critical, unshared information in decision-making groups: The effects of pre-discussion dissent. European Journal of Social Psychology, 32(1), 35–56. https://doi.org/10.1002/ejsp.74
   Web of Science ®Google Scholar
• Broers, N. J. (2021). When the numbers do not add up: The practical limits of stochastologicals for soft psychology. Perspectives on Psychological Science, 16(4), 698–706. https://doi.org/10.1177/1745691620970557
   Web of Science ®Google Scholar
• Chattoe-Brown, E. (2023). Is agent-based modelling the future of prediction? International Journal of Social Research Methodology, 26(2), 143–155. https://doi.org/10.1080/13645579.2022.2137923
   Web of Science ®Google Scholar
• Conte, R., & Paolucci, M. (2014). On agent-based modeling and computational social science. Frontiers in Psychology, 5, 668. https://doi.org/10.3389/fpsyg.2014.00668
   PubMed Web of Science ®Google Scholar
• Conroy-Beam, D. (2021). Couple simulation: A novel approach for evaluating models of human mate choice. Personality and Social Psychology Review , 25(3), 191–228. https://doi.org/10.1177/1088868320971
   PubMed Web of Science ®Google Scholar
• Dabbs, J. M., Jr., & Ruback, R. B. (1987). Dimensions of group process: Amount and structure of vocal interaction. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 20, pp. 123–169). Academic Press.
   Google Scholar
• Eberlen, J., Scholz, G., & Gagliolo, M. (2017). Simulate this! An introduction to agent-based models and their power to improve your research practice. International Review of Social Psychology, 30(1), 149–160. https://doi.org/10.5334/irsp.115
   Web of Science ®Google Scholar
• Edmonds, B. (2010). Bootstrapping knowledge about social phenomena using simulation models. Journal of Artificial Societies and Social Simulation, 13(1), 8. http://jasss.soc.surrey.ac.uk/13/1/8.html https://doi.org/10.18564/jasss.1523
   Web of Science ®Google Scholar
• Edmonds, B., Le Page, C., Bithell, M., Chattoe-Brown, E., Grimm, V., Meyer, R., Montañola-Sales, C., Ormerod, P., Root, H., & I Squazzoni, F. (2019). Different modelling purposes. Journal of Artificial Societies and Social Simulation, 22(3), 6. https://doi.org/10.18564/jasss.3993
   Web of Science ®Google Scholar
• Epstein, J. M. (2008). Why model? Journal of Artificial Societies and Social Simulation, 11(4), 12. http://jasss.soc.surrey.ac.uk/11/4/12.html
   Web of Science ®Google Scholar
• Epstein, J. M. (2013). Agent_zero: Toward neurocognitive foundations for generative social science. Princeton University Press.
   Google Scholar
• Fiske, D. W. (1986). Specificity of method and knowledge in social science. In D. W. Fiske & R. A. Shweder (Eds.), Metatheory in social science: Pluralisms and subjectivities (pp. 61–82). University of Chicago Press.
   Google Scholar
• Flache, A., Mäs, M., Feliciani, T., Chattoe-Brown, E., Deffuant, G., Huet, S., & Lorenz, J. (2017). Models of social influence: Towards the next frontiers. Journal of Artificial Societies and Social Simulation, 20(4), 2. https://doi.org/10.18564/jasss.3521
   Web of Science ®Google Scholar
• Gates, V., Suchow, J. W., & Griffiths, T. L. (2022). Memory transmission in small groups and large networks: An empirical study. Psychonomic Bulletin & Review, 29(2), 581–588. https://doi.org/10.3758/s13423-021-02021-9
   PubMed Web of Science ®Google Scholar
• Gawronski, B., & Bodenhausen, G. V. (2015). Theory evaluation. In B. Gawronski & G. V. Bodenhausen (Eds.), Theory and explanation in social psychology (pp. 3–23). Guilford Press.
   Google Scholar
• Geisser, S. (1993). Predictive inference: An introduction. CRC Press. https://doi.org/10.1007/978-1-4899-4467-2
   Google Scholar
• Gershman, S. J. (2016). Empirical priors for reinforcement learning models. Journal of Mathematical Psychology, 71, 1–6. https://doi.org/10.1016/j.jmp.2016.01.006
   Web of Science ®Google Scholar
• Glöckner, A., & Betsch, T. (2011). The empirical content of theories in judgment and decision making: Shortcomings and remedies. Judgment and Decision Making, 6(8), 711–721. http://journal.sjdm.org/11/m38/m38.html https://doi.org/10.1017/S1930297500004149
   Web of Science ®Google Scholar
• Grahek, I., Schaller, M., & Tackett, J. L. (2021). Anatomy of a psychological theory: Integrating construct-validation and computational-modeling methods to advance theorizing. Perspectives on Psychological Science, 16(4), 803–815. https://doi.org/10.1177/1745691620966794
   Web of Science ®Google Scholar
• Grice, J. W. (2011). Observation oriented modeling: Analysis of cause in the behavioral sciences. Academic Press.
   Google Scholar
• Grice, J. W. (2015). From means and variances to persons and patterns. Frontiers in Psychology, 6, 1007. https://doi.org/10.3389/fpsyg.2015.01007
   PubMed Web of Science ®Google Scholar
• Hastie, R., & Stasser, G. (2000). Computer simulation methods in social psychology. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (pp. 85–114). Cambridge University Press.
   Google Scholar
• Hayes, A. F. (2018). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach (2nd ed.). Guilford Press.
   Google Scholar
• Hoffman, R. L. (1979). The group problem solving process: Studies of a valence model. Praeger.
   Google Scholar
• Hyndman, R. J., & Athanasopoulos, G. (2018). Forecasting: Principles and practice. OTexts. https://www.OTexts.com/fpp2
   Google Scholar
• Jackson, J. C., Rand, D., Lewis, K., Norton, M. I., & Gray, K. (2017). Agent-based modeling: A guide for social psychologists. Social Psychological and Personality Science, 8(4), 387–395. https://doi.org/10.1177/1948550617691100
   Web of Science ®Google Scholar
• Jung, J., Bramson, A., & Crano, W. D. (2018). An agent-based model of indirect minority influence on social change and diversity. Social Influence, 13(1), 18–38. https://doi.org/10.1080/15534510.2017.1415961
   Web of Science ®Google Scholar
• Lakatos, I. (1978). The methodology of scientific research programmes. Vol. 1. Philosophical papers. Cambridge University Press.
   Google Scholar
• Larson, J. R. Jr. (1997). Modeling the entry of shared and unshared information into group discussion: A review and BASIC language computer program. Small Group Research, 28(3), 454–479. https://doi.org/10.1177/1046496497283007
   Web of Science ®Google Scholar
• Larson, J. R. Jr. (2003). Information centrality in decision-making groups: NSF final report for grant #SBR-9809207. Unpublished Report. Department of Psychology, University of Illinois at Chicago.
   Google Scholar
• Larson, J. R. Jr. (2012). Computer simulation methods for groups: From formula translation to agent-based modeling. In A. B. Hollingshead & M. S. Poole (Eds.), Research methods for studying groups and teams: A guide to approaches, tools, and technologies (pp. 79–104). Taylor & Francis/Routledge.
   Google Scholar
• Larson, J. R., Jr., Christensen, C., Franz, T. M., & Abbott, A. S. (1998). Diagnosing groups: The pooling, management, and impact of shared and unshared case information in team-based medical decision making. Journal of Personality and Social Psychology, 75(1), 93–108. https://doi.org/10.1037/0022-3514.75.1.93
   PubMed Web of Science ®Google Scholar
• Larson, J. R., Jr., & Egan, A. C. (2020). Information sharing within groups in organizations: Situational and motivational influences. In L. Argote & J. M. Levine (Eds.), The Oxford handbook of group and organizational learning (pp 127–153). Oxford University Press.
   Google Scholar
• Larson, J. R., Jr., Sargis, E. G., & Bauman, C. W. (2004). Shared knowledge and subgroup influence during decision-making discussions. Journal of Behavioral Decision Making, 17(4), 245–262. https://doi.org/10.1002/bdm.462
   Web of Science ®Google Scholar
• Latané, B. (1996). Dynamic social impact: Robust predictions from simple theory. In R. Hegselmann, U. Mueller, & K. G. Troitzsch (Eds.), Modeling and simulation in the social sciences from the philosophy of science point of view (pp. 287–310). Kluwer.
   Google Scholar
• Latané, B. (2000). Pressure to uniformity and the evolution of cultural norms: Modeling dynamic social impact. In D. R. Ilgen & C. L. Hulin (Eds), Computational modeling of behavior in organizations: The third scientific discipline (pp. 189–215). American Psychological Association.
   Google Scholar
• Laughlin, P. R. (2011). Group problem solving. Princeton University Press.
   Google Scholar
• Laughlin, P. R., & Ellis, A. L. (1986). Demonstrability and social combination processes on mathematical intellective tasks. Journal of Experimental Social Psychology, 22(3), 177–189. https://doi.org/10.1016/0022-1031(86)90022-3
   Web of Science ®Google Scholar
• Letzter, R. (2020, January 28). 2 satellites will narrowly avoid colliding at 32,800 mph over Pittsburgh on Wednesday. Space.com. Retrieved from https://www.space.com
   Google Scholar
• Lewin, K. (1951). Field theory in social science. Harper.
   Google Scholar
• Lorenz, J., Neumann, M., & Schröder, T. (2021). individual attitude change and societal dynamics: Computational experiments with psychological theories. Psychological Review, 128(4), 623–642. https://doi.org/10.1037/rev0000291
   PubMed Web of Science ®Google Scholar
• Lu, L., Yuan, Y. C., & McLeod, P. L. (2012). Twenty-five years of hidden profiles in group decision making: A meta-analysis. Personality and Social Psychology Review, 16(1), 54–75. https://doi.org/10.1177/1088868311417243
   PubMed Web of Science ®Google Scholar
• Luhmann, C. C., & Rajaram, S. (2015). Memory transmission in small groups and large networks: An agent-based model. Psychological Science, 26(12), 1909–1917. https://doi.org/10.1177/0956797615605798
   PubMed Web of Science ®Google Scholar
• Marion, S. B., & Thorley, C. (2016). A meta-analytic review of collaborative inhibition and post-collaborative memory: Testing the predictions of the retrieval strategy disruption hypothesis. Psychological Bulletin, 142(11), 1141–1164. https://doi.org/10.1037/bul0000071
   PubMed Web of Science ®Google Scholar
• Meehl, P. E. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant using it. Psychological Inquiry, 1(2), 108–141. https://doi.org/10.1207/s15327965pli0102_1
   Google Scholar
• Miller, J., & Schwarz, W. (2011). Aggregate and individual replication probability within an explicit model of the research process. Psychological Methods, 16(3), 337–360. https://doi.org/10.1037/a0023347
   PubMed Web of Science ®Google Scholar
• Muthukrishna, M., & Schaller, M. (2020). Are collectivistic cultures more prone to rapid transformation? Computational models of cross-cultural differences, social network structure, dynamic social influence, and cultural change. Personality and Social Psychology Review , 24(2), 103–120. https://doi.org/10.1177/1088868319855783
   PubMed Web of Science ®Google Scholar
• Neyman, J. (1937). Outline of a theory of statistical estimation based on the classical theory of probability. Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences, 236 (767), 333–380. https://doi.org/10.1098/rsta.1937.0005
   Google Scholar
• Oberauer, K., & Lewandowsky, S. (2019). Addressing the theory crisis in psychology. Psychonomic Bulletin & Review, 26(5), 1596–1618. https://doi.org/10.3758/s13423-019-01645-2
   PubMed Web of Science ®Google Scholar
• Oh, W., Moon, J. Y., Hahn, J., & Kim, T. (2016). Leader influence on sustained participation in online collaborative work communities: A simulation-based approach. Information Systems Research, 27(2), 383–402. https://doi.org/10.1287/isre.2016.0632
   Web of Science ®Google Scholar
• Ostrom, T. M. (1988). Computer simulation: The third symbol system. Journal of Experimental Social Psychology, 24(5), 381–392. https://doi.org/10.1016/0022-1031(88)90027-3
   Web of Science ®Google Scholar
• Parker, K. C. H. (1988). Speaking turns in small group interaction: A context-sensitive event sequence model. Journal of Personality and Social Psychology, 54(6), 965–971. https://doi.org/10.1037/0022-3514.54.6.965
   Web of Science ®Google Scholar
• Patil, P., Peng, R. D., & Leek, J. T. (2016). What should researchers expect when they replicate studies? A statistical view of replicability in psychological science. Perspectives on Psychological Science , 11(4), 539–544. https://doi.org/10.1177/1745691616646366
   Web of Science ®Google Scholar
• Popper, K. (1959). The logic of scientific discovery. Hutchinson & Co.
   Google Scholar
• Rajaram, S., & Pereira-Pasarin, L. P. (2010). Collaborative memory: Cognitive research and theory. Perspectives on Psychological Science, 5(6), 649–663. https://doi.org/10.1177/1745691610388763
   Web of Science ®Google Scholar
• Read, S. J., & Monroe, B. M. (2023). Computational models in personality and social psychology. In R. Sun (Ed.), The Cambridge handbook of computational cognitive science (2nd. ed., Vol. 2, pp 795–835). Cambridge University Press.
   Google Scholar
• Reise, S. P., Ainsworth, A. T., & Haviland, M. G. (2005). Item response theory: Fundamentals, applications, and promise in psychological research. Current Directions in Psychological Science, 14(2), 95–101. https://doi.org/10.1111/j.0963-7214.2005.00342.x
   Web of Science ®Google Scholar
• Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current theory and research (pp. 64–99). Appleton-Century-Crofts.
   Google Scholar
• Rocca, R., & Yarkoni, T. (2021). Putting psychology to the test: Rethinking model evaluation through benchmarking and prediction. Advances in Methods and Practices in Psychological Science, 4(3), 251524592110268. https://doi.org/10.1177/25152459211026864
   Google Scholar
• Shaw, M. E., & Costanzo, P. R. (1970). Theories of social psychology. McGraw Hill.
   Google Scholar
• Simon, H. A. (1992). What is an "explanation" of behavior? Psychological Science, 3(3), 150–161. https://doi.org/10.1111/j.1467-9280.1992.tb00017.x
   Web of Science ®Google Scholar
• Smaldino, P. E. (2020). How to translate a verbal theory into a formal model. Social Psychology, 51(4), 207–218. https://doi.org/10.1027/1864-9335/a000425
   Web of Science ®Google Scholar
• Smaldino, P. E. (2023). Modeling social behavior: Mathematical and agent-based models of social dynamics and cultural evolution. Princeton University Press.
   Google Scholar
• Smaldino, P. E., Calanchini, J., & Pickett, C. L. (2015). Theory development with agent-based models. Organizational Psychology Review, 5(4), 300–317. https://doi.org/10.1177/2041386614546944
   Web of Science ®Google Scholar
• Smaldino, P. E., & Turner, M. A. (2022). Covert signaling is an adaptive communication strategy in diverse populations. Psychological Review, 129(4), 812–829. https://doi.org/10.1037/rev0000344
   PubMed Web of Science ®Google Scholar
• Smiley, A. H., Glazier, J. J., & Shoda, Y. (2022). Large enough to matter, too small to be of consequence, or insufficient evidence? A unified framework for a more meaningful statistical inference. . Department of Psychology, University of Washington. https://psyarxiv.com/465ha/
   Google Scholar
• Smith, E. R., & Beasley, A. (2015). Agent-based modeling. In B. Gawronski and G. V. Bodenhausen (Eds.), Theory and explanation in social psychology. (390–407)NY, Guilford Press.
   Google Scholar
• Sohrab, S. G., Waller, M. J., & Kaplan, S. (2015). Exploring the hidden-profile paradigm: A literature review and analysis. Small Group Research, 46(5), 489–535. https://doi.org/10.1177/1046496415599068
   Web of Science ®Google Scholar
• Stasser, G. (1988). Computer simulation as a research tool: The DISCUSS model of group decision making. Journal of Experimental Social Psychology, 24(5), 393–422. https://doi.org/10.1016/0022-1031(88)90028-5
   Web of Science ®Google Scholar
• Stasser, G. (1999). A primer of social decision scheme theory: Models of group influence, competitive model-testing, and prospective modeling. Organizational Behavior and Human Decision Processes, 80(1), 3–20. https://doi.org/10.1006/obhd.1999.2851
   PubMed Web of Science ®Google Scholar
• Stasser, G., & Taylor, L. A. (1991). Speaking turns in face-to-face discussions. Journal of Personality and Social Psychology, 60(5), 675–684. https://doi.org/10.1037/0022-3514.60.5.675
   Web of Science ®Google Scholar
• Stasser, G., & Titus, W. (2003). Hidden profiles: A brief history. Psychological Inquiry, 14(3), 304–313. https://doi.org/10.1080/1047840X.2003.9682897
   Web of Science ®Google Scholar
• Stephan, F. F., & Mishler, E. G. (1952). The distribution of participation in small groups: An exponential approximation. American Sociological Review, 17(5), 598. https://doi.org/10.2307/2088227
   Web of Science ®Google Scholar
• Sun, R. (2009). Theoretical status of computational cognitive modeling. Cognitive Systems Research, 10(2), 124–140. https://doi.org/10.1016/j.cogsys.2008.07.002
   Web of Science ®Google Scholar
• Świątkowski, W., & Carrier, A. (2020). There is nothing magical about Bayesian statistics: An introduction to epistemic probabilities in data analysis for psychology starters. Basic and Applied Social Psychology, 42(6), 387–412. https://doi.org/10.1080/01973533.2020.1792297
   Web of Science ®Google Scholar
• Taylor, L. A. (1987). Communication style and majority versus minority influence in decision-making groups. Unpublished Master’s thesis. Miami University.
   Google Scholar
• Thorngate, W., & Edmonds, B. (2013). Measuring simulation-observation fit: An introduction to ordinal pattern analysis. Journal of Artificial Societies and Social Simulation, 16(2), 4. http://jasss.soc.surrey.ac.uk/16/2/4.html https://doi.org/10.18564/jasss.2139
   Web of Science ®Google Scholar
• Trope, Y. (2004). Theory in social psychology: Seeing the forest and the trees. Personality and Social Psychology Review, 8(2), 193–200. https://doi.org/10.1207/s15327957pspr0802_13
   PubMed Web of Science ®Google Scholar
• Tran, N.-H., van Maanen, L., Heathcote, A., & Matzke, D. (2021). Systematic parameter reviews in cognitive modeling: Towards a robust and cumulative characterization of psychological processes in the diffusion decision model. Frontiers in Psychology, 11, 608287. https://doi.org/10.3389/fpsyg.2020.608287
   PubMed Web of Science ®Google Scholar
• Trafimow, D. (2023). A new way to think about internal and external validity. Perspectives on Psychological Science, 18(5), 1028–1046. https://doi.org/10.1177/17456916221136117
   Web of Science ®Google Scholar
• Van Lange, P. A. M. (2013). What we should expect from theories in social psychology: Truth, abstraction, progress, and applicability as standards (TAPAS). Personality and Social Psychology Review, 17(1), 40–55. https://doi.org/10.1177/1088868312453088
   PubMed Web of Science ®Google Scholar
• Vanpaemel, W. (2020). Strong theory testing using the prior predictive and the data prior. Psychological Review, 127(1), 136–145. https://doi.org/10.1037/rev0000167
   PubMed Web of Science ®Google Scholar
• van Rooij, I., & Baggio, G. (2021). Theory before the test: How to build high verisimilitude explanatory theories in psychological science. Perspectives on Psychological Science, 16(4), 682–697. https://doi.org/10.1177/1745691620970604
   Web of Science ®Google Scholar
• Van Rooy, D., Wood, I., & Tran, E. (2016). Modelling the emergence of shared attitudes from group dynamics using an agent-based model of social comparison theory. Systems Research and Behavioral Science, 33(1), 188–204. https://doi.org/10.1002/sres.2321
   Web of Science ®Google Scholar
• Wagenmakers, E.-J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., Matzke, D., Rouder, J. N., & Morey, R. D. (2018). Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications. Psychonomic Bulletin & Review, 25(1), 35–57. https://doi.org/10.3758/s13423-017-1343-3
   PubMed Web of Science ®Google Scholar
• Wagenmakers, E.-J., Love, J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Selker, R., Gronau, Q. F., Dropmann, D., Boutin, B., Meerhoff, F., Knight, P., Raj, A., van Kesteren, E.-J., van Doorn, J., Šmíra, M., Epskamp, S., Etz, A., Matzke, D., … Morey, R. D. (2018). Bayesian inference for psychology. Part II: Example applications with JASP. Psychonomic Bulletin & Review, 25(1), 58–76. https://doi.org/10.3758/s13423-017-1323-7
   PubMed Web of Science ®Google Scholar
• Wagenmakers, E.-J., Morey, R. D., & Lee, M. D. (2016). Bayesian benefits for the pragmatic researcher. Current Directions in Psychological Science, 25(3), 169–176. https://doi.org/10.1177/0963721416643289
   Web of Science ®Google Scholar
• Ward, M., & Kemp, S. (2019). The probability of conceptual replication and the variability of effect size. Methods in Psychology, 1, 100002. https://doi.org/10.1016/j.metip.2019.100002
   Google Scholar
• Weinhardt, J. M., & Vancouver, J. B. (2012). Computational models and organizational psychology: Opportunities abound. Organizational Psychology Review, 2(4), 267–292. https://doi.org/10.1177/2041386612450455
   Web of Science ®Google Scholar
• Weisberg, M. (2013). Simulation and simplicity: Using models to understand the world. Oxford University Press.
   Google Scholar
• Weldon, M. S., & Bellinger, K. D. (1997). Collective memory: Collaborative and individual processes in remembering. Journal of Experimental Psychology, 23(5), 1160–1175. https://doi.org/10.1037/0278-7393.23.5.1160
   Google Scholar
• Winquist, J. R., & Larson, J. R. Jr. (1998). Information pooling: When it impacts group decision making. Journal of Personality and Social Psychology, 74(2), 371–377. https://doi.org/10.1037/0022-3514.74.2.371
   Web of Science ®Google Scholar
• Wood, M. (2005). Bootstrapped confidence intervals as an approach to statistical inference. Organizational Research Methods, 8(4), 454–470. https://doi.org/10.1177/1094428105280059
   Web of Science ®Google Scholar