Irreplicability in methodology: embracing the historical contingencies of educational technology research during the 2020–2021 United States school year

ABSTRACT

The claim that research should prioritize qualities of replicability is foundational to scientific knowledge production. At the same time, replication is not an algorithm but instead based on a disciplinary community’s values and norms. With this understanding, we reflect on how varying degrees of replicability and irreplicability are always co-constituted in research projects through historically situated relationships between humans, methods, and technologies. We explore our own research collaborations and choices during the 2020–2021 US school year – defined by the COVID-19 pandemic and emergency remote teaching – to recast irreplicability not as a mark of pseudo-science but instead as an inherent part of studying teaching and learning during emergency conditions. We argue that explicit attention to irreplicability can not only be generative in research design but can help researchers think about themselves as historical actors who are accountable to the particularities and contingencies of the conditions in which they work.

KEYWORDS:

• 2020 studies
• replicability
• pandemic pedagogy
• COVID-19
• Black Lives Matter

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. Shapin and Schaffer, Leviathan and the Air-Pump; and Collins, Changing Order.

2. Popper, The Logic of Scientific Discovery; and Schmidt, “Shall We Really Do it Again?” 90–100.

3. Romero, “Philosophy of Science and the Replicability Crisis,” 1–14.

4. Makel and Plucker, “Facts are More Important than Novelty,” 304–316.

5. Leonelli, “Rethinking Reproducibility as a Criterion for Research Quality,” 129–146.

6. Pickering, Constructing Quarks.

7. Collins, Changing Order; and Hess, Science Studies.

8. Aguliera and Nightengale-Lee, “Emergency Remote Teaching,” 471–478.

9. e.g., Cook, “Quasi‐Experimental Design.”

10. e.g., Sandoval, “Developing Learning Theory,” 213–223; and Kafai, “The Classroom as ‘Living Laboratory’,” 28–34.

11. Kuhn, The Structures of Scientific Revolution.

12. Schmidt, “Shall We Really Do It Again?” 90–100.

13. Leonelli, “Rethinking Reproducibility,” 129–146.

14. Shapin and Schaffer, Leviathan and The Air-pump.

15. Collins, Changing Order.

16. Makel and Plucker, “Facts are More Important than Novelty,” 304–316.

17. Schmidt, “Shall We Really Do It Again?” 90–100; and Romero, “Philosophy of Science and The Replicability Crisis,” 1–14.

18. Popper, The Logic of Scientific Discovery, 420.

19. Schmidt, “Shall We Really Do It Again?” 93.

20. Barad, Meeting the Universe Halfway; and Haraway, “Situated Knowledges.”

21. Guttinger, “The Limits of Replicability,” 1–17.

22. Popper, Conjectures and Refutations.

23. Barad, Meeting the Universe Halfway.

24. Schmidt, “Shall We Really Do It Again?” 92.

25. Guttinger, “The Limits of Replicability,” 1–17.

26. Schmidt, “Shall We Really Do It Again?” 90–100.

27. Eglash et al., “Culturally Situated Design Tools,” 347–362; Babbitt et al., “From Ethnomathematics to Ethnocomputing,” 205–219; Bennett, “Ethnocomputational Creativity,” 587–612; and Lachney et al., “Ethnocomputing and Computational Thinking,” 112–135.

28. e.g., Eglash, African Fractals.

29. Dean, Doing Reflexivity.

30. Ritchie, Doing Oral History.

31. Brinkmann and Kvale, InterViews, 279.

32. Tedder, “Biographical Research,” 289; and Haraway, “Situated Knowledges.”

33. Ritchie, Doing Oral History, 1.

34. Brinkmann and Kvale, InterViews, 279.

35. e.g., Haraway, “Situated knowledges,” 575–599; and Harding, Science from Below.

36. Pickering, Mangle of Practice.

37. Noble, Digital Diploma Mills.

38. Ibid., 21.

39. Lopez III et al., “Racial and Ethnic Health Disparities,” 719–720.

40. Hillis et al., “COVID-19–Associated Orphanhood,” 31–43.

41. Trawalter and Hoffman, “Got Pain?” 146–157.

42. Marya and Patel, Inflamed, 233

43. The unique contact zone between Ghanaian adinkra symbols and the polar coordinates Anishinaabeg algorithm emerged when Ghanaian college students were exploring the quilting culturally situated design tool that is based on the works of Anishinaabe artist Minaachimo-Kwe. They combined adinkra symbols with a polar coordinates algorithm that is partially inspired by the medicine wheel design. You can find out more about this at https://csdt.org/

44. Tuck and Yang, “R-Words,” 223–248.

Additional information

Funding

This research was funded by the Joe L. Byers and Lucy Bates-Byers Endowment for Technology and Curriculum at Michigan State University’s College of Education.