Dear Sir
We thank Drs. Kulasegaram and Paradis for their considered letter, and the addition to the emerging discourse around Big Data in health professions education. We would like to respond to a few of the specific points they make.
We agree that ‘datasets of convenience’ should be considered in terms of their objectivity, provenance, and semantic baggage. We had hoped, in preparing the original paper, to provoke a debate on the extent to which data of uncertain provenance or applicability may be used to make decisions that have serious consequences for students, faculty and others in medical education. The expectation that data collected in one context and for one purpose can subsequently be used in and for others should always be tested, both theoretically and empirically. As Big Data begins to be used in health professional education we need to ensure that it is done in a critical and scholarly way. It is not just that the data potentially lacks objectivity and theoretical grounding (a problem for research as a whole); it is also that the practices of Big Data may be found wanting, particularly if they develop in isolation.
We would re-emphasize that, as we stated in the original paper, “traditional and Big Data methods should not be considered as solitudes but rather as different approaches that can be productively combined”. We urge scholars to explore how Big Data techniques can be meaningfully added to the academic repertoire so that analysts and researchers can use them along with other tools and methods to suit their needs and resources.
Health professional education research is a wide field with many intersecting research paradigms. While some research questions undoubtedly depend on better data rather than bigger data, others may need the warts and all messiness of “datasets of convenience” to explore and understand the systems that generate them. The indicators of quality for Big Data scholarship therefore need to relate to the purpose of inquiry as well as the resources it uses.
It would have been hard to select better examples of a Big Data approach than those suggested. For us the key point they make is that their questions were answered by using Big Data in scientific and scholarly ways rather than in ways that were distinct from academic practice. We hope that this trend extends to health professional education.