Reflections on qualitative meta-synthesis studies: ‘getting the search strategy right’

In response to our meta-synthesis on physical therapists’ clinical reasoning competence, recently published in Physical Therapy Reviews, Professor Baxter, Editor-in-Chief, posited the probing question of the selection criteria for databases used in our review. Given this is a germane and thought-provoking issue for qualitative researchers, a guest editorial was timely.

Qualitative research generates knowledge that provides contextual understanding of the factors that inform evidence-based practice in physical therapy. If clinicians, researchers, and policy-makers are to exploit qualitative research to inform practice and clinical decision-making, its findings must be accessible and aggregated in a reproducible, meaningful, and robust manner. Meta-synthesis is a method of aggregating and synthesizing the findings of qualitative studies collectively to construct rich descriptive and contextual meanings about phenomena through interpretative processes.¹ In the health care context, meta-synthesis provides a broad understanding of existing knowledge related to patients’ as well as professionals’ behaviors, experiences, and needs.² In recent years, Richardson and Lindquist³ argued for increased attention to meta-syntheses in physical therapy research. The number of published meta-syntheses has slowly increased over the years. Given this development, there is a need to increase its level of methodological sophistication to augment its validity, reproducibility, and transferability.

In conducting our meta-synthesis study,⁴ we raised concerns about the trustworthiness of the method based on variations in the procedures reported in the literature.^1,5,6 A primary issue we encountered was the ambiguity of the criteria for searching and identifying eligible source studies.

First is the issue related to the balance between an exhaustive search and a manageable and efficient search process. Recall and precision are commonly used measures to describe the extent and boundaries of the search process and have been defined by Sandelowski and Barroso as follows; ‘Recall is the percent of relevant documents in the database that have been retrieved. Precision is the percent of documents in the database that have been retrieved that are relevant’ (p. 35).¹ Recall is often recommended to be emphasized over precision to ensure an exhaustive search.^1,6 However, identifying the scope of a meta-synthesis is central to identifying relevant studies. Data from each source qualitative study need to be systematically extracted, synthesized, and interpreted, a similar process to systematic reviewing quantitative studies. As qualitative articles often include comprehensive and detailed result sections there is a wealth of information for the researcher to handle in the synthesis process. Thus, there is a risk that a broader stance, i.e. a more inclusive approach in the search process, may jeopardize efficient management of the process. The question arises ‘at what point does the number of studies impede a deep analysis and threaten the interpretive validity of the findings’? As there are no established guidelines, each researcher must make a judgement call. Sound decisions in the search process are facilitated by input from investigators with various knowledge and experience. Establishing the ultimate inclusion and exclusion criteria, based on the research question, requires expert insight. In addition, sound skills in database searching are necessary and the involvement of a reference librarian can augment a rigorous meta-synthesis. Such considerations may enable a tighter focus at the outset of the search and the generation of a discrete number of relevant studies, resulting in a sufficiently comprehensive and representative search.

Second is the issue of sample bias versus the risk of including non peer-reviewed studies. Because many qualitative articles are extensive, they are not always published in peer-reviewed scientific journals. Instead, they may appear in books, monographs, and reports. Including only electronic bibliographic databases, i.e. peer-reviewed articles only, ensures quality at the risk of sampling bias. On the other hand, the inclusion of sources not subjected to peer review, such as books, monographs, and reports, could threaten the methodological rigour that is necessary in systematic reviews. According to the methodological literature, the impact of non-published literature and studies published in the grey literature, such as Google Scholar® is recommended for meta-synthesis of the qualitative literature.^1,6 However, can high-quality reviews be assured? Some source studies may be arbitrarily assessed to meet a quality standard by their investigators but not by the research community. Accordingly, the risk of including studies with uncertain quality appraisal is weighed against the risk of missing relevant studies. Thus, an acceptable minimum quality for study inclusion is yet to be determined in the meta-synthesis literature.

Last, the focus on empirical research and its consequences in the database search was a concern in our study. Importantly, meta-syntheses need to include empirical research that is based on primary data to capture the participants’ views and improve confirmability.^1,5 This epistemological stance impacts the selection of studies and databases. Theoretical articles and literature reviews are distinct from qualitative studies as they do not capture the experiences of a given phenomenon from an inside perspective. Consequently, databases of reviews such as the Cochrane database of systematic reviews are typically excluded, which may limit the outcome of the search. Nonetheless, use of the Cochrane database may well be an optimal choice in the early stages of the search process.

Overall, synthesis of qualitative studies has emerged as a valuable research method in physical therapy. The method warrants tightening to ensure its findings are valid and trustworthy. Specifically, we need to pay attention to ambiguities in the search process and strive for consensus by those in the qualitative research area. To ensure trustworthiness of meta-synthesis in physical therapy, researchers need to be aware of the pros and cons of each step in the search process and make decisions accordingly. Furthermore, they need to provide a detailed and transparent description of the search process when reporting the findings. Such steps could further increase the interest and value of meta-synthesis studies and their reproducibility as a source of evidence for improving physical therapy practice, research, and education.