Developing mixed methods research in sport and exercise psychology: potential contributions of a critical realist perspective

ABSTRACT

Notwithstanding diverse opinions and debates about mixing methods, mixed methods research (MMR) is increasingly being used in sport and exercise psychology. In this paper, we describe MMR trends within leading sport and exercise psychology journals and explore critical realism as a possible underpinning framework for conducting MMR. Our meta-study of recent empirical mixed methods studies published in 2017–2019 indicates that eight (36%) of the 22 MMR studies explicitly stated a paradigmatic position (five drew on pragmatism, two switched paradigms between qualitative and quantitative elements of the study, and one was situated in relativist-interpretivism). The remaining 14 (64%) studies did not report their underpinning research philosophical assumptions. Evaluating the merits and limitations of these positions against critical realist assumptions suggests that several paradigmatic disagreements are potentially reconcilable. These include (a) maintaining that ontological and epistemological concerns are important for methodological integrity of a mixed methods study; (b) switching between paradigms in the same study is problematic; and (c) refuting the qualitative-quantitative incommensurability thesis, therefore allowing mixed methods research without compromising philosophical coherence. From a critical realist position, we suggest that both quantitative and qualitative designs are justifiable in a mixed methods study because (1) they help corroborate, refine, or refute plausible explanations of phenomena (epistemological), but (2) with different methodologies utilised to perform different tasks in the same research design related to different psycho-social system features (ontological). We call for a collaborative engagement by researchers across paradigmatic positions to work towards the advancement of methodological pluralism in our research community.

KEYWORDS:

• Mixed methods
• methodological integrity
• methodological pluralism
• research philosophy
• meta-theory
• critical realism

The interest in using mixed methods in psychological and health research accelerated when the National Institutes of Health published “Best Practices for Mixed Methods Research in the Health Sciences” (Creswell et al., 2011) as evidenced by published articles across disciplines and established dedicated journals. The term “mixed methods” may invoke different meanings (Sparkes, 2015); we take it as laid out by the editors of Journal of Mixed Methods Research to be one of the multimethod approaches in which both qualitative and quantitative methods are used in a single study or a programme of inquiry (Fetters & Molina-Azorin, 2017). Our understanding of the term is also aligned with the working group of the American Psychological Association (APA) Publications and Communications Board (Levitt et al., 2018) which characterised mixed methods research (MMR) as (a) gathering and analysing qualitative and quantitative data in response to overarching research aims; (b) using rigorous methods for both qualitative and quantitative research; (c) mixing two forms of data with an intention to generate new insights through data integration; (d) framing the distinct methodology through mixed methods research design or procedures; and (e) using philosophical assumptions or theoretical models to inform the designs.

Although MMR initiated approximately 30 years ago, there is no consensus around issues of mixed methods integrity in the sport psychology community. In 2011, a special issue of Qualitative Research in Sport, Exercise and Health (QRSEH) (Smith & Brown, 2011) provided quantitative and qualitative researchers – mostly from sport and exercise psychology (SEP) – with the opportunity for dialogue on methodological issues, with several contributions focusing their discussion on mixed methods (McGannon & Schweinbenz, 2011; Moran et al., 2011; Scanlan, 2011). The debates in sport psychology about the challenges of mixing methods, methodologies and paradigms in research were made from pragmatist (e.g., Giacobbi et al., 2005; Moran et al., 2011) and relativist-interpretivist¹ (e.g., Culver et al., 2012; Sparkes, 2015) meta-theoretical perspectives. Following methodological work in other fields (e.g., Creswell & Plano Clark, 2011; Johnson & Onwuegbuzie, 2004; Tashakkori & Teddlie, 1998), Collins et al. (2018), Giacobbi et al. (2005), and Moran et al. (2011) advocated pragmatism as a way to unify qualitative and quantitative approaches in a mixed methods study. For example, Giacobbi et al. (2005) drew on Pierce, James, and Rorty to claim pragmatism as a philosophy of knowledge construction that emphasises practical consequences of research inquiry. Although pragmatist such as Peirce (1931–1985) would agree that there is ontological monism (just one reality, not multiple realities) and suggest that truth emerges “at the end of enquiry”, pragmatism, and notably through its operationalisation in contemporary human sciences, has generally come to be associated less with concerns of ontology, epistemology, truth and falsity, and more with practical concerns such as the usefulness of research (Giacobbi et al., 2005). To privilege the practical and the empirical over the ontological and epistemological is consistent with the pragmatic stance (Tashakkori & Teddlie, 1998).

The pragmatist solution to MMR, articulated within a question-driven philosophy of “what works”, was contested by many qualitative researchers who believe that paradigms and metaphysics do matter because ontological and epistemological assumptions will inevitably lead researchers “to generate different questions, develop different research designs, use different techniques to collect various kinds of data, perform different types of analyses, represent their findings in different ways, and judge the ‘quality’ of their studies using different criteria” (Smith & Sparkes, 2016, p. 3). For example, Culver et al. (2012) argued that “mixing of quantitative and qualitative methods grounded in different epistemological foundations makes little sense at all given paradigmatic incommensurability” (p. 275), suggesting that quantitative and qualitative inquiries constitute antagonistic epistemological paradigms. Others have suggested the question about MMR is not whether researchers can use different methods (they can), but rather about the problems of maintaining philosophical consistency and coherence when each method implies particular and potentially incommensurable meta-theoretical assumptions (McGannon & Schweinbenz, 2011; Sparkes, 2015; Whaley & Krane, 2011). Sparkes (2015) also highlighted the controversy surrounding quality criteria of MMR and whether quantitative and qualitative parts of the study should be evaluated separately or whether there should be specific criteria to evaluate the study as a whole given the integrative ambitions of MMR. The arguments put forward by the proponents and opponents of MMR are certainly complex and richly nuanced, reflecting controversies around conceptual, practical and pedagogical issues (see Sparkes, 2015); yet the incommensurability of paradigms thesis – that is, philosophical incompatibility of the quantitative and the qualitative components in a mixed methods study – appears to undergird the debate about MMR in SEP.

It is important to recognise that qualitative research is situated on a continuum of diverse philosophical assumptions and may not necessarily be conducted from non-realist ontological and subjectivist epistemological positions. Indeed, a recent review of 30 years of qualitative research reported that most commonly used implicit assumptions in SEP research stem from realism (Poucher et al., 2020). Some see ontological realism inevitably underpinning scientific epistemologies of positivism and postpositivism with the latter being a softer version of the former (e.g., Smith & Sparkes, 2016). In fact, critical realism is often described as a “post-positivist” position; with some critical realists being comfortable with the term because the prefix “post” helps emphasise its distinction from positivism (see Archer et al., 2016), and others less comfortable (e.g., Wiltshire, 2018) because relativist-interpretivist qualitative researchers (see Smith, 2017; Smith & McGannon, 2018; Sparkes, 1998, 2015) tend to use it in a way that emphasises its similarities with positivism. If one takes the view that post-positivists “tend to use quantitative methods” (Sparkes, 2015, p. 51) or that qualitative research is seen “in opposition to post-positivism” (Gibson, 2016, p. 385) then we would oppose the comparison with critical realism. However, if one takes the view of Smith and McGannon (2018) that post-positivism is informed by “a combination of epistemological constructionism and ontological realism” then we would accept the comparison, albeit with further distinguishing characteristics such as the emphasis on methodological pluralism, theorising, and seeking causal explanations. Although seemingly mere terminological details, these confusions may explain why some methodological debates have tended to confuse or conflate critical realism with aspects of positivism (Brinkmann, 2017). In this paper, we use a hyphenated spelling “post-positivism” to distinguish philosophies of science such as critical realism from the postpositivist paradigm (which is closely connected to positivism).

Critical realism has become influential in other social sciences fields by offering philosophically convincing arguments to resolve issues around the connection between the objective and subjective and to underpin MMR (Bhaskar, 1989; Iosifides, 2017; North, 2017; Sayer, 1992). It has also informed recently published qualitative and mixed methods psychological studies in sport and exercise (e.g., Brown et al., 2018; de Grace et al., 2017; Sorkkila et al., 2020; Swann et al., 2016). We wish to articulate further and build on the case for a critical realist underpinning for MMR in this paper. With the intention to understand present trends, we conducted a meta-study of mixed methods articles published in six sport and exercise psychology journals over the last three years, 2017–2019. Our related objective was to explore whether critical realism could offer a fruitful perspective for developing a mixed methods methodology in SEP. The paper’s overarching purpose is to contribute to the methodological advancement of MMR by giving this mode of inquiry “greater attention and further debate” as urged by Sparkes (2015, p. 58), with a particular focus on evaluating the potential of critical realism as a stance for MMR.

The current mixed methods landscape

In this section, we present the current MMR trends within six sport and exercise psychology journals by analysing how researchers conceptualised and evaluated methodological aspects of their mixed methods studies. Exploring the ways in which various researchers practice mixed methods inevitably involves a critical gaze on their work. We have no interest in caricaturing or misrepresenting their beliefs and paradigmatic stances. Our position is to do appropriate justice to these positions whilst making the case for critical realism as a valuable alternative.

Methodology

In a review of qualitative research in sport psychology journals in 2000–2009, Culver et al. (2012) found that out of 57 studies that used a mixed methods design, only one article addressed meta-theoretical issues. To explore the current landscape, we conducted a meta-study of mixed methods research in leading sport and exercise psychology journals between 2017 and 2019. A meta-study is a method that was introduced as a means to critically interpret existing (qualitative) research with the focus on how theorising, methodologies and societal contexts have shaped knowledge production (Paterson et al., 2001). While meta-studies are typically conducted to synthesise findings within a specific research topic, they are also particularly useful for scrutinising broader methodological developments (e.g., Clarke et al., 2015). We focus on the full range of meta-theoretical issues: ontological, epistemological and methodological considerations to demonstrate how the use of methods has been justified. Specifically, the aims of the meta-study were to (1) discern how the researchers have described the paradigmatic position of the research and (2) examine how concerns about research quality have been addressed in the research methodology.

After setting up the research team and research objectives, we searched the electronic archives of six leading journals – International Journal of Sport and Exercise Psychology (IJSEP), Journal of Applied Sport Psychology (JASP), Journal of Sport and Exercise Psychology (JSEP), Psychology of Sport & Exercise (PSE), Sport, Exercise and Performance Psychology (SEPP), and The Sport Psychologist (TSP) – for empirical mixed methods articles. The searches yielded 22 articles in total, with 10 articles published in PSE, four articles published in JASP, four articles published in TSP, two articles published in IJSEP, and two articles published in SEPP. The characteristics of the studies are described in Table 1, which also presents an analysis of the identified positions on MMR that we problematise from the critical realist perspective later in the article.

Table 1. Mixed methods studies in SEP journals in 2017–2019, by paradigmatic position.

Reference	Position on MMR	Journal	Theme	Types of data	Quality considerations (qualitative + study as a whole)
Kegelaers et al. (2019)	Pragmatism	JASP	Pressure training to develop resilience	Questionnaires and interviews	Building rapport, reflective journal, critical friends
Rumbold et al. (2018)	Pragmatism	PSE	Organisational stress management	Surveys, semi-structured interviews, focus groups	Design quality and interpretive rigour: self-reflexive diary, critical friends, consistency with theories and frameworks, discussions with participants to assess agreement
Thrower et al. (2019)	Pragmatism	JASP	Effectiveness of a web-based education programme for tennis parents	Questionnaires, asynchronous email interviews	Study as a whole: Transparency, relevance, rationale for MM and data integration. Quality criteria of each aspect assessed separately
Jackman et al. (2017)	Pragmatism	PSE	Effectiveness of existing methods to collect data on flow	Questionnaires, event-focused interviews	Prolonged engagement, peer debriefing, member checking/reflections
Karageorghis et al. (2018)	Pragmatism	SEPP	Use of music in psychological preparation	Questionnaires, reflective journals, observations and interviews	Continuous comparison of data to ensure methodological coherence, new iterations of collection, developing a substantive theory, theoretical sampling
Thelwell et al. (2018)	Switching paradigms between quant & qual	PSE	Youth coaches' and parents' perceptions of sport psychology consultants	Semi-structured interviews, surveys	Critical friends
Whitehead et al. (2018)	Switching paradigms between quant & qual	PSE	Cognitions, pacing strategies, and performance in cycling time trials	Think aloud protocol (studies 1-2), interviews (study 3)	Study 1 and 2: inter-rater reliability Study 3: critical friends
Papathomas et al. (2018)	Relativist-interpretivism	SEPP	Athletic retirement and changes in body image	A questionnaire with quantitative and qualitative (open-ended) questions	Ontological and epistemological coherence, critical friends
Choi et al. (2019)	Not stated	IJSEP	Athlete activism	Questionnaires and interviews	Appropriate sample, participant reflections, critical friends, audit trail, ontological, epistemological, methodological coherence
Kristiansen et al. (2019)	Not stated	PSE	Football players’ perceptions of organizational and media stressors	Questionnaires and interviews	Member reflections
Abrantes et al. (2019)	Not stated	PSE	Development of a smart phone app	Questionnaires and focus groups; individual usability sessions	Not addressed
Mallinson-Howard et al. (2018)	Not stated	PSE	The relationship between types of perfectionism and sport experiences	Questionnaires, focus groups, semi-structured interviews	Tracy's (2010) guidelines adopted
Pulling et al. (2018)	Not stated	PSE	Visual exploratory activity	Survey with open-ended questions	Collaborative coding, trustworthiness by consensus.
Pummell and Lavallee (2018)	Not stated	PSE	Preparing athletes to junior-senior transition	Questionnaires and interviews	Not addressed
Souza and Ebbeck (2018)	Not stated	JASP	Attitudes towards larger fitness centre members	Questionnaires, participant brainstorming	Randomising the order of ideas that participants rate to reduce response bias; recruiting a diverse sample to enhance generalisability
Thibodeaux and Winsler (2018)	Not stated	PSE	The content of self-talk in youth tennis players	Questionnaires and observations	Inter-rater reliability (observations)
Voelker et al. (2018)	Not stated	JASP	Correlates and frequency of eating disorder symptomatology in figure skaters	Questionnaires involving open-ended questions	Two independent coders of open-ended items
Deen et al. (2017)	Not stated	TSP	The effect of rational emotive behaviour therapy on irrational beliefs and resilience	Questionnaires and interviews	Using an interviewer other than the practitioner to avoid bias. Researcher triangulation in data analysis
Kacperski and Hall (2017)	Not stated	JASP	Construal levels, imagery and performance outcomes	Questionnaires, performance outcome assessment, and audio-taped questions (manipulation check)	Inter-rater reliability
McCormick et al. (2018)	Not stated	TSP	The effects of a motivational self-talk intervention	Surveys, intake interviews, and questionnaires	Social validity, external validity, follow-up survey as the added value
Middleton et al. (2017)	Not stated	TSP	The effects of music on pre-performance psychobiosocial states	Questionnaires and interviews	Inter-rater reliability
Redwood-Brown et al. (2018)	Not stated	IJSEP	Psychological momentum in elite soccer players	Questionnaires, interviews, and focus groups	Member checking, inter-rater reliability
Richard et al. (2017)	Not stated	TSP	The effect of an improvisation intervention performance, self-esteem, creativity, and mindfulness skills	Questionnaires and interviews	An inter-coder consistency check

Results

Paradigmatic positioning of MMR

From the 22 identified studies, eight (36%) explicitly addressed the philosophical assumptions of the research. More specifically, five studies drew on pragmatism as an overall stance, two studies switched paradigms when moving between quantitative and qualitative phases of the study, and one study drew on the relativist-interpretivist paradigm. The remaining studies did not specify their philosophical positions. This shows similarities to other review work exploring the meta-theoretical underpinnings of sport science research (North, 2013a).

From the studies drawing on pragmatism, Thrower et al. (2019) engaged in the most thorough methodological discussion, justifying the mixed methods strategy on the basis of complementarity, the capacity to generate new insight, and utility for practitioners. Rumbold et al. (2018) also addressed the methodological justifications for MMR and described their work as “a pragmatist perspective with a critical realist ontology” (p. 28), whereas Jackman et al. (2017) noted that MMR is a contested terrain and explained that “a pragmatic stance to the use of mixed methods (…) was adopted” (p. 115). Karageorghis et al. (2018) justified the pragmatist position of their research on the basis of compatability with grounded theory (Corbin & Strauss, 2015) in which methodologies are selected by the applied impact on the groups that are studied.

Two studies described their approach as switching the paradigmatic position when they moved between the quantitative and qualitative phases of the study. Thelwell et al. (2018) stated that they adopted “the post-positivist approach to the data collection” (p. 136) in the qualitative phase, “but progressed via the adoption of a pragmatic stance to address the research question at a different level that in this case, required a shift from epistemological purity” (p. 139). Whitehead et al. (2018) adopted an expansion approach, where a series of three studies were conducted to gain different insights into the same topic. They described assuming a post-positivist stance in the first two studies and switching to a relativist-interpretivist perspective in the third study.

One study adopted a relativist-interpretivist paradigmatic position. Papathomas et al. (2018) employed both quantitative and qualitative methods in the study of retired athletes’ body image perceptions; however, they described their stance as a multimethod approach which they claimed “is different from a mixed-methods approach, as it is merely the data-collection techniques that are mixed and not the underpinning paradigm” (p. 33). Because the multimethod approach in which both qualitative and quantitative methods are used in a single study falls under the definition of mixed methods (Fetters & Molina-Azorin, 2017), we included this study in the review.

Considerations of research quality

In terms of evaluating research rigour, some researchers addressed integrative quality criteria that they framed as applicable to the study as a whole (e.g., Karageorghis et al., 2018; Mallinson-Howard et al., 2018; Rumbold et al., 2018; Thrower et al., 2019), but more often quantitative and qualitative parts of the study were evaluated separately (e.g., Kacperski & Hall, 2017; Middleton et al., 2017; Redwood-Brown et al., 2018; Richard et al., 2017; Whitehead et al., 2018). From the integrative perspective, Thrower et al. (2019) suggested that “taken as a whole, this study can be judged on the capacity to which it is relevant for the research questions, is transparent, has a rationale for using mixed methods, and requires the integration of mixed-method findings”, while at the same time maintaining that “the quality criteria for each aspect of the study should be considered separately” (p. 4). Mallinson-Howard et al. (2018) drew on Tracy’s (2010) eight “universal” criteria (which were originally introduced as guidelines for qualitative researchers) to make a judgement about the quality of their MMR study of experiences associated with different types of perfectionism. Although the examples that the authors provided referred mainly to the qualitative part of their project (e.g., “the research is marked by thick description and the showing rather than telling of the participants’ experiences through inclusion of focus group exchanges between participants and individual participant quotes” (p. 166)), they maintained that the criteria (e.g., worthy topic, sincerity, resonance and ethics) were considered within the project as a whole. In the study by Karageorghis et al. (2018), Corbin and Strauss’s variant of grounded theory was employed as the framework for the entire research process in order to sustain methodological coherence. Papathomas et al. (2018) also highlighted the consistency of the relativist-interpretivist position throughout the study, which links to the coherence principle as an indication of rigour (Smith & McGannon, 2018).

In terms of validity “techniques” specific to qualitative data, the most common approach was using critical friends (six studies) which is likely to reflect Smith and McGannon’s (2018) recommendations. Other commonly mentioned ways to address research quality in the qualitative phase were inter-rater reliability (five studies) and member checking or member reflections (three studies). Tracy’s (2010) criteria were mentioned in one study; other markers of quality included prolonged engagement, collaborative coding, audit trail and self-reflective diary. Whitehead et al. (2018) used inter-rater reliability in their post-positivist phase, and critical friends in their relativist-interpretivist phase, echoing Smith and McGannon’s (2018) and Sparkes’s (2015) suggestion that validity considerations should be specific to the adopted paradigm.

Seven out of eight studies that explicated the paradigmatic position of the overall research (or both quantitative and qualitative components) were published in 2018–2019, suggesting that mixed methods scholars are at least starting to engage with research philosophy, albeit in a way that tends to provide meta-theoretical veneer, rather than detailed consideration. Fourteen (64%) of 22 articles did not report the study’s philosophical assumptions. Moreover, the majority of studies did not evaluate or reflect on whether the mixed methods design was appropriate as a methodological choice given the study’s aims, how various forms of inquiry combined in the study were justified as the added value, and whether the MMR integrative goal was achieved. It would appear that both pragmatist and relativist-interpretivist perspectives, which dominated the debates, have impeded the exploration and the development of MMR as an emerging field of inquiry, albeit for different reasons. While pragmatists (with some notable exceptions) dismissed the meta-theoretical underpinnings by privileging practical and empirical aspects of the research process, the qualitative methodologists aligned interpretive qualitative research with non-realist ontology. By doing so, they affirmed the incommensurability thesis – that is, that quantitative and qualitative parts in the mixed methods study are underpinned by conflicting philosophical assumptions (unless both the qualitative and quantitative components would be underpinned by postpositivism). Perhaps it is not surprising, then, that the majority of the authors in our review avoided discussing the meta-theory underlying their research processes whilst others either opted for pragmatism, and then with questionable depth of understanding or application, treated the qualitative and the quantitative components of their mixed methods studies separately. It is against this methodological backdrop we turn to exploring critical realism as a stance for how to move towards harnessing a coherent engagement with MMR.

Opening up dialogue from a critical realist perspective

To make an alternative contribution to the MMR debate in sport psychology, we next explore how critical realism can help reconceptualise the combination of qualitative and quantitative methods. Prior to that, some clarifications about the central propositions of critical realism are necessary for readers who may be unfamiliar with it.

What is critical realism?

Following Archer et al.’s (2016) lead, we recognise that

there is not one unitary framework, set of beliefs, methodology, or dogma that unites critical realists as a whole. Instead, critical realism is much more like a series of family resemblances in which there are various commonalities that exist between the members.

Indeed, while much of our presentation of critical realism is grounded in the work of authors who appear to be self-described “critical realists” (including Archer, 1998; Bhaskar, 1989; Collier, 1994; Danermark et al., 2005; Maxwell, 2012), we also draw on ideas from scholars who appear to be self-described “realists” (including Emmel et al., 2018; Pawson, 2013; Sayer, 1992; Wong et al., 2012). Because of this inherent heterogeneity it is important to highlight that we do not claim to represent the position of all critical realists here, although we choose to focus on the many shared concepts and underpinning claims.

A starting point for understanding critical realism is the realist distinction between the natural (e.g., global warming), the social (e.g., social stratification) and, in our specific case, the psychological (e.g., learning disabilities) objects and structures, and the activities of science and other knowledge generating processes. Objects and structures are “real” and they are “out there”, they do not exist only in the “constructions” or “interpretations” of researchers. In other words, critical realists claim that, “there is a state of the matter which is what it is, regardless of how we do view it, choose to view it or are somehow manipulated into viewing it” (Archer, 2007, p. 195). Objects and structures are complex, open, and dynamic and so may be described as (multi) “layered” in that they are constituted by different types of things (e.g., physical, biological, psychological), but not “multiple” in the sense that there are multiple and equally valid truths. From this starting point, sport psychologists can, for example, assume that there are people (athletes, coaches, parents etc.) who have actual properties, dispositions, and experiences before they become “participants” of a particular study and subject to researchers’ understanding of their experiences.

This realist claim is always qualified by noting the epistemological point that objects and structures are not easily knowable to science (or other ways of knowing). Bhaskar (1975) was clear to point out that all knowledge of the objective world is fallible, partial, and subject to socio-historical forces. Sayer (1992) states explicitly that “science or the production of any other kind of knowledge is a social practice” (p. 5). As such, while critical realism maintains that the world continues to pre-exist our knowledge of it (ontological realism), critical realism wholly agrees that the concepts, language, methods, and politics of science play a role in producing our knowledge of the world (epistemological constructivism). By maintaining the distinction between “being” (ontological) and “knowing” (epistemological) in this way, critical realism “precludes any collapse of the ontological into the epistemological and convicts those who endorse this move of the epistemic fallacy, namely confusing what is with what we take it to be” (Archer, 2007, p. 195). This position runs counter to much qualitative research where there is evidence of ontology and epistemology being collapsed in how researchers describe their assumptions (see North, 2017; Wiltshire, 2018).

Another important ontological assertion of critical realism is that the world is “deep” or “stratified”, consisting of different “strata” or “domains”. This conceptual move foregrounds the critical realist account of causality which, in practical terms, “means moving beyond describing what can be measured in the social world to explain the deeper causal powers that shape what can be observed” (Emmel et al., 2018, p. 5). While there are objects of research that are in-principle observable through our methods (such as actual “events” like manifest behaviours or neurological responses), ontological depth refers to aspects of reality that are in-principle unobservable with the potential to cause observable events. As North (2013b) explained, these can be “the material, psychological and social objects and structures, with associated causal powers and liabilities, which underlie and govern events” (p. 134). Because these causes may be unobservable, critical realist researchers aim to centralise the theorisation of their existence and their nature. This is just as relevant to the existence of dark matter for physicists (which has not yet been observed), the existence of Shakespeare for historians (who no presently-living person observed writing Hamlet), and the causes of athlete burnout for sports psychologists (which can only be inferred through partial observations with athletes, their coaches and relevant others).

Critical realism also represents a departure from the empiricism emphasised in positivist traditions, which is usually (but not necessarily) associated with quantitative research (North, 2017). As noted above, those criticising critical realism often confuse it with a naïve objectivist exploration using a foundationalist ontology and empiricist epistemology (i.e., experience can be observed/measured) so it may be important to outline some key points of departure. In the context of this discussion, we see critical realism as being distinguishable from positivism and relativist-interpretivism because of its materialist underpinnings, its emphasis on causal powers and dispositions, the concept of emergence that explains why we have more complex psycho-social structures, and the theorising of unobservable mechanisms. According to Anjum and Mumford (2018), the positivist methods adopted in medical research (and often followed in evidence-based sport sciences) are largely underpinned by David Hume’s version of causality which takes the view that effects can be attributed to their causes through the regularity of discrete events (“constant conjunctions”) between different quantifiable variables. In positivistic research causality tends to be reduced to statistical associations between reported or observable events or structures, providing a “thin view” of causality. Given the stratified ontology that we advocate from a critical realist perspective, causal powers are described as being in a different level, domain or realm to actual events because their potential powers may exist, but may not be exercised (Archer, 1998). Hence, causal explanation ought to be a matter of producing theories about mechanisms that explain both the presence and the absence of uniformity (Pawson, 2006). From this view, the adequacy of a causal account is less about assessment of “strength of association” and “consistency” (although they are helpful), but more about the ability of the account to describe and explain the complex pathway whereby a certain behaviour – e.g., physical activity – leads to a certain outcome – e.g., positive mental health – for some people and why the same behaviour does not lead to the same outcomes for others. Critical realism is also explicitly causal which provides a basis for the explanatory component (explanation evokes the notion of cause), seen but neglected theoretically in interpretive positions.

Following from this, another conceptual contribution has been the idea that causal powers are “emergent” and “contingent” in that “they will only exert their generative influence in an arrangement with other parts” (Emmel et al., 2018, p. 6). That is, it may not be physical activity alone that elicits positive mental health outcomes, but physical activity in combination with particular subjective meanings, experienced in particular social contexts, through the material presence of particular neurophysiological conditions which might differ from person to person. Moreover, critical realists propose that causality happens in “open systems” because society cannot be construed as static and sealed. This is highly problematic for experimental designs that attempt to artificially create “closed systems” by controlling for and isolating variables and aggregating populations (McEvoy & Richards, 2006). In the research process guided by the critical realist ontology, a positivist-type question “does physical activity cause improvements in mental health?” is the wrong kind of question (see Clark & Thompson, 2010). Critical realist assumptions require us to ask questions that accommodate the inherent complexity of the relationship between one event and another as well as allow the analysis to encompass a number of levels (physical, biological, psychological, social) (Bekker & Clark, 2016). The summary of critical realist assumptions is presented in Table 2.

Table 2. Summary of critical realist assumptions.

Critical realist assumptions	Description	Example in SEP
Epistemological constructivism	Knowledge is fallible, concept-dependent and constructed; varies across space and time.	Our knowledge of depression in athletes is subject to how “depression” is defined and measured. Qualitative and quantitative methods can help refine, revise or refute what we think we know about depression in athletes.
Ontological realism	The objects of research exist independently of researchers’ conceptions of them.	What we call “depression” remains a reference to real properties and events, experienced by actual people independently of research. Qualitative research is well suited to gaining access to its different forms and quantitative research is well suited to gaining access to its prevalence and severity.
Stratified ontology	Reality is stratified. Parts of reality are in-principle observable at a given time. Other parts of reality are in-principle unobservable at a given time.	There are aspects of depression in athletes that may be unobservable, such as genetic predispositions, neurochemical factors, historical events and unconscious responses. These aspects are no less “real,” but we can only infer their existence.
Causal complexity	Casual explanations are important for advancing knowledge, but the causal properties of phenomena are contingent, complex, open and emergent.	The causes of depression in athletes are multiple and complex. Quantifiable patterns of depression (e.g., in particular socio-demographic groups) may point towards causal explanations, and qualitative research may provide insights into the possible contingencies in the explanatory account.

Furthermore, in Table 3 we summarise problematic issues and tensions in the positions on MMR from a critical realist perspective, identified in our meta-study. We also offer suggestions for potential reconciliation.

Table 3. Summary of MMR landscape and remaining issues from a critical realist (CR) lens.

Position on MMR	Logic of justification	CR critique: Objections, issues and tensions	CR perspective
Pragmatism	MMR is possible because research questions, not philosophical assumptions, guide the method.	The framing of the question entails philosophical assumptions (Clark & Thompson, 2010). Philosophical assumptions should be made transparent for readers. Leads to lack of engagement in philosophical questions.	CR agrees that pragmatism is important for both choice of method and judging the validity of claims. However, CR takes meta-theoretical concerns seriously and asks researchers to theorise about the nature of the phenomenon and then carefully select appropriate methods to match (Pawson & Tilley, 1997).
Switching paradigms	MMR is possible because qualitative and quantitative elements should be treated separately.	Switching between paradigms radically undermines the notion of “basic philosophical beliefs” and hence reduces paradigms to methods. Provides no basis for reconciling the separate elements of the study. Limits methodological integrity to generate new insights.	CR agrees that qualitative and quantitative methods should (usually) be treated differently. However, there is no need to switch paradigms because CR focuses on explanation and advocates methodological pluralism and hence can encompass both qualitative and quantitative elements, provided they share CR assumptions. Separate elements are reconciled on the basis that they provide insight into a different part of the puzzle, or provide a different form of evidence giving insight into the nature of a phenomenon.
Relativist-interpretivism	MMR is possible because researchers can remain faithful to interpretive epistemological commitments for both qualitative and quantitative elements of the study.	Lack of justification for how “objective” matters of interest (body mass, levels of inactivity, years since retirement etc.) are subject to the same interpretation as “subjective” matters of interest (body satisfaction, mood, thoughts etc.). Provides no basis for distinguishing between phenomena that can in-principle be observed and phenomena that cannot in-principle be observed. Applies “multiple truths” assumption equally to both participants’ truth (i.e., participants’ personal perception of reality) and researchers’ truth (i.e., researchers’ interpretation of participants’ reality), removing the reference point from which more or less valid research accounts can be judged (Ronkainen & Wiltshire, 2019).	CR agrees that paradigmatic assumptions are important. However, CR assumes a stratified ontology which distinguishes between the actual (the way things are, which can in-principle be observed) and the real (the underlying causes of events, which cannot in-principle be observed). CR maintains that the notion of Truth is valuable, but accepts that we can never know if it has been found. Participants’ experiences are accepted as useful evidence, constructed perceptions of reality, but this is a different matter to researchers’ interpretations of reality.

MMR through the critical realist philosophical lense

For critical realists, the purpose of mixed methods research is to describe but, in particular, to enhance explanation, interpretation, and understanding of social psychological objects and structures. Consequently, no methods or ways of producing data ought to be rejected a priori and their combination is needed for successful investigation of complex problems (Gill, 2011; Iosifides, 2017; Maxwell, 2012; Moran et al., 2011; North, 2017). MMR is beneficial so long as it adds value to our emerging explanations of phenomena; for example, how sociocultural structures impact as well as constitute individual subjectivity. As Emmel et al. (2018) noted, “realists are rather less interested in methods and very much more interested in how insights, which sometimes are from investigations, add to a pool of theory” (p. 4). Similarly, Danermark et al. (2005) explained, “we consider the search for generative mechanisms as the main undertaking in research work, in which both [quantitative and qualitative] methods need to be applied” (p. 154). Reconceptualising qualitative and quantitative methods as intensive and extensive empirical procedures set in a critical realist metaphysical context (Sayer, 1984), the decisive issue for MMR coherence is “how different methodologies can convey knowledge about generative mechanisms” (Danermark et al., 2005, p. 163). So while critical realists and pragmatists agree that qualitative and quantitative methods are useful in psychological and social research, critical realists have generally paid more attention to the reasons why, and in what ways different methodological approaches (quantitative and qualitative) support the explanatory endeavour.

As noted, critical realists foreground ontology, including specifying the nature of material and psycho-social objects and structures and their emergence. Thus, the epistemological/methodological rationale for both quantitative and qualitative research is rooted primarily in ontological concerns. Physical, psychological, and social systems have different levels of simplicity or complexity. They also have different levels of stability and instability. In a sporting context, bio-mechanical and nutritional systems may be seen to be simpler (although never simple) and more stable over time than psychological or social systems. Particular psychological, or psycho-social, characteristics or dispositions may be simpler and more stable than others. For example, the factors underpinning motivation, may be seen as simpler or more stable than emotion – although it recognised that specialist researchers will take different views on this.

Quantitative research from a critical realist perspective is more appropriate to measuring characteristics of a system that are simpler and more stable. It is no accident that quantitative research has had its greatest successes in the natural sciences (and in the natural scientific gradations of human science) because the systems they explore are more easily isolated and the stabilities between variables measured and subject to statistical association. Qualitative research takes on a different role. It provides a description of the qualities of a system that may allow a researcher to discern whether the system under consideration is simple and/or stable enough to be measured. Most of the time, however, qualitative research concerns conceptual development about real natural and social objects and structures such that we can gain knowledge about them, and with sufficient resource and intellectual energy start to develop an explanatory account. In time, and with sufficient qualitative/conceptual work over a range of contexts and time periods, it may be possible to understand what elements of a system are more stable (including across contexts), whether it can become measured, and how this knowledge might be generalised. However, the conceptual work has to be done first. Thus, we can see both an ontological (system simplicity and stability) and an epistemological (description and explanation) rationale for both quantitative and qualitative research, and understanding about how they could be used together in a philosophically coherent way in the same research design, whilst recognising that in any explanatory endeavour there is always a central role for theory.

Despite critical realism’s opening up the possibility of MMR, it may place new or revised framings on how familiar methods can be used – that is, they may need to be viewed in different ways and perhaps deployed at different stages of the research for different purposes. As above, we have suggested that quantitative research is appropriate when target psycho-social systems are simpler and more stable and have been appropriately conceptualised. Statistical analysis might be useful to test theories about how causal mechanisms operate under particular sets of conditions once those theories are sufficiently formed (Mingers, 2003). However, others, such as McEvoy and Richards (2006) see quantitative method as being useful in the “exploratory phase” of projects to “identify patterns and associations that may otherwise be masked” in order to “tease out new and unexpected causal mechanisms” (p. 70). Similarly, experimental designs such as Randomised Control Trials (RCTs) can potentially be used from a critical realist perspective (Bonell et al., 2012) but there remain on-going debates about what insights can be drawn from them given that RCT designs do not “enable the identification of the dynamic interplay among the intervention, actors, context, mechanisms and outcomes which is at the core of realist research” (Van Belle et al., 2016, p. 1).

Qualitative research from a critical realist approach appears more appropriate to studying complex, unstable and conceptually undeveloped psycho-social objects and structures. Sayer (1984) argued that “qualitative information is needed on the nature of the objects involved and not merely more quantitative data on empirical associations” (p. 114). Indeed, ethnographers and interviewers alike are likely to be able to reveal illuminating information about participants’ lived experiences as well as the historical, political and cultural contexts in which those experiences take place. However, qualitative methods are usually reframed by critical realists as being useful for understanding how processes work in particular cases and, thus, building a case for causal mechanisms. In a similarly flexible way to quantitative methods, qualitative methods can be used in an exploratory way or in a verification/falsification way (see Pawson’s (2006) Realist Interview technique).

What comes through strongly in methodological pluralism, including the emerging methods of “realist evaluation” and “realist synthesis” (the realist alternatives to RCTs and meta-analyses), for example, is the need to engage transparently with a diverse range of data and interpretations of that data to “corroborate, refute or refine” the most plausible theories available (Pawson & Tilley, 1997; Wong, 2018). While both quantitative and qualitative methods are viewed as important in critical realist research, what we do with that data is of much greater significance. The summary of justifications for MMR from a critical realist perspective is presented in Table 4.

Table 4. Summary of critical realist justification for MMR.

Critical Realist assumptions	Justification for MMR
Epistemological constructivism	Methodological pluralism is justifiable because data (qualitative and quantitative) must be treated as merely data, not reified as the ultimate representation of the objects of research. The imperfection of any single method is a warrant for seeking additional, supplementary methods. There is a central role for theory managing existing research/data and new data adaptively with the researcher being the key editor/decision maker, but guided by a number of ideas, process and quality assurance mechanisms.
Ontological realism	Research is an attempt to align our data, interpretations and theories with reality. Since reality contains phenomena that are both qualitative (e.g., of different forms, types and qualities) and quantitative (e.g., more or less frequent, and of greater or lesser extent), then both qualitative and quantitative methods are required.
Stratified ontology	Research ought to draw on observations to theorise and speculate about unobservable entities. Both qualitative and quantitative data are helpful in the process of refining, revising or refuting these speculative theories.
Causal complexity	Quantitative data can help identify regularities (observable patterns of events) and qualitative data can help understand for whom, in what circumstances and how these regularities have a tendency to occur.

To conclude this section, we do not advocate for the use of critical realism in MMR simply because it is compatible with and appreciates both qualitative and quantitative research. It would be problematic to claim the critical realist stance without adhering to its assumptions, commitments to the foregrounding of ontology and explanatory framework, and applications in mixed methods practice. As we have suggested, critical realism privileges ontological realism and approaches research as an attempt to align data, interpretations and theories with reality in order to derive most plausible explanations of empirical data, such as individual experience. The social and cultural contexts are crucial for the understanding of generative mechanisms because the context, to a greater or lesser extent, is a part of that causal process (Archer, 1998). Both designs, extensive (to address population patterns/regularities) and intensive (to address how processes work in particular cases), in a critical realist MMR project are important for examining the questions that are typically overlooked or dismissed by quantitative and qualitative researchers working independently (Danermark et al., 2005). The suggestion that we put forward to the sport and exercise psychology community, therefore, is that critical realism potentially offers a fruitful ground for collaboration between qualitative and quantitative researchers in producing more coherent, interdisciplinary and impactful MMR.

Conclusion and future challenges

This paper began with the premise that mixing methods can be a worthy endeavour within research projects. Although there is an increasing number of mixed methods studies, our three-year (2017–2019) meta-study of MMR published in leading SEP journals indicates that there is still fairly limited engagement with the philosophical underpinnings and implications of mixing research strategies by researchers conducting MMR. We proposed critical realism as one of the possible trajectories for developing a mixed methods methodology in the field. Our purpose was not to provide universal guidelines on how to conduct and assess MMR but to discuss, from a critical realist perspective, some of the crucial ontological, epistemological and methodological premises of a mixed methods study.

Recently there has been a tendency in sport psychology to stipulate the necessary alignment of epistemological constructivism and ontological non-realism in qualitative inquiry (Smith & McGannon, 2018), which is central to the incompatibility of paradigms thesis. We offer an alternative account based on publications advocating a (critical) realist perspectives (North, 2017; Ronkainen & Wiltshire, 2019; Wiltshire, 2018), which suggests ontological realism and epistemological relativism/constructivism. The former presents ontological realism and epistemological constructivism as contradictory. However, on our account the contradiction relies on the erroneous conflation of ontological assumptions (a social reality independent of researchers) with epistemological assumptions (researchers’ knowledge of that reality). Moreover, while there may exist distinguishable research cultures and traditions between some qualitative and quantitative researchers, Weed (2009) argued that such divisions might be sustained by “protectionist paradigmatic behaviour” rather than logical coherence. He claimed that this adversely affects the research community because “paradigmatic behaviour can reduce debates to mere contradiction of the position of the ‘other’, with the dismissal of ‘their’ position being justified on the basis that it is derived from an incommensurable paradigm” (p. 312). As warned by Atkinson et al. (1988), classifying research projects neatly into “paradigms” or “traditions” does not reflect untidy realities of real scholars and may displace researchers from gathering data on important problems and/or building theories (see also North, 2017). Critical realism seeks to address this and help scholars to navigate muddy research waters.

Acknowledging that it is impossible to derive knowledge from outside of a particular point of view or theoretical/discursive framework, we have claimed that – through the adoption of the critical realist metaphysical assumptions – quantitative and qualitative designs become ontologically and epistemologically compatible. We drew on critical realist scholarship to suggest viewing qualitative and quantitative methods as empirical approaches of producing data about different layers of social and psychological reality. The disconnection of quantitative research from positivism and qualitative research from relativist-interpretivism is necessary in order to achieve compatibility between different methodological approaches and to avoid “methodological eclecticism” that would lead to serious problems in explanatory power of research inquiry (Iosifides, 2017, p. 137). It is not our intention to bring scholars with different paradigmatic positionalities together in reconciliation on mixed methods, but to advocate for methodological pluralism and engagement with research philosophy in our field. While much future work is needed to develop and agree on core quality criteria for integrating qualitative and quantitative designs as well as for judging MMR rigour, the contribution of this paper lies in using critical realism to reconceptualise MMR as a philosophically coherent project that can increase the explanatory power of our research endeavours.

To move further toward developing a mixed methods methodology in the sport psychology community, we encourage the colleagues interested in or already conducting MMR from a critical realist perspective to seriously consider the implications of the meta-theoretical assumptions for their methodological decisions and ways of evaluating research process and product when preparing manuscripts for publications. By being transparent in reporting MMR designs and the ways methodological integrity is assessed, mixed methods researchers will develop a deeper understanding of this research inquiry while building a stronger MMR community in sport and exercise psychology.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Interpretivism encompasses a variety of paradigms; however, it is often presented as a qualitative research adhering to ontological relativism (see Smith & Sparkes, 2016). To signify this position, we refer to it as relativist-interpretivism.