Thinking again about the use of think aloud and stimulated recall methods in sport coaching

ABSTRACT

Understanding sport coaches’ thought processes in situ is a developing area within both research and applied practice. Research methods such as the Think Aloud protocol (TA) and Stimulated Recall (SR) interviewing have been used to gain qualitative insights into cognitive and behavioural aspects of coaching practice, offering value in moving beyond popular yet somewhat simplistic, traditionally decontextualised uses of standard interview-based methods. However, TA and SR have limitations when used in isolation, and their application has been varied. This article provides an overview of both methods, comparing the practicalities of using each, as well as paradigmatic considerations, applications and considerations for extending their use in sport coaching research. We suggest that with appropriate critical appraisal of some of the methodological issues raised, researchers can use both methods to better understand, integrate and develop collaborative theory-in-action, research and practice in sport coaching.

KEYWORDS:

• Qualitative methods
• experience sampling
• ecological momentary assessment
• cognition

Introduction

Understanding the underpinning in situ thinking processes of sport coaches is a growing area of interest within research and practice, despite limitations to methods of obtaining insight into coaches’ unseen reasoning. Inspired by shared disciplinary roots in psychology (Ericsson and Simon 1980) and education (Bloom 1953), both Think Aloud (TA) and Stimulated Recall (SR) interviewing have gained traction as methods for collecting cognitive data and investigating cognitions and experiences in naturalistic sport coaching research. Where TA provides concurrent capture of in-situ thought verbalisations in real time, SR methods involve inviting participants to retrospectively recall their cognitive activity during an event, typically aided by video or audio clips of their behaviour (Lyle 2003). Methods such as TA and SR developed from positivist origins, where researchers employ them seeking ‘objective’, ‘accurate’ and ‘reliable’ insight into thought and action, tending to reduce the complexity of an often complex context (Cushion 2022; Nichol and Hall 2024). Although these methods have more recently been recommended in sport coaching and wider sport research adopting a relativist epistemology (e.g. Eccles and Arsal 2017; Nichol et al. 2023, 2024), there is an orthodoxy of research in sport coaching adopting a positivist or post-positivist approach towards data collection methods and methods of data analysis (Potrac, Jones and Nelson 2014). The origins of a positivistic paradigm may remain in TA and SR work, for example, in the pursuit of ‘accurate’ recall in relation to cues of stimuli (Bloom 1953) or in quantifying verbalisations which appear to occur in a social vacuum (McGreary et al. 2021, 2024).

In contrast, it is important to consider that the coach and their cognitions are not isolated, decontextualised units of measurement and analysis. Underlying research approaches, strategies and practices (methodologies), combined with appropriate tools of data collection (methods) that allow for exploration of coaching practice in context should consider connections between coaches’ conceptual understanding, pedagogical practices and the wider socio-cultural realities of coaching (Cushion 2022). Here, (post)positivist-informed approaches are limited in their assumptions and methodologies to examine and explain the physical world, and their concern for generating ‘lawlike accounts of action’ (Potrac, Jones, and Nelson 2014, 32). Indeed, while TA and SR methods have value in providing additional insight or types of data that may not be obtained via lab-based or some more traditional interview-based methods, they are not unproblematic (Lyle 2003). Each method has strengths and weaknesses, and within current sport and coaching research and practice, there is an opportunity to apply the strengths of TA and SR with greater accuracy, flexibility and more explicit consideration of wider methodology and philosophical assumptions (Eccles and Arsal 2017; Nash, MacPherson and Collins 2022; Nichol and Hall 2024). To promote robust qualitative research, the ontological and epistemological underpinnings of methodologies must be examined, and then methods accurately applied to make the most of their strengths as part of a coherent approach. It is also important to articulate how and why each method is being used in line with the aims of research, enhancing the quality of conclusions, knowledge claims, and significantly for coaching, the practical implications made possible. This is of particular importance where the task of researching sport coaching is far from complete, and rare discussions beyond disciplinary perspectives towards philosophical and methodological levels are required to make progress (North 2017). Conversely, continuing to adopt the status quo of traditional approaches to TA and SR interviewing may result in surface-level, replicative inquiry into sport coaching, with coaching research and practice ‘left behind’ other progressive areas of sport, exercise and health.

As sport coaching attempts to erase the scholar-practitioner divide and mature towards greater methodological engagement and critical exploration of novel approaches (Jones, Corsby, and Thomas 2023), it holds great potential to contribute to a recent participatory turn in sport, exercise and health sciences (Smith et al. 2023). In this article, we aim to contribute to the discipline-specific qualitative methodological literature by providing an overview that describes the background to the development of TA and SR interview methods, appraising challenges and providing critical methodological considerations for researchers applying these methods to data collection (Jackman et al. 2022). The significance of this paper lies in examining the ontological and epistemological underpinnings of TA and SR methods, exploring new ways to engage in using them, and to what extent they could in combination align with participatory methodologies (Smith et al. 2023). We highlight the potential for sport coaching scholars and practitioners to benefit from these methods as an option to be ‘done better together’ (Pettican et al. 2023) with coaches, to develop action and cognition-related research and practice in sport coaching.

What are think aloud and stimulated recall interviews?

The think aloud method

The Think Aloud (TA) method requires participants to verbalise one’s thoughts whilst performing a task (concurrent TA), or recall thought immediately after the completion of a task (immediate retrospective TA) (Eccles and Arsal 2017). TA allows data regarding thought processes to be ‘captured’ in real time and reduces the influence that memory decay and retrospective ‘bias’ has on information gathered (e.g. Folkman and Moskowitz 2004). The potential limitations of collecting retrospective reports of cognition, thoughts and decisions were identified as far back as 1920’s, with researchers such as Watson (1920), suggesting that accessing participants’ thought processes during the completion of a task could be studied by asking them to think aloud. Ericsson & Simon (1980, 1993) then proposed a verbal protocol analysis method, aimed at identifying the cognitive processes of individuals while completing a task. Individuals are asked to verbalise their thinking (Think Aloud) during or immediately after an action in order to examine the sequences of a cognitive task or event occurring between the presentation of a problem and the generation of an answer. Ericsson and Simon (1980) proposed that the only information about mental processes that an individual is thought to be able to access and in turn verbalise is that attended to in the short-term memory (STM) during task execution. With some exceptions, the products of those processes are held in STM. These products or verbalisations are said to allow the ‘experimenter’ to make inferences about the processes themselves.

Ericsson and Simon (1980) describe the cognitive processes by which participants arrive at verbalisation of their cognitions, using a framework made up of three levels. Level 1 verbalisation involves simply verbalising inner thoughts during task performance. Level 2 involves the verbal encoding and vocalisation of an internal representation that is not originally in verbal code (e.g. verbal encoding and vocalisation of scents, visual stimuli, or movement). Level 3 verbalisation is where an individual is prompted to explain their thought processes. This requires retrieval of additional information from long-term memory beyond that already present in short-term memory (Ericsson and Simon 1993). Most research discussed here will refer to Level 1 or 2 verbalisation. We will refer to Level 3 verbalisation later, which has relevance for coaching research yet deviates from its original proposed use (Ericsson and Simon 1980, 1993).

Across sports, health and exercise, a growing number of researchers have used TA as a method of collecting information about individuals’ thought processes, typically in athlete-focused studies (e.g. Calmeiro and Tenenbaum 2011; Whitehead, Taylor, and Polman 2016; Elliott, Whitehead and Magias 2020; Kaiseler, Polman and Nicholls 2012; McGreary et al. 2021; Swettenham et al. 2020; Whitehead et al. 2018). More recently, TA has been applied in sport coaching to investigate football coaches’ cognitions (Horan and Whitehead 2020). The development of this research led to the application of TA in relation to coach reflection (Stephenson, Cronin and Whitehead 2020; Swettenham and Whitehead 2021; Whitehead, Taylor, and Polman 2016). This cluster of studies adopted level 3 verbalisation in an attempt to promote metacognition by the coaches engaging in TA. These studies offer some practical implications for the use of TA within coaching research and practice, which will be discussed later in the manuscript.

Stimulated recall

A key feature of sport coaching is that practitioners cannot report on their interactive cognitions during action – which requires verbalisation – without interrupting practice (e.g. the coaching session). Where TA protocols may therefore be limited by coaches’ ability to verbalise thoughts during practice without task interference, stimulated recall is a family of introspective procedures which aim to access cognitions by helping educators to ‘relive’ and review after the event. Participants are invited to retrospectively recall their cognitive activity, aided by a stimulus such as video or audio clips of their behaviour (Lyle 2003). Video SR was first described by Bloom in 1953 and picked up by Ericsson and Simon (1980) and Calderhead (1981) among other educational researchers, with variations used across decades of teaching, nursing and counselling research (e.g. Housner and Griffey 1985; van Driel et al. 2022). Examples from different paradigmatic standpoints have included ‘signalling’ in-action cognitions, using eye-tracking camera glasses which act as prompts for on-action follow-up interviews (van Driel et al. 2022), and SR interviews used in parallel with conversation analysis, rooted in ethnomethodology (Fosgaerau et al. 2021). These researchers aimed to explore professional interactions and actions, decision-making, strategies, values, beliefs and assumptions in ‘real life’ rather than experimental environments (Gazdag, Nagy and Szivak 2019). More recently, teaching research has emphasised SR’s ability to capture ‘interactive cognitions’, namely dynamic, situation-specific, ‘split-second’ thoughts embedded in practice and informed by situational context (van Driel et al. 2022). Its adoption in sport coaching was inspired by similarities in the complex, uncertain and interactive contexts (Lyle 2003). In other words, there is something about the educative context that lends itself to the use of this research method for addressing unseen reasoning.

Some researchers have highlighted the ability afforded by SR interviewing to activate more stable, substantial and tacit aspects that guide immediate behaviour, such as knowledge and beliefs (van Driel et al. 2022). Indeed, the retrospective nature of SR interviewing allows additional layers of depth to be addressed over and above participants’ thoughts at the time of action. Accordingly, there has been a recent shift towards the use of SR procedures in relation to practitioners’ professional development, promoting the developmental aspects of the method itself and providing insight into learning and reflection (Gazdag, Nagy, and Szivak 2019), a move paralleled in sport coaching research (e.g. Nash, MacPherson and Collins 2022; Santos, Camire, and MacDonald 2022; Stodter and Cushion 2019). Here, the SR interview procedure can be tailored to reflect the focus of the research question, for example, designing questions linking participants’ recalled cognitions to their changing knowledge-in-use, reasoning and learning (Stodter and Cushion 2019). Moreover, SR interviewing fits well in extending the popular use of video to observe coaching behaviour while allowing exploration of underpinning rationales of practice (see Cope, Partington, and Harvey 2016). These strengths have meant that the method has established support through its promotion of coaches as active ‘participants’ rather than passive ‘objects’ of research (Cope et al. 2022; Trudel, Gilbert and Tochon 2001).

Practicalities of using TA and SR

Logistically, conducting TA within coaching simply requires a clip microphone and dictaphone, or coaches may use wireless headphones and their own mobile phone to record verbalisations (depending on ethical requirements). Meanwhile, there is variation and discrepancy in data collection methods employed in relation to SR techniques (Gazdag, Nagy, and Szivak 2019; Nichol and Hall 2024). The use of video as a stimulus for interview recall of cognitions firstly requires filming the activity of interest, such as coach-led training sessions or sport competition, with a video camera and a remote microphone for audio. Although video recordings tend to focus on the coach rather than the athletes, researchers may need to gain additional consent from athletes (and/or their parents/guardians in the case of under 18). Activities can be filmed from the coach’s point of view or the corner of the playing or competition area. The latter allows much of the action to be captured while recording what the coach is doing and seeing as they move around conducting their ‘normal’ activity. In line with the origins of SR interviewing and where researchers subscribe to ontological realism and epistemological objectivity, the aim would be to capture footage (a stimuli) as complete and as close to ‘reality’ as possible. Nevertheless, it may be inevitable that certain actions, interactions, or dynamics are missed, highlighting the issue of how SR methods are conceptualised. A more relativist position might seek to use footage less as ‘stimuli’ for ‘truth’, rather as one tool through which the researcher can generate in-depth data, interpretations and rationalisation (Nichol and Hall 2024) in line with their research questions.

Following filmed sessions, imported video files can either be reviewed in full to scan for participants’ in-action signals or ‘clipped’ into short incidents or general activity. In line with early SR interview protocols outlined by Calderhead (1981), Lyle (2003) and Wilcox and Trudel (1998), participants can be asked after each coaching session if there were any incidents that they might discuss in the forthcoming interview. A drawback here is that participants, intentionally or unintentionally positioned as ‘separate’ from the researcher and the research questions, may not make suggestions. Accordingly, the SR method may work better when there is something specific of interest for participants to recall and where this is generated collaboratively in advance. Depending on the research questions, incidents for discussion may include coach interventions, interactions with athletes, athlete behaviours, decisions, passages of play, demonstrations or practice set-ups. Interview guides are created in line with research questions to correspond with video clips, following a semi-structured format based on occurrences chosen by participants and the researcher together (Bernier et al. 2011; Stodter and Cushion 2019).

The process of introducing and familiarising participants to TA and SR methods is important, however some previous research has provided limited or non-domain specific information on training methods. Eccles and Arsal’s (2017) ‘The think aloud method: what is it and how do I use it?’ I suggest following Ericsson and Simon’s (1993) procedures, which involve participants thinking aloud whilst solving alphabet-based problems then engaging in putting practice. Birch and Whitehead (2019) highlight the importance of task-specific TA practice to increase familiarity and to facilitate a more ‘natural’ collection of cognition data. Where participants feel at ease with the process, they are not overthinking the procedure. Within sport coaching, Whitehead et al. (2016) used role-play exercises, where coaches would verbalise their thoughts to the person next to them to get used to the experience of thinking aloud. Participants also had the opportunity to discuss the practical applications of TA within their coaching environment. Nevertheless, there are no agreed guidelines for training coaches to use TA, something which future research should consider.

Likewise, the preparation of participants is an important stage in the SR protocol that is often missed or not reported. Participants may find this method of data collection trying, due to the direct, intense and emotionally demanding nature of watching and hearing themselves in action, and being immediately asked to comment. This is exemplified by participant coaches reflecting that ‘there’s nowhere to hide’ (Stodter 2014, 68). Yet the use of SR can be a powerful tool to illustrate the in-action thinking of coaches, and if required, added layers of rationale and other data of relevance to research questions. For example, the following quotation demonstrates a participant’s justification for intervening in a coaching session, indicating the content of the intervention as well as links to where they perceived learning the underpinning knowledge from:

I stepped into the whole group a couple of times to get, again, some of the basics out and then some of the technical info of running with the ball.

So just those technical details in different situations that you’ve learnt, but not only through mainstream courses, but also like your experiences of playing and also things that you see other coaches and other players do in situations. (Stodter and Cushion 2019, 2089)

On face value, these data seem to be more retrospectively- and rationale-focused than the immediate thought of TA examples (below). Although the data afforded by SR interviews may be more complete than TA in terms of full sentences and time available to explain reasoning more fully, the quality of the data can depend on the individual participant’s memory, capacity to report introspective reasoning (Lyle 2003) and openness or honesty. There are issues worth considering around participants’ performance, distortion, or presentation of self (Partington and Cushion 2013) in interplay with researcher-participant power dynamics. The researcher’s skill in listening, rapport-building and challenging while remaining judgement-free can enhance the data yielded, but similar to TA, the method takes time for participants to acclimatise to. Over time, repeated SR interviews can become more conversational with collaboration and exchange of ideas, leading to qualitatively rich data. The retrospective and collaborative nature of the method may also initiate discussion of ‘a lot of outside influences that perhaps isn’t evident when you’re watching it’ (Stodter 2014, 68). Participants have reported finding the use of video footage and the SR interview process itself useful for their coaching development and even the co-construction of knowledge (Cope et al. 2022; Nash, MacPherson and Collins 2022; Stodter and Cushion 2019). This strength in the depth and range of data that the SR interviewing process affords can also be a weakness where what is collected may deviate from researchers’ intentions, which is important to acknowledge in the application of this method as well as the analysis and interpretation of the data produced.

What kinds of data are produced?

TA data from sport coaching provide varying degrees of ‘completeness’. For example, Whitehead, Taylor, and Polman (2016) provide short examples of TA where golfers verbalise statements such as ‘It broke at the end’ and ‘arms bent’. However, coaches have been seen to share stressors through Level 1 and 2 verbalisations. In relation to referee interference with play, one coach said:

So the noise there was the ref who was in the way of the play. Third time he’s been in the way of the play. Two seconds later he was in the way. Josh fouled again because you can see the frustration in him from trying to make key passes and the referee is constantly in the way. We’re constantly trying to play central balls but he keeps on being in the centre too much. (Horan and Whitehead 2020, 25)

Where TA has been used as a reflection tool, it can be seen (below) that there are multiple reasonings for why thoughts and actions occur. This moves into ‘Level 3’ verbalisation where providing explanations promotes coaches’ metacognition or thinking about their own thought processes:

Ok so this is making me feel good because they’re working hard and scoring goals and it’s challenging them with the two touches but I think the intensity has dropped a little but I’m not sure why. (TA session 1). (Stephenson, Cronin, and Whitehead 2020, 16)

It is important to highlight that the instruction researchers give participants surrounding the level of verbalisation should be influenced by the aims of the research, either 1) to ‘accurately’ understand coach cognition as it occurs in situ (e.g. Horan and Whitehead 2020) or 2) to promote ‘thinking about thinking’ and even making sense of thoughts in the moment, which disrupts naturalistic thought processes and deviates from the original TA method (Ericsson and Simon 1980, 1993; Stephenson, Cronin and Whitehead 2020; Swettenham and Whitehead 2021). Similarly, researchers using stimulated recall methods have struggled with the separation of recalling in-action thought processes from additional layers of reflective thought and sense-making after the event. The timing of both SR interviews and the presentation of stimuli within interviews has been subject to much debate linked to philosophical assumptions. In aiming to minimise the data collected diverging from the cognitive processes being employed at the time of the coaching event, there is consensus that interviews should happen as soon as possible after the event (Gilbert and Trudel 1999). It has also been recommended that video clips be shown after participants have recalled each incident as an additional prompt, to prevent them immediately reporting additional layers of retrospective reflection on reviewing the video (Lyle 2003). These strategies reflect a positivist concern for accuracy ‘true’ to the cognitive processes happening at the time of coaching action. These challenges reflect ongoing underlying ontological and epistemological tensions, which we will explore after addressing the analysis of data produced through these methods.

Analysis of data

The approach taken to applying TA and SR methods of data collection, and in turn analysing data arising from TA or SR interviews, is influenced by research paradigm, which has been acknowledged in the sport coaching literature to varying extents. In line with these methods’ origins in positivist, empiricist paradigms, researchers employing TA methods have approached data analysis using techniques designed to reduce complexity and test hypotheses, such as protocol analysis technique (Ericsson and Simon 1993). Similarly, creating and applying coding schemes is popular across research using TA and SR (e.g. Calmeiro and Tenenbaum 2011; Calmeiro, Tenenbaum and Eccles 2010; van Driel et al. 2022). For example, in a golf putting study, Arsal et al. (2016) transcribed TA audio, then broke up participants’ verbal reports into separate thought statements by identifying units of meaning and natural pauses in participants’ speech. The authors then engaged in a task analysis to identify the types of functions and thoughts verbalised during the putting task. From here, a coding scheme was developed which included codes relating to overarching categories. Quantitative analysis was then conducted to compare the frequency of codes within each category and compare differences between skill levels of performers, with inter-rater reliability checks conducted. Ericsson and Simon (1980) highlight the importance of knowing the data and data collection context, where coders who fully understand the task can code with reliability scores of up to .8 or .9.

An issue with this type of analysis is that the reader will gain very short insight into some verbalised codes of information (especially within TA), rather than gaining an understanding of how these thoughts could influence the performance of coaches or athletes in contexts assumed to be associated with consistent, ‘universal’ cognitive processes. Uniform application of coding frameworks across contexts and over time, for example, in different phases of professional development, holds potential in simplifying coaches’ complex cognitions, how they relate to behaviours and how they might or might not change in different situations. Depending on the approach taken though, collecting and then analysing cognitive data in this way can present challenges for the analysis process, particularly in aiming to move beyond description or simply counting codes. The deductive approach to analysis and the use of statistical methods can lead to relevant or worthwhile data being overlooked (Calmeiro and Tenenbaum 2011). The underpinning assumptions that TA or SR data analysed through coding schemes constitute valid and objective ways to ‘capture’ the ‘universal’ phenomenon of coaching, as understood through coaches’ cognitive processes, can also be seen as problematic given the socially interdependent, context-specific and complex nature of sport coaching.

Whitehead and Jackman (2021) challenged previous quantitative work on TA by arguing that (athletes’) cognitive processes are more complex than simply following a sequential or linear process, and participants may deliberate and make recursive attentional shifts that necessitate qualitative exploration. These authors adopted inductive thematic analysis, and measures of trustworthiness were outlined to provide information on steps taken to improve the quality of the research (Sparkes and Smith 2014). Similarly, recent SR research in sport coaching has tended to employ a range of inductive, deductive and ‘abductive’ qualitative analyses including thematic analysis and grounded theory (e.g. McGuckin et al. 2022; Nash, MacPherson and Collins 2022; Santos, Camire and MacDonald 2022; Stodter 2014). A further under-addressed challenge relates to selecting appropriate processes for longitudinal qualitative data in analysing change over time, for example, when addressing coaches’ learning or development (Saldaña 2003). It is clear that care must be taken over research designs (Lyle 2003) for researchers to be sure that the way they collect and analyse TA or SR interview data aligns with their paradigmatic approach and allows for robust answers to the intended research questions (or hypotheses, depending on the approach taken).

Ontological and epistemological issues

In previous sections, we addressed the background to and practicalities of data collection using TA and SR methods, and many of the considerations raised link to the coherence of underpinning paradigms which rigorous approaches to the research process must address. In this section, we consider the theoretical bases of TA and SR and their location within the developing literature on sport coaching. By doing so, we hope to promote a more secure ontological and epistemological basis for understanding and implementing each method as part of a coherent research process that does justice to the social complexity of the sport coaching world.

Based on its disciplinary roots in psychology, TA stems from cognitivist assumptions, where the participant is seen as an information-processing machine (Eccles and Arsal 2017) that can be objectively accessed and their thoughts accurately measured through verbal reports. As previously described, typical examples have taken a positivist or post-positivist approach (e.g. Arsal, Eccles and Ericsson 2016; Whitehead et al. 2018). In a recent mapping review of the use of TA within sport and exercise psychology research, the heavy emphasis on post-positivist approaches was evident (McGreary et al. 2024). Out of 36 studies only three reported adopting a subjective, constructivist-informed or critical realist approach (McGreary et al. 2021; Welsh, Dewhurst and Perry 2018; Whitehead and Jackman 2021). Further emphasising how within much of the TA research verbal data is seen as an objective and valid reflection of actual thought processes and may be grouped into themes and quantified, seeking statistically significant differences between themes in line with researcher-driven, predetermined hypotheses.

SR protocols originated from a similar desire to objectively ‘get at’ unseen reasoning. Such positivist process-product paradigms, by focusing on controlled, objective, reliable and valid data, tend to reduce complexity and assume transferability across contexts, which is particularly problematic in applied coaching research (Cushion 2022). Santos et al. (2022) used SR interviewing within an interpretivist paradigm, allowing coaches to reflectively discuss how they had applied new material after a coach education course. Eccles and Arsal (2017) claim that from these standpoints, verbal reports would simply not be positioned as valid and objective measurements of participants’ thoughts, instead acknowledging the data as relationally and socially constructed leading to different, not better or worse, results and knowledge claims. Pragmatic (e.g. Stodter and Cushion 2019) and critical realist paradigms (e.g. McGuckin et al. 2022; Nichol et al. 2023) present further opportunities for the selection and coherent application of TA and SR interviewing methods in coaching. For the former, the selection of these methods is driven by ‘what works’ in helping to answer the research question or problem of interest, yet different variants foreground different ideologies and philosophical systems (Jenkins 2017). Critical realism, meanwhile, acknowledges the limitations of methods for getting as close as possible to approximating ‘real’ causal mechanisms (McGuckin et al. 2022) with some positions holding interactions and relations between structure and agency central (e.g. Nichol et al. 2023). Nevertheless, the deployment of these methods within some pragmatic and critical realist approaches can retain a fundamentally internal, individualistic focus, running the risk of neglecting the need for information about and understanding of interplay with social, cultural and historical context to be included in coaching research (Cushion 2022).

To capture only TA or SR data without augmentation from other methods means that we may not fulfil a desire to understand coaching contexts and the complex socially interdependent dynamics present. Given the time and flexibility available for addressing reasoning and implicit theories, the method of SR interviewing appears better placed than TA to allow for embracing and considering the significant contextual, social, cultural and historical influences on sport coaching and associated cognitions. Equally, while video SR interviewing works well as a method for starting conversations about sport coaching practice, it does not lend itself to uncovering ‘true’ cognitions ‘as-they-happen’ or as a universal technique for research (Nguyen et al. 2013). We suggest then that researchers can move beyond the original, limiting use of these methods to seek a universally applicable, objective reality that risks positioning coaches as rational, homogenous, information-processers. Rather, if we consider other applications underpinned by onto-epistemological relativism, when ‘what can be known’ depends on specific cultural settings and contexts, we can alter traditional researcher-participant power relations and open up further possibilities for understanding sport coaching.

Practical applications – thinking again

Coaching researchers might usefully ask themselves then whether they believe cognitive processes truly or independently exist, and whether TA and SR methods are valid ways to produce knowledge about them. Ericsson and Simon claim that level 3 verbalisation, departing from short-term memory to bring in information not attended to during task performance, such as implicit theories, reflections or meta-cognition, constitutes ‘less-valid data’ (Eccles and Arsal 2017). Early SR interview approaches reinforce that the key to validity issues lies in ensuring that questions/prompts do not alter the cognitive process being employed at the time of the event (Nguyen et al. 2013). Where the original approaches intend to control for and ‘strip out’ these layers of cognitions, we contend that this is not necessarily a negative issue for the applied field of sport coaching. Allowing additional layers of cognition may even become an advantage for working with coaches, for the generation of more ecologically valid data embedded in ‘real world’ contexts. Indeed, asking coaches to explain and make sense of their thoughts in the moment (Level 3 verbalisation, as outlined below) can assist researchers to gain valuable insight into contextual, cultural and social influences within the coaching process as it occurs in situ.

Studies adopting TA or SR have found that these research methods can promote coaches’ reflection and metacognition (e.g. Stodter and Cushion 2019), which are significant processes for coaches’ learning. Whitehead et al. (2016) deviated from the traditional use of TA to apply the method as a reflection tool with rugby league coaches. Supported by reflective workshops, coaches were instructed to use Level 3 verbalisations which required the coach to explain their thoughts where possible during a live coaching session, in turn promoting a process of coaches ‘thinking about their thinking’. Participants’ reflections moved from descriptive to analytical as they developed their reflective skills. Coaches reported developing self-awareness, improving communication and improvement in their pedagogical practices. Similar findings have been presented in soccer coaching (Stephenson, Cronin and Whitehead 2020; Swettenham and Whitehead 2021). Within this body of work Stephenson et al. (2020) adopted an interpretivist lens, which allowed them to consider multiple methods of data collection, such as written reflections and retrospective interviews, in addition to TA. The authors acknowledged the complex and socially situated accounts of coaching experience and emphasised the importance of the researcher and the participant (coach) working together to make sense of their world. Through working with coaches we can move away from ‘research parasitism’ (Stone and Priestley 1996) and involve participant groups (coaches) within decision-making processes. For example, within Stephenson et al. (2020) the participant was not instructed what or how to engage in TA and was given autonomy on if, when and how to engage with reflection-on (coaching) action. The focus of this research was shifted to support the coach in their development, in addition to gaining an understanding of experiences. Member reflections (Smith and McGannon 2018) were used to understand how researcher and participant interpretations of the data aligned or contradicted.

Likewise, video footage of coaching practice can provide structure to reflective conversations and trigger coaches’ behaviour change (Partington et al. 2015). As such, the video stimulus part of SR interviews means that the method has the potential to function – intentionally or unintentionally – as a ‘guided reflection’ intervention for participants, besides its use as a data collection technique. In a departure from its original purposes, SR interviewing provides the opportunity for participants to view their own behaviours on video and think about and discuss the origins and outcomes of knowledge-in-action and practice. It may be that shared video feedback and reflection afforded by SR interviewing facilitate learning by bringing tacit cognitive processes to consciousness and conceptualising practice, then integrating altered communally developed theory into action (Gilbert and Trudel 2001). Indeed, Eraut (2000) has claimed that practitioners’ performance could be enhanced by making procedural knowledge, such as the type vital for knowing how to implement concepts in context, more explicit. The ‘genuine feedback on the outcomes of action’ afforded by video methods is crucial in allowing practitioners to step ‘outside their taken-for-granted world’ and close the distance between practical theories-in-use and more abstract espoused theories of practice (Eraut 2000, 123). Therefore, SR interviewing has much potential, in tandem with forms of systematic observation, as a coach development tool where through dialogue, practically relevant forms of knowledge can be co-constructed for the advancement of coaching practice (Cope et al. 2022; Nash, MacPherson and Collins 2022). The combination of TA and SR could allow for more holistic rationalisation and meta-cognition through both watching and listening back to in situ coaching actions and different types of thinking in different moments of practice. Repositioning TA and SR interviewing in this way can also enable practice-linked investigations into how reflection guided by these methods works as an informal learning sourcet (Stodter and Cushion 2019). It is important for researchers conducting this kind of work to maintain an awareness of the knowledge claims made possible, where any objective, impassive judgements of cognitive processes would be problematic. From an ethical perspective too, researchers need to consider their potential influence on coaches’ naturalistic practice and the possible wider implications of this, playing into the need for participants’ (and potentially athletes’) informed consent.

Research has largely failed to impact practice in sport coaching (Lyle 2018) with academics and intellectualism sometimes viewed suspiciously by coaching practitioners (Jones, Corsby, and Thomas 2023). In contrast, TA and SR methods offer strength in their accessibility and applicability for coaches to be involved in and benefit from. Challenging researcher-participant power relations and working to minimise risks that methods and resulting data may be used for surveillance or making judgements of what underpins ‘good coaching’, and these methods can be adapted for the context to enable research with not on coaches, facilitating ‘collaborative action’ between research and practice (Cushion 2022). Rather than superficial application of these methods which maintains the focus on coaches as merely the object of study, sport coaching scholars could benefit from engaging with methods aligned to participatory methodology, along with its particular ontological and epistemological underpinnings. Specifically, a participatory approach would enable coaches (those with personal experience in the topic of research), wider stakeholders such as athletes, and scholars as co-researchers to rethink established forms of interactions, power relationships and interpretations of situations in a more reflexive and democratic collaboration (Smith et al. 2023). Following a recent participatory turn in the wider sport, health and exercise sciences (Smith et al. 2023), this is a promising avenue for engaging diverse or marginalised coaches working in different domains who are often excluded in processes of knowledge production (Pettican et al. 2023). These concerns for ‘rehumanising’ sport coaches, their craft, and their outcomes through research that works towards social justice are particularly relevant to sport coaching, where research participants have been predominantly male, European and North American, and working at performance levels of sport (Hirsch et al. 2023). Following the Moving Social Work Co-production Collective, Smith et al’.s (2023) typology, equitable and experientially informed co-produced research allows inquiry to be done by coaches, embedded in context, and begins to rebalance academic knowledge and practitioner knowledge on an even plane (Lyle 2018). Sport coaching and TA and SR methods are ideally placed here in bridging the ‘know-do-gap’ (Leggat et al. 2021) through co-producing the differences that can be made to coaching practice (Buckley et al. 2022). TA and SR methods could be applied as options for coaches involved in shaping research methodology that supports their coaching development while also increasing the diversity of knowledge produced, valued and shared, enhancing epistemic justice (Smith et al. 2023). Equally, this type of approach is far from straightforward. Participants would need to be introduced to and trained in each method, with flexibility afforded in ongoing conversations and leadership shifts with researchers to enable adapting or changing methods as projects continue over time and across dynamic contexts (Buckley et al. 2022). Participants might also decide that other methods would be better suited to the issues of their interest and may even generate their own tools for data collection.

Bringing it together – recommendations

Educational research has often combined the use of SR and TA together with other methods in order to enhance validity and methodological triangulation (Gazdag, Nagy and Szivak 2019). A hybrid of both approaches is fairly common, but given that TA data could lack completeness and links to context, ‘nor is the [SR Interview] method likely to be of use entirely on its own’ (Calderhead 1981), we suggest that a well-considered use of both TA and SR could lead to the generation of richer practice-linked data in sport coaching. Understanding the strengths of TA to gain a partial picture of ‘in event’ actions and cognitions in addition to the use of SR to gain a wider view of what surrounding cognitions occur and the socioculturally linked reasons why they may be occurring within the context of the activity will provide researchers with more nuanced and layered data to work with. A singular focus on using any one of these methods alone will not allow us to gain a complete picture on something so complex and dynamic as sport coaching in context (Cushion 2022). Researchers will then need to be both inventive and modest with their aspirations (Eraut 2000) and cognisant of the underpinning paradigm as explored above.

Further, research should consider the longitudinal nature of data collection in situ with coaches, rather than one-off ‘drive by’ work that is done to coaches. Designs involving multiple points and types of data collection over time can better account for issues of early discomfort, impression management and observer effects – in other words, beginning to level the outcomes of power imbalances in moving towards a more participatory approach. They also enable the involvement of individual coaches, groups of coaches and even athletes or other stakeholders, to provide a self-referenced index of temporality and change over time. This lends itself to investigations of dynamic processes such as coaching (non)influence or learning (e.g. Nichol et al. 2023). However, the limited available guidance on the analysis of such longitudinal, layered and interdependent data can present challenges for researchers.

A related point to consider for researchers interested in using TA and SR methods together relates to the use of theory to guide implementation, analysis and interpretation. While some research in coaching has been criticised for being atheoretical, for example, describing what coaches say (or think) without more deeply engaging in what that might mean and why, researchers could also be accused of over-relying on methods like TA and SR without engaging in theories of relevance to sport coaching first. Theories that have been used with TA research tend to reflect the dominant psychological roots of the method, for example, around stress and coping (Horan and Whitehead 2020) or expertise (e.g. Deliberate Practice, Ericsson and Simon 1993). Meanwhile, coaching studies have used SR interviewing to assess coaches’ perceptions or application of Transformational Leadership (McGuckin et al. 2022) and Positive Youth Development approaches (Santos, Camire and MacDonald 2022) rather than using theory explicitly to guide what is seen in the data and how it is interpreted. We encourage researchers adopting these methods to draw from good examples of considering and reporting the interlinks between theory and data in making sense of what they are trying to understand, in line with the aims of their research (e.g. Nichol et al. 2023).

That is not to say that the combination of TA and SR is without limitations. For example, investment of time and buy-in are crucial. As previously highlighted, the potential power dynamics at play and the perception of these will impact what a coach verbalises and how they behave. Finally, methods other than TA and SR such as participant observation and event focused interviews (Jackman et al. 2022) also offer valuable insight into the complex nature of thought processes, cognition, and decision-making. It is not our intention to discourage researchers from avoiding these methods, and we encourage scholars to consider how these methods can be used complementarily within a coherent paradigm, in the light of the wider participatory turn in sport, exercise and health (Smith et al. 2023).

Conclusion

In this article, we have provided a comprehensive overview of the background, benefits, limitations and considerations for using TA and SR interviewing as methods for better understanding the cognitive processes underpinning in situ coaching practice. Despite prevalent issues around accessing ‘accurate’ or ‘true’ concurrent thoughts and limiting layers of reflective reasoning, each method alone may be considered better than decontextualised laboratory simulations or the ubiquitous traditional application of semi-structured interviews, in their use of ‘what coaches do’ as a conversation starter to identify what is important to coaches and how they make meaning through their day-to-day practice. We highlighted the potential for TA and SR to function as pedagogical tools, extending research as a powerful process of professional development where coaches can be involved in enhancing self-awareness, more explicitly connect theories with practice, advance their metacognition and critical reflection skills and make meaningful changes to practice (Cope et al. 2022; Stodter and Cushion 2019). In other words, developing the notion of doing research on coaches and moving to doing research with, by and for coaches and coaching practice.

It is hoped that this article can inform sport coaching researchers when considering both TA and SR methods and help avoid confusion as to their differences and application (e.g. Nash, MacPherson and Collins 2022). Beyond this, we have expanded upon recent calls for more attention towards rigorous and coherently applied methodologies in sport coaching research, in particular, ones that directly link to coaching practice in context (Cushion 2022). While acknowledging the methods-driven focus of this article, underpinning paradigm, theory and importantly, coaches’ issues of practice and coaching practice contexts, are ideally understood as a starting point rather than relying on a ‘methods-first’ or retrospectively rationalised approach to research. Here, the methods of TA and SR interviewing have vast potential to be done together with coaches, who can usefully shape and benefit from research moving towards a wider participatory turn in sport, exercise and health research (Cope et al. 2022; Smith et al. 2023). By combining methods and considering their ontological and epistemological development, we enable not only the investigation of cognitions as they occur in real time but also the meanings, origins and social influences that underpin cognition. Such efforts to think again about methods and methodology and involve diverse coaches and wider stakeholders across contexts in integrating research and practice can guide joint collaborative action and bring about positive impacts for the field of sport coaching.

Acknowledgements

The authors would like to thank Professor John Lyle and the reviewers for sharing helpful thoughts during the preparation of this manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Anna Stodter

Anna Stodter is a senior lecturer in sport coaching within the Carnegie School of Sport at Leeds Beckett University, United Kingdom. She is currently vice-president of the Cluster for Research into Coaching (CRiC). Her research interests in sport coaches’ learning and education inform and integrate with her teaching and applied practice as a coach in rugby union.

Amy Whitehead

Amy Whitehead is a Reader (Associate Professor) in Sport Psychology and Coaching at Liverpool John Moores University and is the lead of the Coaching and Pedagogy Research Group at LJMU. Amy is also a BPS Accredited and HCPC registered Sport and Exercise Psychologist and works as a sport psychology consultant with a range of sports and athletes.