Examination of depth of written reflective practice for speech-language therapy students; the impact of time and clinical competency

ABSTRACT

Written reflective practice (WRP) is a teaching and learning activity utilized in clinical programmes. Identifying depth of WRP aims to provide an overall rating of students’ written reflective abilities. Engaging in reflective activities has been proposed to have a positive relationship with clinical competency. Aims: 1. To examine the impact of time on depth of WRP for SLT students across and within each year group of the SLT clinical programme. 2. To determine whether a relationship exists between depth of WRP and clinical competency of SLT students. Methods: Participants were 77 undergraduate SLT students (first, second or final professional year) in the clinical programme. Participants wrote reflections as part of their clinical education experiences. Depth of WRP was measured utilizing a modification of Plack et al.’s [2005. A method for assessing reflective journal writing. Journal of Allied Health, 34(4), 199–208] coding schema. This was completed at four time points across the academic year for each professional year. Level of clinical competency was assessed using the Competency Assessment in Speech Pathology (COMPASS^®) at the end of both semesters. Results: There was a statistically significant association between time and development of depth of WRP for students in their final professional year (β = .66 (.30), z = 2.22, p < .05). There was no association between depth of WRP and level of clinical competency. Conclusion: A one-off judgement of WRP depth may be useful for supporting overall judgements of RP ability and provide observable behaviours for competency assessments. This research contributes to the evidence base examining how WRP is assessed and utilized in clinical programmes.

KEYWORDS:

• Reflective practice
• speech-language therapy student
• clinical competency

Introduction

The objective of Speech-language Therapy (SLT) programmes is to develop students into competent clinicians. Competency development is described as both the participation in clinical practice and learning from clinical practice (Walker, 1985). Specific to SLT, competency is defined as ‘ … combinations of knowledge, skills and personal qualities that contribute to sets of occupational and professional competencies that combine to create competent professional performance … ’ (Sheepway, Lincoln, & McAllister, 2014, p. 190).

Across allied health, nursing and medical clinical education programmes competency is measured in a variety of ways and differs between programme, governing body, licencing board and country of training. In general, the governing bodies or licencing boards have identified the core areas, minimum skills or certification standards required on entry to their specific profession. This level of competency is also termed entry-level practice (Fouad et al., 2009; Nursing Council of New Zealand, 2022; Occupational Therapy Board of New Zealand, 2022; Physiotherapy Board of New Zealand, 2018; Speech Pathology Australia, 2020; Standards of Proficiency for Speech and Language Therapists, 2014).

Clinical competency for SLT clinical programmes

Specific to SLT students training programmes in Australia, New Zealand and South East Asia the entry-level practice competencies are derived from seven Competency Based Occupational Standards for Speech Pathologists (CBOS) competencies (Speech Pathology Australia, 2011). An additional four professional competencies (Reasoning, Communication, Learning and Professionalism) are combined with the CBOS standards and comprise the Competency Assessment in Speech Pathology (COMPASS^®). This valid and reliable tool assesses and tracks competency development for SLT students (McAllister, Lincoln, Ferguson, & McAllister, 2013a; Speech Pathology Australia, 2011). The COMPASS^® instrument has been utilized in research as the measure of competency in studies examining clinical competency as influenced by placement type and simulation (Hill et al., 2021; Sheepway et al., 2014).

Additionally, requirements for assessing each area of competency are individualized across SLT clinical programmes and are tailored to meet the requirements of that programme’s university quality assurance process (e.g., Academic Quality Assurance of New Zealand Universities) and accreditors (e.g., New Zealand Speech-language Therapists’ Association Programme Accreditation Committee) One such example, is that the New Zealand Speech-language Therapists’ Association Programme Accreditation Committee requires all programmes to be accredited against The Aotearoa New Zealand Context. This is specific to SLT programmes in New Zealand. However, the exact choice and design of assessment is typically determined by the clinical programme, and considers the needs of their student population and university regulations.

Predicting student clinical competency

Identifying factors that could either predict or contribute to student development of clinical competency is of particular interest to researchers in academic and clinical education programmes. The identification of factors critical to the development of clinical competency would allow educators to streamline the admissions process and/or provide bespoke and tailored support during training that better meets student needs (Boles, 2018; Middlemas, Manning, Gazzillo, & Young, 2001). Past studies in athletic, dental, SLT, nursing and medical student training programmes examining the ability to predict academic and clinical competency prior to admission to the clinical programmes or prior to workplace practice have yielded varying results (Blackman, Hall, & Darmawan, 2007; Boles, 2018; Gadbury-Amyot et al., 2005; Stacey & Whittaker, 2005; Troche & Towson, 2018). One key measure used to predict competency is that of grade point average (GPA). GPA alone has thus far been found to be an ineffective predictor of future academic and/or clinical competency (Blackman et al., 2007; Boles, 2018; Stacey & Whittaker, 2005; Troche & Towson, 2018). Moving away from GPA, use of a one-off objective structured clinical examination (OSCE) early in medical training did predict future success for clinical competency in clinical programmes and postgraduate training (Wallenstein, Heron, Santen, Shayne, & Ander, 2010).

Beyond academic and clinical performance, an individual’s ability to accurately rate their own competence in clinical practice has also been suggested and examined as a measure of competency (Blackman et al., 2007; Chambers, 1993). Self-assessment as part of student portfolios is commonly used in clinical programmes and has been suggested as a more robust way to assess competency over time rather than a one-off assessment (Gadbury-Amyot et al., 2005). Self-assessment is also utilized as one part of the COMPASS® assessment for SLP students (McAllister et al., 2013a; McAllister, Lincoln, Ferguson, & McAllister, 2013b ).

Reflective practice (RP) is a further learning tool and form of self-assessment that is utilized with students in clinical programmes. Specific to SLT, RP is defined as a ‘ … means by which learners can make sense of and integrate new learning into existing knowledge’ (McAllister & Lincoln, 2004, p. 125). RP activities have also been used as a piece of assessment, to contribute to either clinical competency or academic grades, or a combination of both in clinical education programmes (Boud, 1995; Levett-Jones, 2005; Musolino, 2006). The utilization of RP can be found as written reflections (WR) included in ePortfolio assignments (Gadbury-Amyot et al., 2005; Walton, Gardner, & Aleksejuniene, 2016); reflective discussions to support practical assessments (VIVA VOCE) (Orrock, Grace, Vaughan, & Coutts, 2014), stand-alone WRP assignments (Cook et al., 2019), and evaluations of clinical competency (e.g., COMPASS®, (McAllister et al., 2013a, 2013b)).

Reflective practice and development of competency

RP skill has been long described as a stepping stone towards clinical competency, yet empirical evidence supporting this relationship is limited and outcomes of studies have found mixed results (Caty, Kinsella, & Doyle, 2015; Schön, 1983; Schön, 1987). To date, examination of clinical competency and RP has largely been presented via one-off assessment tasks, or student perception of RP and competency development (Chabeli, 2010; Domac, Anderson, O’Reilly, & Smith, 2015; Lim & Low, 2008b, 2008a; Mamede, Schmidt, & Penaforte, 2008; Tsingos-Lucas, Bosnic-Anticevich, Schneider, & Smith, 2017). Two studies examining the perceptions of nursing student have suggested that a positive relationship exists between RP and clinical competence level (Eng & Pai, 2015; Pai, 2016). In comparison, a study examining clinical competency of occupational therapy students revealed a positive relationship between the student self-assessment scales for reflective practice and occupational therapy competence, but not between student self-assessment of reflective practice and the educator rated clinical performance level (Iliff et al., 2021).

WRP is a RP activity that is regularly utilized in clinical programmes, including SLT where development of RP skills over time as clinical experience increases has been reported (Cook, Messick, & McAuliffe, 2022; Dunne, Nisbet, Penman, & McAllister, 2019). For students in clinical programmes, educators suggest that WRP skills are important for clinical competency development or contribute to clinical competency. However, this has not been tested for students despite being embedded into clinical programmes (Aronson, Niehaus, Hill-Sakurai, Lai, & O’Sullivan, 2012; Caty et al., 2015; Chabeli, 2010; Cook et al., 2019; Cook et al., 2022; Halton, Murphy, & Dempsey, 2007; Ng, Bartlett, & Lucy, 2012; Plack, Driscoll, Blissett, McKenna, & Plack, 2005). Furthermore, studies examining changes in SLT student WRP skills over time or with increased clinical experience have indicated a positive relationship exists for depth of WRP over a six-week period (Cook et al., 2019). Depth of WRP is an overall judgement of WRP skill, whereby SLT students are judged as falling into one of five categories (‘Non-reflector’, ‘Emerging-reflector’, ‘Reflector’, ‘Emerging-critical-reflector’, or ‘Critical reflector’. See Table 1 for definitions) (Cook et al., 2019; Plack et al., 2005). Alternatively, three trajectories of WRP skill development were identified over a period of ten weeks for SLT students: ‘steady growth’, ‘no clear change’ and ‘gradual decline’ (Dunne et al., 2019). However, SLT student WRP skills have not been examined in relation to clinical competency.

Table 1. Coding schema for depth of written reflective practice, modified from Plack et al. (2005) (first modification for Cook et al., 2019, second modification by Cook et al., 2022).

DEPTH = A judgement of the overall level of the written reflection
Category	Description	Breadth elements that contribute to depth category
Non-reflector	No evidence of reflection. No consideration of things that could change. Writing appears ‘mechanical’. Students are not demonstrating RP processes in their writing. Students are typically describing events or emotions. Students may reject possibility of change as a result of describing they are right.	Return and/or Attend.
Emerging-reflector	Students are demonstrating minimal or inconsistent low-level elements of RP process in their writing. For example, students are first describing an event, then examine their learning, such as reflecting on their learning, identifying future plans based on the events described or describing clinical techniques or strategies utilized during the experience.	Return and one other (Reflection on action, Reflection for action, Process).
Reflector	The reflector is attempting to understand, examine or question clinical events and themselves. These students are more consistent in their reflection of their learning and are utilizing more than one element of reflection in their writing whereby they pause to examine their learning past/future, and strategies that supported or could support their clinical experience.	More than one of the following (Reflection on action, Reflection for action, Process), and any other lower level elements may also be present.
Emerging-critical-reflector	Students are using at least two RP breadth elements that are indicative critical reflectors. This involves moving beyond describing the experience and their learning. Instead, students examine the situation or event from a different point of view, make comparison to previous learning or describe things that they identified and attempted to change in the moment and the result of the attempted change.	Inconsistently demonstrating some higher level elements of reflection in writing. Two or more of the following (Reflection in action, Content, Re-evaluation) but not Premise.
Critical reflector	Students who are consistent in their examination of events. Students utilize all of the higher-level RP elements or breadth element of premise is included. The higher-level breadth code ‘premise’ (examination and, questioning and/or possible modification of bias, assumptions or values), is pivotal to the category of ‘critical reflector’ therefore, this alone can move the student into the critical reflector category. (Boud, Keogh, & Walker, 1985; Mezirow, 1991; Plack et al., 2005; Schön, 1987)	Consistently demonstrating higher-level reflective practice elements. Premise or three higher level reflective practice elements (Reflection in action, Content, Re-evaluation).

In summary, the ability to predict clinical competency has been largely investigated through one-off snap-shots of skills or varied types of skills such as GPA, self-rating scales, educator rating scales, portfolios, practical assessments and clinical placements in medical and allied health programmes with varying outcomes (Blackman et al., 2007; Boles, 2018; Chambers, 1993; Gadbury-Amyot et al., 2005; Iliff et al., 2021; Middlemas et al., 2001; Stacey & Whittaker, 2005; Troche & Towson, 2018; Wallenstein et al., 2010). There remains an assumption that clinical competency development is aided or enhanced by RP activities such as WRP, however, further studies are needed to provide empirical evidence. This includes examining the relationship between SLT student clinical competency and development of RP abilities across a clinical programme (Caty et al., 2015; Schön, 1983; Schön, 1987). To our knowledge no such study has explored an association between WRP depth and clinical competency across a speech-language therapy (SLT) clinical programme. Investigation of such an association is a precursor to examination into an ability to predict clinical competency through utilization of WRP activities. As such, this study aims to:

1. Examine the impact of time on depth of WRP for SLT students across and within each year group (first, second and final) of the SLT the clinical programme.

2. Determine whether a relationship exists between depth of WRP and level of clinical competency of SLT students.

Methods

This study received ethical approval from [University Name] Human Ethics Committee, [university location]. All participants provided written consent to participate.

Study context

This cross-sectional and repeated measures design study was conducted as part of the speech-language therapy (SLT) clinical programme at the University of Canterbury, New Zealand (NZ). The students included in this study were all completing a 4-year undergraduate honours degree in SLT. The first year of study does not include clinical education, and years two through four are professionally orientated (renamed first, second and final professional year) and include a focus on clinical placement experiences. Students are eligible to practice as a Speech-language Therapist at the conclusion of the final professional year.

As part of the programme, and reported in Cook et al. (2022), students engaged in a variety of RP activities. Clinical educators (CE) utilized dialogic teaching, class discussions, metacognitive discussions, informal discussions with clinical educators (individual or group) written reflections (informal and assessed) and verbal reflective practice groups (discussion groups with student peers and a clinical educator facilitating professional topics). See Tillard, Cook, Gerhard, Keast, and McAuliffe (2018) for a detailed report on the structure of verbal reflective practice groups for SLT students. Appendix 1 describes the RP clinical education programme followed by the SLT clinical education programme (Cook et al., 2022). Inclusion of weekly written reflections (WR) was usual practice for all clinical courses and part of clinical education learning outcomes (see Appendix 2 for Guiding questions).

This study examined usual practice, therefore for each professional year, students completed two semesters of academic study. This included 12 weeks per semester of clinical experiences in a range of clinical environments and populations including preschool, school-aged children and adults under the guidance of a Clinical Educator (CE). At two points during clinical programme, SLT students engaged in clinical placements described as ‘block placements’. A block placement is a full-time placement (i.e., 40 h per week) with no academic class requirements (McAllister et al., 2013b).

Participants

The study included the same 77 undergraduate students enrolled in clinical courses as part of the Speech-Language Pathology honours programme, and reported in Cook et al. (2022). The average age of the participants was 21.5 years (SD = 3.95) with 75 females and 2 males participating in the study. See Table 2 for participant details by professional year. The study excluded any students who chose to withdraw from a clinical course during the semester.

Table 2. Demographic details of student participants by professional year.

Professional year	Number of students	Male/Female	Average age/SD	Prior clinical experience*
First	26	1 Male/25 Females	20.5 (2.24)	None
Second	29	1 Male/28 Females	21.2 (0.81)	2 semesters
Final	22	22 Females	23 (6.34)	4 semesters

Note: *1 semester is 12 weeks.

Instruments

1. Coding of participants’ WR was initially undertaken to identify breadth of WR present in each participant’s WR using the procedure defined by Plack et al. (2005), modified by Cook et al. (2019) and reported in Cook et al. (2022) (see Appendix 3 for breadth coding framework). The Cook et al. (2022) study utilized and reported on the data gained from the breadth of reflection component. The current study utilized data generated from the depth of reflection component as the outcome measure for this study. Depth of reflection is described as an overall level of reflective practice skill (Hill, Davidson, Theodoros, 2012 ; Plack et al., 2005). Depth of reflection has also been described as an overall measure of reflective competency (Plack et al., 2005). The five depth categories (‘Non-reflector’, ‘Emerging-reflector’, ‘Reflector’, ‘Emerging-critical-reflector’, or ‘Critical reflector’. See Table 1 for definitions) allowed for comparison of category of reflection to level of clinical competency gained as part of the COMPASS^® assessment. For identification of depth of reflection, in keeping with the Cook et al. (2019) methodology, the authors applied a rule-based system where by inclusion of specific breadth of reflection codes informed the condition for depth of reflection (Table 1). For example, a WR with the breath codes ‘return’ and or ‘attend’ only, would fall into the ‘Non-reflector’ depth category. The depth categories were developed and based on the Plack et al. (2005) schema and further defined from readings from Schön (1987), Mezirow (1991), and Boud et al. (1985). As per the 2019 study, modifications to the ‘depth’ categories proposed by Plack et al. (2005) were utilized whereby ‘emerging reflector’ and ‘emerging critical reflector’ were introduced following a review of the reflective practice literature in an attempt to make the instrument more sensitive to subtle changes in depth of student reflection. The description of depth categories and allocation of breadth codes to depth categories are found in Table 1.

2. Competency Assessment in Speech Pathology (COMPASS^®) is a valid and reliable online outcome measure of SLT student performance (McAllister et al., 2013a). COMPASS^® is used in all SLT programmes in Australia, New Zealand and South East Asia as one assessment of clinical competency. For the purpose of this assessment tool, the term competence is defined as an ‘observable competent action’ (McAllister et al., 2013a). As per recommendations by McAllister et al. (2013a), students are required to complete their own ratings for all 11 competencies as a self-evaluation practice and to further inform the CE’s judgement. The student and the CE then meet to discuss the student’s competency and complete the visual analogue scale (VAS) to rate student performance. The 11 competencies are then automatically combined via a scoring system to provide a judgement of overall competency in SLT practice (McAllister et al., 2013a). The automatic scoring system converts the VAS into one of seven categories representing increases in clinical performance, then to interval-level data and finally parametric statistics are completed (McAllister et al., 2013b). The output includes a ‘raw score’ (range 11–77, the sum of the 11 competencies), an ‘overall competency score’ (range 144–835.25), an interval measure expressed as a scaled score and an overall ‘zone of competency’ (ZOC) (range 1–7, an interval measure indicating one of seven developmental ZOC determined by the student’s competency score). For this study, and in keeping with the recommendations from the COMPASS® technical manual, ZOC represents each student’s clinical competency score. ZOC has been reported in past studies examining SLT clinical competency (e.g., Hill et al., 2021; Sheepway et al., 2014). For further reading on the development, validation and statistical properties of COMPASS^® see McAllister et al. (2013b).

Procedure

This study employed the same procedure for the WR data published by Cook et al. (2022). In brief, as part of usual practice students wrote and submitted WR weekly. Students were encouraged to submit their WR within 24 h of the clinical experience they were reflecting on. Guiding questions were utilized to support engagement in the WR process (see Appendix 2 for guiding questions) (Cook et al., 2022). Students received at least two pieces of written feedback per WR from their CE. CE were encouraged to provide feedback based on the process of reflection the student had undertaken and could utilize the coding framework published in Cook et al. (2022) to assist (see Appendix 3 for coding framework). Beyond the encouragement provided to CEs, all CEs had full ownership of amount, frequency and style of formative feedback they chose to provide to student SLT. For this study and Cook et al. (2022) the WR selected for analysis were taken from the start and end of each clinical course for each semester. This totalled four WR for each student across the professional year (reported as T1, T2 (start and end of semester one), T3 and T4 (start and end of semester two)).

With respect to the COMPASS^® instrument, in the final week of semester one and two, as part of usual practice, the end placement COMPASS^® Competency Assessment in Speech Pathology was completed with the participant and their CE. This instrument was administered via a secure website. COMPASS^® data were inspected to check for variation between individual student scores (no variation found) and that ZOC data were available for extraction (in keeping with COMPASS^® no ZOC data would be available for any students where more than three units were not scored). This did not occur for any students. Following this the COMPASS^® ZOC data for each participant was extracted from ACCESS 2013 software. COMPASS^® end placement clinical competency data were available for T2 and T4 (semester one and two) for second and final professional year students and T4 (semester 2) for first professional year students. As part of usual practice, the first clinical course, for the first professional year group, does not utilize the COMPASS^® tool as students are deemed ‘pre novice’.

Data analysis

Each WR was deidentified, given a randomly generated number and then coded for breadth of reflection by trained research assistants, (see Cook et al. (2022) study for full description of training, analysis and reliability). The research assistants had no relationship to the students or CEs. The depth category for each WR was then automatically generated in excel for each participant at each timepoint (T1-4) (Table 1 illustrates how breadth elements contributed to the judgement of depth). A total of 273 of a possible 308 WR were submitted and given a depth category (from 77 participants, across the three-year groups, at four time points—referred to as T1, T2, T3 and T4). Table 3 indicates the number of participants by year group who submitted a WR for each time point. COMPASS^® end placement clinical competency data extracted for this study was the ZOC (1–7 interval level zones). Any fractions of zones were converted to whole numbers for assessment purposes (Hill et al., 2021). WR depth category was matched to COMPASS® ZOC scores for each participant and at each timepoint. For each timepoint, any data that could not be matched was excluded from the analysis. Average scores for the depth categories were reported using descriptive statistics. Visual inspection in the form of both table and figure was completed initially to compare WR depth category for time 2 and 4 to COMPASS^® ZOC score.

Table 3. Number of participants who submitted a written reflection for each time point across the academic year (% of written reflections compared to the expected number).

Professional year	T1	T2	T3	T4
First	26 (100%)	24 (92%)	19 (73%)	18 (69%)
Second	29 (100%)	28 (96%)	27 (93%)	24 (82%)
Final	22 (100%)	20 (91%)	15 (68%)	13 (59%)
Totals	77(100%)	72 (94%)	66 (79%)	55 (71%)

Statistical analysis

To examine the effect of time both within professional year groups (T1–T4) and across professional year groups (First, Second and Final) on WR depth, mixed effects modelling was utilized (Bates, Mächler, Bolker, & Walker, 2015). This was used in keeping with the Cook et al. (2022) analysis. Analysis was undertaken in R (R Core Team, 2015), using the add-on packages lme4 (Bates et al., 2015) and ordinal (Christensen, 2015). A mixed-effects cumulative link logistic regression model for ordered categories was utilized. The aim was to predict the probability of participants to fall into ordered depth of reflection categories both across and within year groups (with alpha at 0.5). Slope parameters were included to model the change in probability over time for being allocated into a specific reflection depths category. Participant-specific changes in slopes were included in the model as normally distributed random effects.

To determine if a relationship (positive or negative) existed between WR depth and ZOC for clinical competency a second mixed-effects cumulative link logistic regression model for ordered categories was utilized. This examined depth of reflection categories both across and within year groups compared to ZOC score for T2 (second and final professional year group) and T4 (all year groups).

Results

A total of 46 participants (60% of possible participants who consented to participate in the study) submitted a WR at each of the four time points (T1, T2, T3, T4). Table 3 provides details of participants, organized by professional year and time point and indicates participant WR submissions. All professional year groups demonstrated participant attrition over time.

The effect of time on depth of written reflective practice across and within SLT year groups

Figure 1 illustrates the proportion of SLT students categorized as Non-reflector, Emerging-reflector, Reflector, Emerging-critical-reflector or Critical reflector and the change within these depth categories across time. No participants were categorized as Emerging-critical-reflector. At any time point, the majority of participants were categorized as Reflectors, with Emerging-reflector as the next largest RP depth category. At the conclusion of the assessment period (T4) the majority of participants in the final professional year group had progressed into the higher categories of Reflector and Critical reflector. To examine the effect of time and clinical experience on depth of reflection category a logistic regression model for ordered categories was run with the fixed effects of time (T1–T4), and professional year (first, second and final) in comparison to the threshold coefficients (Model 1 Table 4). The threshold coefficients are the estimated thresholds for first professional year participant T1 depth categories. Table 4 Model 1 indicates there was a statistically significant association between increasing time and the probability to develop depth of reflection for final professional year students only (β = .66 (.30), z = 2.22, p < .05).

Figure 1. Proportion of participants by professional year demonstrating a category for depth of reflection at that particular time point (time 1–4), with the predicted thresholds between categories (black lines). See Table 2 for a definition of each depth category.

Table 4. Coefficients of the mixed effects model for depth of written reflection with the fixed effects of time and professional year in comparison to the first professional year (Threshold coefficients).

Model	Threshold coefficients & fixed effects	Estimate	Std. Error	Z	Pr(>|z|)
Model 1	Threshold coefficients:
	Non-Reflector| Emerging-Reflector	−2.73	0.42	−6.52
	Emerging-Reflector| Reflector	−0.78	0.35	−2.19
	Reflector| Critical-Reflector	2.15	0.40	5.44
	Fixed effects:
	Second professional year	0.42	0.48	0.86	0.39
	Final professional year	−0.59	0.51	−1.16	0.25
	Time (T1–T4)	−0.18	0.19	−0.95	0.34
	Second professional year student development over time compared to first professional year students	−0.14	0.26	−0.54	0.59
	Final year student development over time compared to first professional year students	0.66	0.30	2.22	0.03	*
Model 2	Fixed effects:
	Zone of competency score (ZOC)	0.00	0.01	−0.20	0.84
	Second professional year student depth category over time (T2 and T4) compared to ZOC scores	0.01	0.01	0.56	0.58
	Final year student depth category over time (T2 and T4)compared to ZOC scores.	−0.01	0.01	−1.18	0.24
	Time (T2 and T4) compared to ZOC	0.00	0.00	−0.46	0.65

Note: * p = 0.05.

Determining a relationship between WRP and clinical competency

To determine a relationship between WRP and clinical competency, depth of reflection data from SLT student participants in time 2 and time 4 were extracted and compared to the COMPASS^® clinical competency data (ZOC) at the same time points. Table 5 presents SLT student participants by professional year group for whom both depth of reflection category and ZOC was available for time 2 (43 students) and 4 (50 students). This excluded ZOC data for time 2 for first professional year students as no COMPASS^® data were collected for this group as part of usual practice. Participant attrition is seen over time. It can be seen that over time as a group, students in the second professional and final year groups demonstrate increasing ZOC median.

Table 5. Number (percentage) of participants in each zone of competency (ZOC) by professional year group and timepoint (time two (T2) and time four (T4)).

ZOC	First professional year (T4)	Second professional year (T2)	Second professional year (T4)	Final professional year (T2)	Final professional year (T4)
1	6 (35.3)	3 (12.5)
2	11 (64.7)	14 (58.3)	6 (28.6)
3		5 (20.8)	11 (52.4)	3 (15.8)
4		2 (8.3)	4 (19)	7 (36.8)
5				5 (26.3)
6				4 (21.1)
7					12 (100)
ZOC Median	2	2	3	4	7
ZOC Range	1–2	1–4	2–4	3–6	7
Number of participants*	17 (65%)	24 (89%)	21 (72%)	19 (86%)	12 (54%)

*number and percentage of participants for whom both written reflections and COMPASS^® were available for each timepoint compared to the original number of participants.

Figure 2 illustrates all SLT students by professional year group, their category for depth for of reflection and their ZOC as taken from the COMPASS® instrument (time 2 and time 4). To examine a relationship between depth category and ZOC visual inspection of Figure 2 was first completed and indicated that there was no association between WRP and ZOC as time and clinical experience increased. Students judged as having higher depth of WRP categories (e.g., Emerging-critical-reflector or Critical reflector) did not have a higher ZOC. A second logistic regression model for ordered categories was run with the fixed effects of time (Model 2 Table 4). Model 2 confirmed no significant effect can be found between change in depth category and ZOC as time and clinical experience (across or within professional year group) increased. An interpretation of this finding is that both year and ZOC share the effect. Figure 2 instead illustrates that professional year and ZOC could be intertwined, in that, it was not clear if SLT student year of experience predicted ZOC or ZOC predicts SLT professional year.

Figure 2. Number of participants by professional year group that demonstrated a category for depth of reflection at that particular time point (T2/end of semester 1 or T4/end of semester 2) and their zone of competency (1–7). See Table 1 for a definition of depth categories.

Discussion

The purposes of this study were to: (1) examine the impact of time on depth of WRP for SLT students across and within each year group (first, second and final) of the SLT the clinical programme; and (2). Determine whether a relationship exists between depth of WRP and level of clinical competency of SLT students. The results indicated that the majority of SLT students fell into the depth category of Reflector. Category of depth of reflection increased as clinical experience increased for final professional year SLT students. First and second professional year SLT students were variable in their demonstration of depth of WRP across and within the clinical programme. No relationship was indicated for depth of WRP and clinical competency across and within the clinical programme. The findings are discussed with implications for clinical education, limitations and future research.

Depth of written reflective practice as clinical experience increases

This study found that at any one point in time the majority of SLT students in the study were categorized as Reflectors for depth of WRP. This finding is consistent with past studies of SLT and Physical therapy students’ depth of WRP category (Cook et al., 2019; Hill et al., 2012; Plack et al., 2005). While this finding highlights similarities across SLT student groups, it may in fact be related to similar guiding questions being employed for SLT students regardless of professional year group. It may be that, the tailoring of guiding questions for WRP to specific professional year groups rather than placement type yields a different outcome. Better tailoring of guiding questions, may be useful to offer additional opportunities for observable behaviours of ‘critical reflectors’. Therefore, examination of guiding questions employed for WRP warrants systematic examination in the future.

Over time, final-year SLT students significantly enhanced their depth of WRP category. In comparison, the two professional year groups with less clinical experience (first and second) did not. This finding is in contrast to earlier SLT WRP studies and is discussed in detail below (Cook et al., 2019; Hill et al., 2012). The finding for final year students however, produced similar results to Plack et al.’s (2005) depth of WRP results for a one-off examination of final year physical therapy students. The current study then extends Plack et al.’s (2005) findings by identifying significant positive change over time for final professional year SLT students’ depth of WRP. One reason for this finding may relate to the combination of RP activities, feedback and teaching that was part of usual clinical practice for the SLT students in the current study (see Appendix 2). It may be that students in the final professional year were better able to use their extra knowledge about RP, as well as the feedback provided by CEs about their RP process, as a self-directed learning tool (Dunne et al., 2019). Extending on this, as part of their clinical placement experience, students in the final professional year may have been exposed to and able to reflect on more complex clinical situations compared to first and second professional year students. Complex clinical situations have also been described as a catalyst for RP (Mann, Gordon, & MacLeod, 2009). Finally, this significant finding may highlight final professional year students’ progression towards workplace readiness and demonstration of life-long learning practices, where they continue to engage in learning and personal development activities such as RP, beyond formal education settings.

First and second professional year SLT students were more variable in their depth category over time. This finding is in keeping with descriptions of variability for reports of clinical competency and another mixed methods investigation into WRP for SLT students, for the second professional year SLT students (Dunne et al., 2019; McAllister et al., 2013b). However, it was unexpected for first professional year students, who one might expect to begin at the non-reflector category and progress through emerging reflector categories . One reason for these unexpected findings for both groups of students again likely relates to the focus on usual practice for this study. Usual practice meant the clinical placements were varied. Placements varied by population (e.g., adult, child or mixed), environment (e.g., school, hospital or private practice settings), and time (e.g., part-time placement and block placement) within and across professional year groups. This finding may indicate that while in the earlier stages of the clinical programme, consistency of clinical placement, may allow students to better demonstrate higher depth of RP as seen in earlier studies of SLT students WRP abilities in the first and second professional year that used a consistent clinical placement experience (Cook et al., 2019; Hill et al., 2012). Consistency of clinical placement may allow students to: (1) build on repeated experiences, (2) receive formative feedback from the same CE and (3) discuss the impact of the repeated experiences on their learning in WRP in more depth over time. Studies describing simulation activities and strategies for supporting culturally and linguistically diverse SLT students, have identified similar benefits as a result of repeated practice or consistency of clinical placement. The benefits included enhanced clinical skill and confidence for SLT students (Attrill, Lincoln, & McAllister, 2015; Becker, Rose, Berg, Park, & Shatzer, 2006; Penman, Hill, Hewat, & Scarinci, 2021). A second reason for these findings may be that the variability indicates each student’s unique background, past life experiences and past experiences with WRP. This would be a useful avenue for future research.

The relationship between WRP and clinical competency

No association was detected between depth of WRP and clinical competency. Therefore, there was no ability to predict clinical competency based on depth of WRP. Instead, clinical competency appeared to be associated to professional year group. The findings contrast with prior medical and allied health studies that suggested positive relationships with RP and competency, and a positive impact on future clinical practice (Aronson et al., 2012; Caty et al., 2015; Chabeli, 2010; Cook et al., 2019; Halton et al., 2007; Ng et al., 2012; Plack et al., 2005). This finding may again be due to the RP guiding questions enlisted to guide student WRP. Use of these guiding questions aimed to promote reflective questioning, however, the questions may in fact be restricting some students or offer limited opportunities to demonstrate higher-level reflective skills and therefore any relationship between WRP and clinical competency (Tillard et al., 2018). Additionally, the finding of clinical competency appearing to be associated to professional year group may be due to CE being aware of the general guide listed for both SLT students and CE in each course outline that stated student competency by the end of the course e.g., ‘students will be approaching intermediate level’.

Given no relationship was found between WRP and clinical competency, the results of the current study, further reinforce that no one assessment tool (such as an overall measure of WRP skill) can be used to predict future clinical competency outcomes. This finding now adds WRP to the body of literature in clinical education where possible relationships between clinical competency, GPA and self-assessment had been examined in order to attempt to predict future clinical competency, or streamline admissions to clinical programmes (Blackman et al., 2007; Boles, 2018; Chambers, 1993; Gadbury-Amyot et al., 2005; Hardy et al., 2021; Iliff et al., 2021; Middlemas et al., 2001; Stacey & Whittaker, 2005; Troche & Towson, 2018; Wallenstein et al., 2010).

Implication for clinical education

The findings of this study serve to further inform the use of WRP as part of a clinical education programme of learning for SLT students. Judgements of either depth or breadth of reflection (as reported in Cook et al., 2022; Hill et al., 2012; Plack et al. 2005) continue to keep the focus of the judgement on the process undertaken for RP rather than the content covered in reflection. This practice aims to reduce student report of feeling unsafe and students’ writing what they perceive educators want them to write to gain a high grade (Bourner, 2003; Cook et al., 2019; Dunne et al., 2019; Plack et al., 2005).

While the results of this study suggest that the depth of WRP classification system cannot be used to predict student clinical competency as measured by COMPASS^®, WRP could instead be used as an example of observable behaviour to inform competency ratings such as COMPASS^® (Boud, 1995; Levett-Jones, 2005; Musolino, 2006). A core requirement of COMPASS^® is to only rate observable behaviours, yet, RP is sometimes described as difficult to visualize (McAllister, 2005; McAllister et al., 2013a).

Limitations and future research

A number of limitations and opportunities for future research avenues are apparent as a result of this study. Firstly, the reorganization of depth categories utilizing a rule-based system implemented by Cook et al. (2019) requires further examination. The depth judgement utilized here requires assessment of breadth elements only, rather than allowing raters to make their own subjective depth judgement, as was originally intended by Plack et al. (2005). Examination of agreement between the rule-based system and inter-rater judgement is a useful next step, if educators want to use the depth categories only, for one-off assessments of RP. Additionally, in this study no students were found to be ‘emerging-critical-reflectors’, a category also introduced by Cook et al. (2019) to make the depth instrument more sensitive to SLP students developing RP skills and to map to an academic system of grading (Kember, Mckay, Sinclair, & Kam Yuet Wong, 2008). If the goal really is to credit students who are ‘emerging’ in demonstrative of RP abilities then re-examination of the description and criteria of what makes an Emerging-critical-reflector is recommended. Secondly, attrition is seen over time in all year groups in the WRP component (100% to 66%), and when matching participants to their COMPASS^® competency scores for both time points (50% of possible participants). This was not unexpected, as attrition in repeated measures designs are expected (Pan & Zhan, 2020). However, this may be a source of bias in over interpreting the results (Dumville, Torgerson, & Hewitt, 2006). Further studies examining RP and clinical competency over time, could follow a longitudinal approach to focus on examining individual participant characteristics in more detail at each time point in data collection (Dumville et al., 2006). Additionally, future research could examine what makes a question reflective? What questions are better posed to final-year students? Or, what questions are useful for all students to address? This could offer educators and students a list of questions to choose from for both WR and reflective discussions, as well as different questions based on SLT student clinical experience.

Conclusion

This study found a positive association with time for demonstration of depth of WRP skills for final year SLT students. First and second professional year students were variable in their demonstration of depth of WRP across time within an academic year. No relationship was seen between depth of WRP and clinical competency across the degree programme. Examination of student depth of WRP maybe useful as a one-off judgement of reflective skill to continue to place the focus of grading on the process undertaken rather than the content of WRP activities. Additionally, judgements of depth of WRP may be useful as an analysis that can contribute to judgements made about student RP abilities as part of the COMPASS^® assessment tool and other clinical competency tools.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Appendices

Appendix 1. Clinical education program of learning for reflective practice by year group and semester (x indicates type of RP activity completed) (as published in Cook et al. (2022))

Table

Professional year	First	First	Second	Second	Third/ Final	Third/ Final
Semester	One	Two	One	Two	One	Two
Reflective practice (RP) education
Full class teaching: group reflections (what went well, what was surprising, what would you do next time), dialogic teaching, Journal article discussion: topic RP	X	X	X		X
Mentoring / Peer learning		Mentee				Mentor
Reflective discussions with Clinical Educator pre and/or post clinical interactions (small group or one-on-one)		X	X	X	X	X
Verbal RP group. 1 per week. 50 min duration				X		X
Written RP. 1 per week. Formative feedback given		X		X	X	X
Written RP: Assessment. Summative feedback given.	X	X	X	X	X	X
Type and sequence of Questions utilized for written RP*	Set 1	Set 2	Set 2, 3	Set 2	Set 2	Set 2, 3
Clinical practice requirements**
Observation placement	12 weeks
Part time placement		12 weeks	6 weeks	12 weeks	12 weeks	6 weeks
Block placement: full time			5 weeks			12 weeks

* See Appendix 3 for full list of questions utilized as part of standard practice for the clinical education program.

** An observation placement is one whereby students are not actively involved in SLT, a part time placement is completed in conjunction with academic teaching requirements, a block placement is a full time placement (i.e., 40 h per week) with no academic teaching requirements (McAllister et al., 2013b).

Appendix 2: Guiding questions utilized for WRP (As published in Cook et al. (2022)

Table

Set 1	Set 2	Set 3
What happened? Brief summary of what you did, what you talked about, any new experiences What did not go well and why? Explain what you learned from this What next? What have you learned from this visit and how will this experience influence your future interactions? Think about specific actions and how you are going to put them into practice, describe any resources you might need	What was your overall impression of the session?* What things went well during the session and what did you learn from these?* What things went wrong during the session and what did you learn from these?* What emotions can you remember feeling during the session?* What did you observe or think about client emotions or behaviours during the session? Did the session follow your plan? Why or why not?* What theoretical knowledge did you use or could have used during this session?* What past experiences did you use or could have used during this session?* What do you need to learn or find out about before the next session?* What impact do you feel your own assumptions, values, beliefs or biases may have had on the session or observation?** *McAllister and Lincoln (2004) ** Plack et al. (2005)	Account of the incident What happened, where and when; who was involved? What was your role/involvement in the incident? What was the context of this incident, e.g., previous involvement of yourself or workplace staff with this client/client group? What was the purpose and focus of your contact/intervention at this point? Initial response to the incident What were your thoughts and feelings at the time of the incident? What were the responses of the other key people to this incident? If not known, what do you think they might have been? Issues and dilemmas highlighted by this incident What practice dilemmas were identified as a result of this incident? What are the values and ethical issues which are highlighted by this incident? Are there any implications for your collaborations with any of the following? Clients, Their family members, Peers, Supervisors, SLT clinicians, Inter-disciplinary team members Learning What have you learned e.g., about yourself, relationship with others, the SLT task, organizational policies and procedures? What theory (or theories) has (or might have) helped develop your understanding about some aspect of this incident? What future learning needs have you identified as a result of this incident? How might this be achieved? Outcome What were the outcomes for the various participants? Are there ways in which this incident has led (or might lead to) changes in how you think, feel or act in particular situations? What are your thoughts and feelings now about this incident? (Crisp, Lister, & Dutton2005, )

Appendix 3. Coding schema for written reflective practice, modified from Plack et al. (2005) (first modification for Cook et al., 2019, second modification by Cook et al. (2022)

Table

1.	Breadth = made up of reflective practice elements that can be identified at the word, sentence or paragraph level. Elements are organized by level of RP with 1 = lowest level of RP.
No.	Element (code)	Description of breadth elements	Key words/Phrases	Sentence example	What it is not	Co-occurring elements	Guiding question where element may be identified
1	Returns to the experience (RETURN)	Describes the experience. Narrative retell/replay of the session. (‘in some detail’ Plack et al., 2005.)	‘Today’s session’	‘Today we did x, y, z today’	Listing items with no description of event	All can co-occur	What was your overall impression of the session?
2	Attends to feelings (ATTEND)	Acknowledges and begins to work with feelings. Needs to do more than state an emotion or feeling –give the why – some discussion of impact of emotion on situation (Plack et al., 2005).	Emotions e.g., nervous, sad, happy, excited, Feelings – Tired, shy, interest	‘I felt sad because … ’ ‘I felt nervous because … ’	‘I was nervous for this session’ – no why given.	All can co-occur	What emotions can you remember feeling during the session? Did you observe or think about client emotions or behaviours during the session? What were your thoughts and feelings at the time of the incident?
3	Reflection on action (ROA)	Occurs after the action has been completed.	"Learnt/learning"	‘I did x,y,z and from this I learnt … ’	Describing the event Or Only describing what learnt (with no description of the event)	Must co-occur with either: return, attend	What things went well during the session and what did you learn from these? What things went wrong during the session and what did you learn from these? What have you learned e.g., about yourself, relationship with others, the SLT task, organizational policies and procedures?
4	Reflection for action (RFA)	Occurs before being faced with the situation; begins to plan for the future.	‘Next time/session’ ‘I should have’ ‘in the future’ ‘I will’	‘I did x, y, z and next time I will … ’	Describing the event Or Only describing what will do next time (with no description of the event)	Must co-occur with either: return, attend	What things went well during the session and what did you learn from these? What things went wrong during the session and what did you learn from these? What do you need to learn or find out about before the next session? What future learning needs have you identified as a result of this incident? How might this be achieved?
5	Process (PROC)	Describes the strategies/clinical techniques used or available for use and the impact/ or reason for use.	Cues or strategies e.g., stopwatch, cheat sheet, role play, questioning techniques ‘I tried’ ‘I could have tried’ ‘reminded’	‘I used x cues to … ’ ‘I reminded the patient to use his loud voice’ ‘I could have used my information sheet to … ’	Listing strategies/clinical techniques with no explanation of the why/what used for. General mention of ‘strategies’ or ‘processes’ - need to be specific.	Can co-occur with all other codes. This is a sig. correlation between premise and process co-occurring.	What things went wrong during the session and what did you learn from these? Did you observe or think about client emotions or behaviours during the session? Did the session follow your plan? Why or why not? What do you need to learn or find out about before the next session? What are the areas you feel you need to develop further about yourself and your communication?
6	Reflection in action (RIA)	Occurs while in the midst of an action; on the spot decisions or experiment. The impact of the change will be described. Distinguishing feature: the immediate significance for action. Ask yourself: did they make a difference to the situation at hand? (Schön, 1987).	‘During’ ‘Change’ ‘In the middle/ moment’ ‘and this meant:	‘In the moment I changed … ’ ‘I decided to change xx during the session as … or … the result was’	Describing what learnt (ROA) Describing what to do next time (RFA)	Must co-occur with either: Return, attend	What things went wrong during the session and what did you learn from these? What things went well during the session and what did you learn from these?
7	Content (CON)	Explores the experience from another perspective (beyond description) – for example patient, client, family or supervisor. New understanding of an event. (Updated from Plack et al., 2005, similar to Hill et al., 2012 definition.)	‘point of view’ ‘different beliefs’	‘Another way I could look at this is … ’ ‘From my client’ssupervisor/ physio’s point of view … ’ ‘I think the patient felt  …  this was because … . The result was/this meant I needed to … *	A statement of emotions/feelings and the impact this had (attend) E.g. ‘I thought about how I would feel if I had 5 students observing me and know that I wouldn’t like it at all!’	Can co-occur: Return likely as they describe the situation Re-evaluate	What things went wrong during the session and what did you learn from these? Did the session follow your plan? Why or why not? What are your thoughts and feelings now about this incident? What were the responses of the other key people to this incident? If not known, what do you think they might have been?
8	Re-evaluates (RE-EVAL)	Reappraises the situation vis-à-vis past experiences. New understanding of an event.	‘last time/patient  …  this time’ ‘I could have  …  this would have’	‘In the past I have done x, I used this again and the result was … ’ ‘I used my previous knowledge from clinical notes at this hospital to help orientate myself on how to set these notes out’.	Stating textbooks, clinical notes, lecture notes e.g., ‘I used my lecture notes to help me know what to expect’ – needs to be specific and give the comparison from learning to current experience.	Can co-occur: Return likely as they describe the situation Content.	What theoretical knowledge did you use or could have used during this session? What past experiences did you use or could have used during this session? Are there ways in which this incident has led (or might lead to) changes in how you think, feel or act in particular situations? What are your thoughts and feelings now about this incident? What theory (or theories) has (or might have) helped develop your understanding about some aspect of this incident?
9	Premise (PREM)	Recognizes and explores own assumptions, values, beliefs and biases. New understanding of an event. So what, now what?	‘my opinion’ ‘my family values’ ‘assumption’ ‘belief’ ‘before I met the patient I thought  …  instead"	‘I had thought all clients/parents would want to … As this is the way I would do it … however now I can see another side/ understand why this is important for them which is … ’	Description of the assumption/value/bias/ belief given – no change/confirmation of perspective given	Can co-occur: Return likely as they describe the situation. This is a sig. correlation between premise and process co-occurring.	What emotions can you remember feeling during the session? What impact do you feel your own assumptions, values, beliefs or biases may have had on the session or observation? What are your thoughts and feelings now about this incident? What are the values and ethical issues which are highlighted by this incident?