Discussing the Case for Use of Practically Assessed Structured Scenarios (PASS) as a Mode of Assessment in Physical Education Teacher Education and Sport Coaching Programs

Abstract

Historically, written and oral assessments such as essays and group presentations have been the predominant modes of assessment used to assess pre-service physical education students’ and sport coaching students’ teaching and coaching knowledge the pedagogical content knowledge of preservice physical education students and sport coaching students. Yet, at a time when the call for more practice-based teacher education is commanding more and more attention in the literature, key consideration should be given to the authenticity of assessment modes being used. Thus, one particular mode of assessment that could be considered is use of Practically Assessed Structured Scenarios (PASS). This article presents a four-way conversation conducted by university educators from four different countries who discuss the benefits and challenges of using PASS as a mode of assessment within physical education teacher education (PETE) and sport coaching education programs. An example of how a PASS assessment session might be organised is also included.

Historically, written assessments in the form of essays, unit outlines, question–response examinations and observation reports, and oral assessments in the form of group presentations, individual vivas and round-table discussions have been used to assess preservice physical education (PE) students’ and sport coaching students’ teaching and coaching knowledge (Capel et al., 2009; Semiz & Ince, 2012). Yet, at a time when the call for more practice-based teacher education is getting louder (Ward et al., 2020), we believe that further consideration should be given to the authenticity of assessment modes being used to develop and determine preservice PE students’ and sport coaching students’ understanding of the students they teach, the learning environments they are responsible for, and the pedagogical choices they are making.

One particular mode of assessment that could be considered is practically assessed structured scenarios (PASS). This predominantly spoken word mode of assessment is similar to objective structured clinical examinations used in medical and veterinary education (Kirton & Kravitz, 2011) in that it utilizes practical scenarios to test practical knowledge. This article presents comments from a four-way professional conversation conducted by university educators from four different countries who discussed the suitability and/or implementation of PASS as a mode of assessment within physical education teacher education (PETE) and sport coaching education programs. Thus, the purpose of this article is to introduce and develop readers’ understanding of PASS and provide access to university educators’ views on the benefits and challenges of PASS implementation. Furthermore, it is intended that discussion within the article will stimulate debate among teacher educators on the range of assessments currently offered within PETE and sport coaching programs around the world.

Assessment in Sport Coach Education and PETE Programs

Over the past decade there has been increasing commentary by academics on the need for assessment literacy development for preservice PE teachers and sport coaches (Dinan-Thompson, 2013; Dinan-Thompson & Penney, 2015; Hay & Penney, 2013; Hurley, 2018). Across PETE programs, assessment is often viewed as “a particularly troublesome process” given the practical nature of the subject (Lorente-Catalán & Kirk, 2016, p. 66) and drives the perceived need for teachers of PE to receive additional assessment support (MacPhail & Halbert, 2010). With assessment practices in PE remaining relatively underdeveloped and arguably “off-trend” (Starck, 2018; Ward & Cho, 2020), investigating whether and how tertiary PE students and sport coaching students learn about assessment continues to be of importance to PETE and sport coaching program faculty (Lorente-Catalán & Kirk, 2016). Moreover, there still appears to be a dearth of published research exploring alternative forms of assessment in PETE programs (Lorente-Catalán & Kirk, 2014).

In addition, commentary on modes of assessment being utilized within sport coach education and development programs reveals a similar trend. This includes inadequate attention being given to the quality of assessment practices in coach accreditation programs (Hay et al., 2012) and a questioning of the purpose of practical assessments that require participation to imitate the behaviors and practices of course leaders (Nash & Sproule, 2012). Furthermore, Hay et al. (2012, p. 192) have suggested that the prevalent use of certain assessment modes “such as tests or quizzes provide little, if any, indication of the breadth and integration of necessary coaching competencies.”

Hay and Penney (2009, p. 391) stated that for assessment to work optimally in PE and sport coach education contexts, there should be a prevalence of “authentic assessment concerned with the relationships between learning content and contexts.” Aligned with this view of the need for authentic assessment opportunities are many of the policies that influence PETE and sport coaching education assessment design. For example, assessment requirements prescribed in the United Kingdom’s Subject Benchmark Statements (Quality Assurance Agency, 2016, p. 14) support “assessment of live practice or simulations”; in Australia, the Australian Institute for Teaching and School Leadership (a national agency committed to the quality assurance of all accredited initial teacher education programs) has stated that their standards and procedures are not intended to “constrain the ability of initial teacher education providers to be innovative” (Australian Institute of Teaching & School Leadership, 2016, p. 3); and in the United States, Standard 5a of the SHAPE America Initial Physical Education Teacher Education Standards (SHAPE America – Society of Health and Physical Educators, 2017) refers to preservice candidates being able to select or create authentic, formal assessments that measure student attainment of short and long-term objective. Arguably, the development of such a skill dictates that preservice candidates be exposed to a range of “authentic, formal assessments” themselves to help develop their craft.

In addition to changes in educational policy relating to the structure and identity of PETE and sport coach education programs, there have been widespread changes in areas such as practice expectations, employment agendas and instructional discourse (Olson et al., 2017). For example, Kirk et al. (1997) raised the issue of PETE programs having at their core a focus on studies in biophysical sciences (e.g., exercise physiology, biomechanics, human anatomy). Indeed, PETE programs were (and still are) seen as a “professional or vocational application of human movement studies” (Kirk et al., 1997, p. 295). In recent years, however, Cliff (2012) and Olson et al. (2017) have observed a decrease in the number of sport science units in undergraduate PETE degrees and an increased number of social science– and pedagogy-focused units. One of the legacies of this earlier focus on sports science, however, is the continued use of assessment modes typically associated with the study of biophysical sciences (e.g. written reports, multiple-choice exams). That is not to say that innovative modes of assessment in PETE and sport coaching education programs are not being used. For example, Meldrum (2011) described the use of scenario presentations (e.g., role-plays, school meetings and interviews), Pedersen et al. (2014) highlighted the use of responses to a case study vignette and annotated bibliographies, Araya et al. (2015) articulated how they used authentic scenarios (such as inheriting a new team) and observation reports, and Calderón et al. (2021) explored the use of blogs and debates as modes of assessment within blended learning environments. However, when coupled with the prevalence of essay writing as a default mode of assessment in higher education (Brown, 2011), the positioning of academic writing as being more valued than practical abilities (Johnson, 2013), and the traditional alignment of written assessment modes with specific PE subject areas (Larsson et al., 2018), the range of authentic assessment modes used in PETE and sport coaching programs appears limited. Lorente-Catalán and Kirk (2013, p. 79) offered a reminder of why exposure to innovative, authentic modes of assessment should be more commonplace in PETE and sport coaching programs: “If PE teachers are to be competent users of alternative assessment in schools, then it is of crucial importance that they are able to experience such practices as part of their professional preparation.”

Practically Assessed Structured Scenarios

Adapted from objective structured clinical examinations used in medical and veterinary education (Kirton & Kravitz, 2011), PASS is predominantly a spoken word mode of assessment, authentic in its use of practical scenarios to test practical knowledge. When students engage in a PASS assessment, they move through a series of pre-prepared assessment scenarios, either individually or in a group. Scenarios are designed to examine understanding of learned elements throughout a program, from curriculum content and teaching and learning context to the application of health and safety protocols. A typical PASS assessment may include the following elements:

1. Individual responses/group discussion offered in response to pedagogical or content focused questions

2. Group responses to a range of verbal, written or visual cues designed to assess teaching and/or coaching proficiency

3. Individual verbal response outlining key observations made of practical performance, still or video images relating to specific teaching and learning scenarios.

A Dynamic Assessment. PASS is largely based on principles of dynamic assessment—an alternative assessment that attempts to measure learning potential and is often used in conjunction with standardized tests (Caffrey et al., 2008). Alony and Kozulin (2007) noted that dynamic assessment lends itself to assessment of “fluid” abilities (i.e., those in a state of change or varying in the way they are used and applied), rather than assessing “crystallized” abilities that represent an outcome of learning or acquisition. Such an approach is arguably well suited to assessing the skills, abilities and understanding that closely align with PETE and sport coach education programs.

A Development-Focused Assessment. The aim of PASS is to assess understanding and application as well as potential for further development, rather than a static level of achievement, and “it does this by prompting, cueing or mediating within the assessment, and evaluating the enhanced performance” (Barjesteh & Niknezhad, 2013, p. 527). Therefore, in alignment with Vygotsky’s (1978) concept of the zone of proximal development, PASS can capture evidence of higher levels of development than that which would be determined by an entirely independent performance and, in so doing, provides both students and staff with a basis for optimizing growth as well as planning for further development.

Consequently, the assessment process itself, rather than being an “end point,” provides further learning opportunities for students, thus reflecting an assessment for learning approach. Here, the assessment process is designed to identify strengths as well as next steps as opposed to exposing deficiencies in learning. This dynamic and collaborative assessment process enables students to demonstrate higher levels of understanding, being guided by the assessor to make links and draw inferences that they might not otherwise do. Moreover, when PASS involves assessment in groups, the collaborative nature of the experience provides opportunities for students to “pool” their expertise and therefore achieve better marks than they might otherwise have done alone. They have opportunities to check their understanding with each other and build on or extend what their peers can contribute. For some students this is a welcome positive alongside other types of group assessment (e.g., oral presentations), in which they often perceive that the performance of others has negatively affected their own (group) mark.

An Adaptable Assessment. Table 1 provides an example of how a PASS session might be organized. In the example provided, four 60-min assessment sessions are organized for four separate cohorts (1a, 1b, 2a, 2b). Each cohort contains three separate groups of up to eight students. Groups rotate around three separate assessment stations, with the fourth assessment station visited by all students simultaneously. This arrangement allows for two staff members to conduct an entire cohort’s assessment session (e.g., one staff member supervising the group demo on one-half of a basketball court, with the other staff member supervising both the group discussion and individual written task). The PASS session outlined in Table 1 was designed for a first-year undergraduate games and activities module with content having both a theoretical component (e.g., classification of games, safeguarding of the learning environment) and a practical/applied component (e.g., modifying elements of a game, adapting instructional pedagogies to improve performance). By catering to both individual and group assessment needs, PASS can be adapted to assess a range of practice and knowledge-based learning outcomes.

Table 1. Example Organization of a PASS Assessment Session

Groups, Staff and Equip	Date and Time	Station 1 Group Demo 10 min	Station 2 Individual (Written) 10 min	Station 3 Group Discussion 10 min	Station 4 All Individual (Written) 10 min
Group: 1a Staff: ×2 Equip:	Tuesday 2–2:50 p.m.	Invasion games Devise and demonstrate a learning activity focusing on pupils’ development of these three defensive elements of game play in European handball: Defending space Closing down space Defensive guarding	Elements of gameplay Complete Games Tree (Hastie, 2010)	Dance and gymnastics What is Laban’s analysis of movement, and how can it be used to develop creativity in dance and gym?	Invasion games Identify and discuss the difference between two goal orientations synonymous with pupil motivation in PE.
Group: 1b Staff: ×2 Equip:	Tuesday 3–3:50 p.m.	Net/wall games Using the equipment provided, devise and demonstrate a (net/wall) activity that focuses on helping pupils address the generic tactical problem of setting up for attack.	Invasion games Identify the offensive elements of game play relating to European handball.	Elements of gameplay Discuss the place of competition in school PE.	Dance and gymnastics What is “creativity”? Explain two teaching approaches related to creativity in PE and give some practical examples from gym and/or dance to support your views.
Group: 2a Staff: ×2 Equip:	Tuesday 4–4:50 p.m.	Dance and gymnastics Listen to the piece of music and demonstrate how you can use this as a stimulus for “teaching for creativity.” Consider the teaching styles that you could use. OR Using a bench and mats, demonstrate how you can develop a creative response in gymnastics using the divergent style of teaching.	Invasion games Object of play: Indicate which objects of play align with each of the games classifications.	Elements of gameplay How might an understanding of developmental phases of gameplay assist a teacher’s session planning?	Invasion games Suggest an appropriate order for the following game progressions: Games to a target Games to a target zone Possession games Games to a target player Justify why you have suggested games progress in this order.
Group: 2b Staff: ×2 Equip:	Tuesday 5–5:50 p.m.	Adapted PE Using the equipment and space provided, devise and demonstrate an activity that helps develop a beginner player’s understanding of five key rules associated with goalball.	Striking/fielding games What are four main knowledge domains of games teachers and coaches? Describe a gameplay situation within a striking/fielding game that requires sacrificial decision making.	Dance and gymnastics Creativity is often associated with dance and gymnastic activities. Discuss how creativity can be developed in other areas of the PE national curriculum.	Invasion games Devise a list of positive and negative aspects of invasion game competition involvement.

A Collective Desire to Develop Our Craft

Considering the importance of both authentic assessment in teacher education programs and our professional development responsibilities as a global faculty of teacher educators, the authors of this article engaged in a 6-month asynchronous professional conversation exploring the utility of PASS usage in their respective PETE and/or sport coaching education programs. A professional conversations model was used as a guiding framework for conversation within which a series of predetermined prompt questions were set to steer discussion (Leonard, 2012). For a detailed overview of the participatory research method used to capture authors’ experiences and perceptions of PASS, see Jarrett et al. (2021).

In essence, authors’ involvement in this professional conversation could be attributed to their collective desire to develop their craft as sport pedagogues and teacher/coach educators. All conversation participants had at least 12 years of university teaching experience, with all currently delivering a range of undergraduate and postgraduate PETE and sport coaching programs. Two authors had previous experience of implementing PASS and two had yet to trial PASS in any form. All participants had previously either worked at the same institution or been involved in research collaborations together. When the professional conversation initially began, each participant was affiliated to a university in four separate countries: Australia, the United States, the United Kingdom and Catalonia (Spain). This spread of participants across four different countries was by design, because seeking out international voices helps us to develop our glocal identities as educators, defined as the mixing of global and local values to enhance cultural efficacies, creativity and diversity of thought (Dvir et al., 2019; Jarrett, 2021). Engaging in conversation with educators based abroad also provides a deeper understanding of practice beyond local learning environments.

Benefits and Challenges of Using PASS

The 6-month professional conversation undertaken by participants included the discussion of a range of experienced and perceived benefits and challenges related to PASS implementation. At the end of the conversation period, a review of the conversation transcript was completed by all participants, with key comments relating to the benefits or challenges of PASS usage collectively identified. A key comment was determined when all four authors identified (e.g., highlighted) the comment during their review of the transcript. These key comments are outlined in Tables 2 and 3. In relation to each key comment identified, supplementary commentary from the conversation transcript was included to engage readers in a form of silent conversation, whereby the reader takes on a silent but active role in reflecting on and responding to commentary (e.g., in a verbal or written manner) to bring into focus their own responses to each articulated PASS benefit or challenge. This supplementary commentary was then used to inform the development of key questions designed to further scaffold practitioners’ reflections on PASS-related decision making, planning and implementation.

Table 2. Benefits of Using PASS

Practitioner Benefits	Key Questions
Key comment: “Efficiency of the marking process which is done mainly then and there.”	Have you considered …
“We not only achieve remarkable consistency between assessors, but just as importantly we send very strong messages to students from these assessments about what we, and our profession, values.”	using PASS to showcase a team-teaching approach within sessions to help set learning (and professional) expectations around collegiately and teamwork?
“One option we have is to trial a ‘scaled down’ PASS using just a few scenarios.”	using PASS as a framework for session activity ahead of any formative or summative assessment?
Key comment: “I found PASS assessment sessions to be quite streamlined and effective.”	Have you considered …
“One of my roles was to assess the group discussion question. In essence, I acted as a passive contributor to discussion, there to clarify, invite further comment, or move discussion on. With marks assigned to group work I rarely had to invite comment from a specific individual as group members collectively policed their own inclusive discussion protocols.”	incorporating more discussion-based learning opportunities within content exposure and assessment opportunities?
“I am convinced that assessment should be designed to enhance the learning experience rather than simply as an end point. Thus, I use PASS in some of my modules due to its authenticity and students’ feedback on it being that it is more engaging.”	how content being learnt in your session can be more applied in nature?
Student benefit	Key questions
Key comment: “Our students feel more confident in ‘practical’ situations than they do as writers.”	Have you considered …
“With PASS we can help to tease out what students really know and can do (and don’t know!) and avoid some of the issues which arise in more traditional approaches such as ‘I could have done that or told you that but I didn’t understand the question.’”	how inclusive your assessment structures are currently?
“Using PASS focused on assessing the experiences of everyday teaching in PE.”	how the experience of teaching PE/coaching sport is showcased to your students through their degree programme?
“PASS, through its design, can help bridge the gap between theory and practice by putting practical knowledge at the heart of the learning process.”	how your students develop their identity as coaches of sport/teachers of PE?
“I have noticed that PASS is one of very few assessments which students complete which allows them to draw on previous as well as current experience. Just as importantly, it provides a rare example of where better marks can be gained via group assessment (shared endeavour) as opposed to individual assessment, since it is the combination of contributions which determines the mark.”	how often your students are exposed to assessment for learning practices within their degree program?

Table 3. Challenges of Using PASS

Practitioner Challenges	Key Questions
Key comment: “I see preparation time and personnel as potential challenges.”	Have you considered …
“It usually takes considerably more time to design the tasks, assessment criteria and mark schemes than with assessments which are not practical in nature.”	how you might establish a working group to pilot the use of PASS in order to better understand the time and personnel requirements?
“If most faculty lecturers used PASS or were sensitive to this model of assessment, it would be easier to implement it.”	how you might develop a culture of shared assessment/marking responsibility to minimize single points of assessment/marking failure?
“With questions and scenarios required for a formative PASS, a summative PASS, and a supplementary PASS, I became aware first-hand of the sheer amount of time required to plan and develop a PASS assessment. For me this was a steep learning curve, as was the need to manage expectations of both colleagues and students new to PASS assessment.”	asking the students you teach to develop a batch of questions/scenarios relevant to topic content and utilize these as a foundation for formative PASS assessment practice?
“Other instructors may wonder why I am taking so much time to assess students.”	inviting colleagues to observe a formative PASS assessment?
“I am still struggling with the thought of implementing PASS due to the nature of some of my classes, especially ones in online spaces being extremely short in time (i.e., 7-weeks).”	developing synchronous or asynchronous digital versions of PASS whereby students work collectively (e.g., synchronously in a Microsoft Teams breakout room) to respond to a scenario or discuss a set question?
Key comment: “Some of the challenges with PASS implementation appear to be at sector and institution level.”	Have you considered …
“Scheduling might also become overwhelming.”	garnering support for a PASS-pilot to determine the extent of scheduling challenges?
“The main difficulty is probably the amount of lecturing time we have and the total amount of time we can spend in each subject, especially in the first years.”	the broadcasting of scenarios in lectures (or through an online platform) that have been captured digitally as means to economize students’ exposure to aspects of PASS assessment?
“I am not completely sure that some ‘traditional’ assessment tools are any better or worse than suggested alternatives. Partially because I am not sure about the robustness of empirical evidence.”	trialing use of PASS through a pilot experience which includes a formal evaluation of learning outcome and assessment experience that is more criteria-referenced rather than comparative in nature?
Student challenge	Key Questions
Key comment: “Some challenges occur at a more conceptual level and the need to manage students’ assessment expectations.”	Have you considered …
“Finding the time to introduce PASS to students and getting them engaged with formative PASS opportunities to alleviate their anxiety was challenging with only finite timetabled contact time.”	introducing PASS to students in the first year of their studies as a means to expose them to a “tertiary-level” form of assessment?
“Most of the classes I teach are classroom-based and/or online.”	the use of online breakout rooms, videos of teaching practice, and online collaboration boards (e.g., Scrumblr) as means to engage students in a PASS-related assessment format?
“I do not have ANY group-based grades in ANY of my classes due to the historical strain of dealing with student complaints.”	using the PASS format to structure an assessment whereby practical group engagements are viewed as formative and act as a precursor to more summative individual assessment tasks?
“I am hesitant about asking my students to trust this new mode of assessment without empirical evidence to highlight pros and cons.”	engaging in small-scale research on PASS via a collaborative scholarship of teaching and learning project with students?

Discussion

Perhaps it is somewhat predictable to see that within Table 2 the majority of statements categorized as a benefit came from participants with prior experience of PASS use. Accordingly, those without experience of PASS commented more on the organizational and resource barriers they envision if a change to PASS were made. These organizational and resource barriers should be viewed as contributing factors to the call for assessment capacity building that Dinan-Thompson and Penney (2015), Hay and Penney (2009, 2013) and Lorente-Catalán and Kirk (2016) have previously described.

Of note within Table 3 were participants’ views around reluctance of fellow staff to embrace new modes of assessment. Comments such as “[there] was the need to manage expectations of both colleagues and students new to PASS assessment” and “some of my colleagues do not like the variability and less tightly defined nature of the PASS criteria” allude to the need for relevant institutional leaders to support colleagues and their use of innovative assessment practices such as PASS. Offering such support will no doubt become easier with the commissioning and publication of relevant empirical research and colleagues’ involvement in PASS-related scholarship of teaching activity. It is also important to point out that encouraging and supporting colleagues to experiment with new modes of assessment falls well within PETE assessment and innovation standards previously detailed.

Anecdotally, the list of benefits presented in Table 2 provides some level of justification for PASS to be explored as an alternative mode of assessment that promotes engagement in and assessment of what Ward et al. (2020) described as the core practices of teaching PE and coaching sport. Through recognition of the reported value of authentic assessment practices such as PASS, the collegial practices that are such a large part of core “everyday” PE teaching and sport coaching can potentially be given their deserved attention within PETE and sport coaching programs. What is also important is that any value associated with use of PASS must be recognized by students as well. The comment made that “I do not have ANY group-based grades in ANY of my classes due to the historical strain of dealing with student complaints” indicates an assessment culture in contrast to the opportunities for learning that assessment modes such as PASS can offer.

Conclusion

The professional conversations detailed within this study highlight a number of benefits and challenges relating to PASS integrity and feasibility. With a collective desire to see more authentic modes of assessment used within preservice PE teacher and sport coaching education programs, the authors of this study considered the use of PASS as a viable and authentic method of assessment. Organizational and conceptual challenges to PASS implementation were raised, and it is in response to these challenges that further research into PASS development and implementation can and should be focused.

Disclosure Statement

We confirm that the research commented on in this article received ethical clearance and has not been submitted for review to any other journal/publication. No external funding was received to complete the research.

Additional information

Notes on contributors

Kendall Jarrett

Kendall Jarrett ([email protected]) was an associate professor in learning & teaching at the University of Greenwich in London, UK, during the authorship of this article. He is now a senior lecturer in educational leadership at the University of Chester, UK.

Belinda Cooke

Belinda Cooke is a head of subject in physical education and outdoor education at Leeds Beckett University in Leeds, UK.

Stephen Harvey

Stephen Harvey is professor in sport pedagogy at Ohio University in Athens, OH.

Victor Lopez-Ros

Víctor López-Ros is an associate professor and chair of sport and physical education within the Faculty of Education and Psychology at the University of Girona in Girona, Spain.