Educational programs for learning to observe movement quality in physical therapy: a design-based research approach

ABSTRACT

Introduction: Movement observation is a core aspect in physical therapists’ diagnosis to determine which interventions are adequate to improve functional abilities. The aim of this study was to derive design principles for an educational program for the development of observational skills.

Methods: We used a qualitative approach within a design-based research methodology. In four rounds, 8 physical therapy students, 16 teachers, and 9 practitioners participated in five Nominal Group Technique meetings and six interviews. Meetings and interviews were transcribed verbatim and analyzed using thematic analysis.

Results: We identified three themes, each with several design principles: (1) didactics; (2) professional content; and (3) conditions for optimal learning. We developed a proto-theory underpinned with underlying educational theories.

Conclusions: To learn observational skills, students, facilitated by an experienced teacher, need to take the lead in their own learning process. This might imply a need for additional training for teachers. A realistic context is a precondition for learning; it might be necessary to increase possibilities for observations in clinical contexts or to invest in training for (simulated) patients as participants in education. Further research is needed to test the applicability of the design principles and a proto-theory for other professionals with a focus on observation and analysis of movements.

KEYWORDS:

• Design-based research
• qualitative research
• physical therapy
• professional education
• observational skills

Introduction

A core aspect of diagnosis in rehabilitation is the observation of movements as a basis for interventions to improve functional abilities (Skjaerven, Kristoffersen, and Gard, 2008). During the assessment, a physical therapist observes the quantity and quality of movements. The quantity of movements reflects the acquisition or re-acquisition of new or more complex motor skills (Schmidt and Wrisberg, 2008). The quality of movement gives an impression of how movements are controlled and coordinated (Magill and Anderson, 2014). In this way, movement quality represents the interaction between personal characteristics and experiences, the task difficulty, and the environmental conditions; and it gives insight into the possibilities and potential of the person’s system for reacting or adapting to changing conditions (Skjaerven, Kristoffersen, and Gard, 2008). For the assessment of movement quality which is important for recognizing motor problems, designing and evaluating interventions, and predicting recovery (Knudson and Morrison, 2002; Skjaerven, Kristoffersen, and Gard, 2008) physical therapists must rely on observational skills. The observation process involves gathering, organizing, and giving meaning to visual, auditory, and sensory information obtained while observing the moving person (Knudson and Morrison, 2002; Skjaerven, Kristoffersen, and Gard, 2008). Earlier studies (Janssen et al., 2012; Skjaerven, Kristoffersen, and Gard, 2008) found that how physical therapists describe their observations is not standardized; it varies among therapists, depending on the theoretical constructs used, which precludes comparability and longitudinal evaluation.

Observation is a fundamental skill for physical therapists (Skjaerven, Kristoffersen, and Gard, 2008). Curricula for bachelor students in physical therapy address knowledge and observational skills focused on the measurement and interpretation of the quantity and quality of human movements. However, there is disagreement about the details of what constitutes good clinical observation, its conceptual basis, and how it is learned or developed (Boudreau, Cassell, and Fuks, 2008). General principles for observation were described by Boudreau, Cassell, and Fuks (2008) as follows: observation has a sensory perceptive and cognitive component; observation is distinct from inference and made concrete through description; and observation is goal-oriented, occurs over time, and on different levels. These levels for observation refer to the whole person observed, a body part, the personal or environmental context, behaviors, and interactions, and the characteristics of the observer on emotional and esthetic planes (Berger, 1980; Boudreau, Cassell, and Fuks, 2008). It seems evident that the observer should be considered an influencing factor when teaching and evaluating observational skills (Al-Moteri, Plummer, Cooper, and Symmons, 2015; Jarodzka, Scheiter, Gerjets, and van Gog, 2010) because there is a tendency for perception, interpretation of what is seen, to interfere with observation, and initial observation should be without any judgment (Boudreau, Cassell, and Fuks, 2008; Shapiro, Rucker, and Beck, 2006).

A clear description of how teachers and students in physical therapy perceive learning observational skills and what didactic principles facilitate learning has not been widely investigated (Jarodzka et al., 2012). Teaching observational skills, seems to depend on personal experiences, leading to a variety of information presented and instruction provided, which could cause uncertainty in students. The literature lacks specific educational strategies to support the development of skills for observing movement quality (Al-Moteri, Plummer, Cooper, and Symmons, 2015). Existing literature on observational skills learning mainly focuses on learning in the domain of medical education and nursing (Al-Moteri, Plummer, Cooper, and Symmons, 2015; Berger, 1980; Boudreau, Cassell, and Fuks, 2008; Jarodzka et al., 2012; Jarodzka, Scheiter, Gerjets, and van Gog, 2010; Shapiro, Rucker, and Beck, 2006), not on guiding principles for teaching observational skills in physical therapy. The overall conclusion for the training approaches under study was that bachelor and master students’ observational skill improved in the educational setting, but not in real and complex clinical situations where the incidence of perception failure may increase. In addition to this, currently available research is predominantly limited to learning strategies using static images, as was indicated in the scoping review of Al-Moteri, Plummer, Cooper, and Symmons (2015), while students in physical therapy need to learn to observe movement. The overall aim of our study was to derive design principles for an educational program to develop observational skills for students in physical therapy. In addition, we were interested in developing a proto-theory (i.e. a set of theoretical concepts that guides ongoing development and refinement of the educational design) for educational program design for the development of observational skills.

Method

Design-based research

Design-based research is an important methodology for understanding how, when, and why educational innovations work in practice (Design-Based Research Collective, 2003). The design-based methodology is an accepted qualitative research approach in educational sciences, and it triangulates multiple sources of evidence (Anderson and Shattuck, 2012; Barab and Squire, 2009; Brown, 1992; Design-Based Research Collective, 2003; McKenney and Reeves, 2013; Vanderhoven, Schellens, Vanderlinde, and Valcke, 2016). Wang and Hannafin (2005) described the definition of design-based research as follows: A systematic, but flexible methodology aimed to improve educational practices through iterative analysis, design, development, and implementation, based on collaboration among researchers and practitioners in a real-world setting, and leading to contextually sensitive design principles and theories.

The design-based research methodology was applied throughout our study to derive design principles for an educational program to develop observational skills in physical therapy students. This approach allows for collaboration among students, teachers, practitioners, and researchers, the main stakeholders in our study. Furthermore, the method allows for conducting reflective analyses to iteratively test and refine innovative learning environments as well as to define design principles (Kennedy-Clark, 2013; Reeves, 2006). We followed four sequential steps as stated by Reeves (2006) and modified the sequential steps according to our research question, described in Figure 1.

Figure 1. Design-based research, based on Reeves (2006).

Approach and selection of participants

To collect and derive design principles for an educational program, stakeholders from both education and physical therapeutic practice were asked to participate. We aimed at a maximum of variation in the participants regarding the level of professional development and expertise. Furthermore, we collaborated with various universities of applied sciences in the Netherlands to ensure the integration of different points of view: (1) Bachelor students in physical therapy – they must learn to observe movement quality; (2) Master students in pediatric physical therapy – they have already gained some experience in observing movement quality. However, to observe the quality of movement in pediatrics, students must understand typical motor development to identify atypical movement quality, which creates an extra challenge in mastering observational skills; (3) Teachers of physical therapy who have experience in teaching students to observe movement quality and have knowledge and experience in different teaching strategies; and (4) Practitioners currently working in the field of pediatric physical therapy – they have experience in developing and mastering observational skills.

Bachelor and master students and teachers were invited by mail to participate in our study. Practitioners were verbally informed about and invited to participate in our study during a post-graduate module to which they were invited. All interested participants received oral and written information about our study, explaining the aim and the total time investment of about 1 to 2 h. The study was performed in accordance with the Declaration of Helsinki. This study was submitted to the Ethical Advisory Committee of the Faculty of Health at the HAN University of Applied Sciences, Nijmegen, the Netherlands. The Committee reviewed and discussed our research proposal including the consent form for participants. They approved our study, deeming it exempt from further review (Registration : EACO17.12/90). All participants volunteered to participate, and anonymity and confidentiality were assured. The participants signed informed consent documents.

Data collection

In design-based research, different research methods can be used (Brown, 1992; Herrington, McKenney, Reeves, and Oliver, 2007; McKenney and Reeves, 2013). We used semi-structured interviews and focus group meetings, using the Nominal Group Technique (NGT), which encouraged contributions from everyone (Potter, Gordon, and Hamer, 2004; van de Ven and Delbecq, 1972). For each interview or focus group meeting, new participants were invited, to ensure maximum variety in perspectives, using purposive sampling. The researchers: LD, pediatric physical therapist and educational scientist, and AJ, pediatric physical therapist developed an interview guide; research assistants who were trained in qualitative research conducted the interviews and took notes. Subsequently, the researchers prepared five NGT meetings; the first three meetings were conducted by research assistants and supervised by the principal researcher (LD). The principal researcher (LD) conducted the final NGT meeting; a second researcher (AJ) took notes. Both the interviews and NGT meetings took place in university settings. In our study, data collection and analysis were an ongoing iterative process guided by the steps of design-based research (Anderson and Shattuck, 2012; Reeves, 2006). The aim, methods, and participants for each step are described below (Figure 1).

Step 1: an analysis of needs, beliefs, and experiences regarding observation in practice

The aim was to investigate and analyze the educational needs, beliefs, and experiences, and possible challenges in teaching and learning methods identified and prioritized by participants. A global search for existing literature on learning observational skills was performed prior to designing an NGT meeting. The NGT meeting was pre-structured and lasted for a maximum of 2 h. The NGT meeting (n = 5) included one teacher specialized in neurologic physical therapy, and four bachelor students in physical therapy all in their fourth and final year of training. During this meeting, information was generated by the participants responding to this question: ‘What are the best ways to learn to observe movement quality for bachelor students in physical therapy?’ The group meeting was followed by six semi-structured interviews (n = 6) with teachers of physical therapy from different universities of applied sciences from the middle and south regions of the Netherlands to further investigate the generated needs, beliefs, and experiences in teaching and learning methods for observational skills. The teachers were experts in different domains of clinical practice: musculoskeletal physical therapy (n = 3), neurological physical therapy (n = 2), and sports physical therapy (n = 1). Following the interviews, another NGT meeting was conducted with four teachers of physical therapy (n = 4) from a university in the southeast region of the Netherlands, with the following professional expertise: musculoskeletal physical therapy (n = 3), and neurologic physical therapy (n = 1). Participants responded to the following question: ‘What do you think are critical elements for bachelor students in physical therapy for learning to observe movement quality in the classroom context?’ Through the triangulation of data resources, existing literature, and data from semi-structured interviews and focus groups, a conceptual underpinning and identification of the educational problem were defined at the end of the first step in our study.

Step 2: definition of preliminary design principles toward solutions

The aim of step 2 was to define a preliminary set of design principles to guide the design of the educational program for learning observational skills as a solution to the identified problem. To define a preliminary set of design principles, two separate NGT meetings were conducted together with the different stakeholders. Those NGT meetings enhanced in-depth exploration of both students’ and teachers’ needs, beliefs, and experiences in teaching and learning methods for observational skills. Unclear details about needs, beliefs, and experiences, and differences in perspectives on teaching and learning methods among the participants were explained and discussed in the group during the NGT meeting and combined into categories, without the elimination of ideas. We conducted one NGT meeting with four master students of pediatric physical therapy (n = 4) and another meeting with five teachers at a university of applied sciences in physical therapy (n = 5), which professional expertise in musculoskeletal physical therapy (n = 2), neurologic physical therapy (n = 2), and sports physical therapy (n = 1). The question asked was this: ‘What is the best method to learn to observe quality of movement, and what supporting materials are needed?’ All master students were educated at a university of applied sciences in the middle of the Netherlands, whereas their former bachelor education in physical therapy was conducted at different universities of applied sciences based in the north, middle, and southeast regions of the Netherlands. The university teachers worked at the same university of applied sciences in the southeast region of the Netherlands. A set of preliminary design principles was developed at the end of step 2.

Step 3: first iterative cycle in regard to refinement of design principles

A draft educational program was designed based on the preliminary set of design principles and evaluated in an authentic setting in clinical practice. Nine pediatric physical therapists (n = 9) were trained in observing human movement using the draft educational program in a postgraduate program at a university for pediatric physical therapy. At the end of the program, an NGT meeting was conducted to explore participants’ perceptions of the new educational program. The participants were asked to respond to the following question: ‘What are important elements for you in an educational program to learn how to observe movement quality?’ The draft versions of the manual and the supplemental materials were discussed within the study group and revised to meet the needs of all the different participants at the end of step 3. Subsequently, the preliminary set of design principles was refined and redefined.

Step 4: reflection to produce design principles and proposal for a proto-theory

As the Design-Based Research Collective (2003) noted, it is important that design-based research does not end with designing and testing particular interventions; rather, it should lead to a shareable ‘proto-theory’. This proto-theory aims to support researchers and practitioners in the ongoing development of the educational design (Design-Based Research Collective, 2003). To attain the final goal of our study, we reflected on our overall research procedure, on the results leading to the development of the educational materials, and on the content and use of the materials. During our reflections, we focused on theoretical understanding, which resulted in suggestions for context-specific theoretical design principles. Finally, step 4 was concluded with the design of a proto-theory.

Data analysis

Interviews and NGT meetings were video-recorded and transcribed verbatim. The transcripts were analyzed by LD and TS using ATLAS.ti version 8 (ATLAS.ti Scientific Software Development GmbH, Berlin). For each phase of the analysis process, the six steps of thematic analyses, as described by Braun and Clarke (2006) were followed (Table 1). The transcripts were coded by LD and TS; codes were discussed until consensus was reached. To enhance credibility, IG, who also specializes in qualitative research methods, reviewed the analysis process. The initial codes were grouped and resulted in a data matrix. Potential themes and preliminary design principles emerged from the data through constant comparison of codes and themes. No contradictory codes or themes needed to be accommodated. The names and definitions of themes and their design principles were discussed with the research team until consensus was reached. During the development of the proto-theory, the construction of a model that depicted the interrelation between the design principles helped us in further discussion about underlying educational theories. Finally, a proto-theory was developed that reflects the interrelations between the identified design principles and the existing theories on learning observational skills.

Table 1. Phases of thematic analysis, based on Braun and Clarke (2006)

Phase of thematic analysis	Aim	Result
1. Familiarization with the data	Identifying meaningful units of text relevant to learning and teaching observational skills	Initial ideas were noted down
2. Generating initial codes	Grouping together units of text dealing with the same issue	Provisional definitions were given
3. Searching for themes	Different codes were sorted into potential themes regarding learning and teaching principles in learning the observational skill	Collating all the relevant coded data extracts within the identified themes
4. Reviewing themes (based on potential themes in previous steps in the process of design-based research)	Checking if the themes work in relation to the coded extracts and the entire data-set	Thematic map of the analyses resulting in preliminary design principles for our conceptual model
5. Defining and naming themes	Refining the specifics of each theme and sub-theme	Generating clear definitions and names for each theme for the design principles
6. Producing final recommendations for the proto-theory and report	Final analysis; relating back of analysis to the aim of our project and to literature	A proto-theory with explanation and a justification in the report for our project

Results

The overall aim of our design-based research was to derive design principles for an educational program to develop observational skills for physical therapy students. Our design-based research resulted in the identification of three themes each of which comprised a cluster of design principles which are described below and supported by quotes (labeled by participant’s role [S = Student, MS = Master Student, T = Teacher, P = Practitioner]). To facilitate students’ learning of observational skills, the design of the educational program includes the following: (1) Didactics: How to learn observational skills, referring to different aspects of student learning, including theoretically, and underpinning perspectives on student learning; (2) Professional content: What needs to be mastered, referring to the understanding of human movement and observation; and (3) Conditions for optimal learning: Referring to recommendations for physical therapists that enable good qualitative observations.

Figure 2 shows the interrelation between the themes and the design principles in the development of an educational program for observational skills. Furthermore, a proto-theory (Table 2) was developed that displays the didactic principles with their underlying educational theories. We will start with the description of the themes and design principles and end with the description of the proto-theory.

Table 2. Proto-theory for educational program design to learn observational skills

Design principle	Recommendations for didactic approach	Educational theory
Student in the lead	Building on a sense of urgency for learning knowledge and skills. Designing an improvement plan with personal learning goals, based on feedback.	Self-regulated learning theory (Greene and Azevedo, 2007; Schön, 1983; Zimmerman, 1990; Zimmerman, 2002)
	Students take responsibility for their own learning process.	Self-determination theory (Ryan and Deci, 2000)
Teacher as facilitator	Stimulating critical reflection on performances; reflection-in-action and reflection-on-action.	Self-regulated learning theory (Greene and Azevedo, 2007; Schön, 1983; Zimmerman, 1990; Zimmerman, 2002)
	Fading guidance and control. Stimulating deep questioning among each other. Behaving as a coach, not as a lecturer (e.g. empowering students to take responsibility).	Self-determination theory (Ryan and Deci, 2000)
	Designing a meaningful learning environment with meaningful learning tasks that apply to professional clinical practice. Tailoring instruction or coaching to the actual level of performance. Simulating students to transfer learning experiences to the context of clinical practice.	Social constructivist learning theory (Norman, Bordage, Page, and Keane, 2006)
		Social cognitive learning theory (Bandura, 1986; Bandura and Locke, 2003)
	Designing a learning environment that enables students to identify similarities and differences in various patterns of human movement.	Variation theory (Marton and Säljö, 1976; Marton and Trigwell, 2000)
Peer learning	Working together, explaining ideas to peers, questioning each other’s views.	Social constructivist learning theory (Norman, Bordage, Page, and Keane, 2006)
Assessment and feedback for learning	Developing a critical attitude toward own and others’ performances. Providing performance feedback; emphasizing strengths, challenges, and next steps. Discussing the improvement plan with others.	Self-regulated learning theory (Greene and Azevedo, 2007; Schön, 1983; Zimmerman, 1990; Zimmerman, 2002)
		Social cognitive learning theory (Bandura, 1986; Bandura and Locke, 2003)
		Feedback intervention theory (Kluger and DeNisi, 1996; Kluger and van Dijk, 2010)
Contextual learning	Performing the observational skill individually, by reasoning aloud, and applying in professional context.	Social constructivist learning theory (Norman, Bordage, Page, and Keane, 2006)
Building complexity	Helping students to make the transfer of observed differences and similarities to a new context and with higher complexity. Presenting a variety of problems that reflect clinical practice.	Variation theory (Marton and Säljö, 1976; Marton and Trigwell, 2000)
Educational tools	Enabling the observations of human movement in (real life, or video recorded) patients to support the learning process.	Social constructivist learning theory (Norman, Bordage, Page, and Keane, 2006)

Figure 2. Interrelation between design principles and themes for the development of an educational program.

Didactics: how to learn observational skills

Students, teachers, and practitioners described different ways to facilitate the learning of observational skills. Comments by students and practitioners focused on the process of gathering knowledge, skills, and attitudes from their individual points of view, whereas the lecturers’ focus reflected a broader perspective. This perspective included facilitation by teachers, peer students, and students’ self-regulation. Seven interrelated design principles were defined for this theme.

Students in the lead

Students reported that learning to observe human movement requires a sense of urgency in terms of closing gaps in their knowledge and skills. This perceived urgency gives meaning to learning and promotes involvement and an intrinsic motivation to learn. This can be conceptualized as ‘taking the lead’ or self-regulation.

When I observe a patient with a complex health problem, I experience my shortcomings in knowledge and skills. That motivates me to find out what I actually need. (MS-4)

Students and teachers explained that self-regulation also involves reflection in action, reflection on action, and setting personal learning goals.

You just have to practice! At the start I just thought … what do I actually see? And when a teacher said what he saw, I thought: ‘No way! I didn’t see that at all’. But now I do, because I practiced’. (MS-2)

Teacher as facilitator

Teachers described how they facilitate the learning processes of their students. They emphasized the importance of interacting with students to monitor their learning process and to identify their learning needs.

Working with students requires interaction. This provides you with the necessary insights on their current performance level, whether they actually grasp it, or if you have to repeat some of the steps before a step forwards can be made. I have learned you cannot standardize teaching strategies. (T-3)

Another teacher commented:

It has to make sense for the student; only then does it become important. If you can connect the problems they encounter in their learning and what is important for them to learn, that’s when learning occurs. (T-6)

In addition, teachers explained how they facilitate students’ taking responsibility for their own learning process by supporting conscious goal setting and action planning.

As a teacher, you make sure a student sets personal learning goals related to what they already have learned. You give them their own responsibility to build their body of knowledge and skills. (T-8)

Teachers reported that students experience observing human movement as difficult. They can watch, but they cannot automatically give meaning to what they see. They need help. Guided instruction, deep questioning, and performance feedback on the observed are the strategies teachers apply to give meaning to the observations. Scaffolding is used as a strategy to facilitate students in reflecting critically on what they have observed, independent of teacher involvement.

Learning to observe is very difficult. You have to offer students a framework to guide them. You start to instruct what to observe. Next you ask, ‘What did you observe, and what did you notice?’ Subsequently you give feedback so that they become triggered to observe independently, being able to notice without being pointed at. (T-7)

Thus, the necessity for variation and flexibility in both learning environments and strategies were mentioned as important for encouraging learning.

Variation in the educational process: that is what I aspire to. Sometimes I start with performing an observation in a classroom setting; the next time, I let students start with an active observation of a patient. The same with information: sometimes I give necessary information beforehand; other times, I let them search for necessary information and help them to find this later. (T-2)

Peer learning

Learning with and from each other as peers was described as an important aspect of learning. This was perceived as more fun than individual learning. However, explaining ideas to peers, talking about and questioning each other’s views, was perceived as the most important benefit for learning.

I learned a lot from working with peers because you have to explain what you think and incorporate the theoretical background in your explanation. (S-1)

An additional benefit for peer learning was mentioned by lecturers: students learn to understand existing differences of opinion between students and between professionals.

The goal is not to state what is correct and what is wrong, but just to recognize and acknowledge the differences. The students have to experience that there are differences and try to figure out what the differences are and why they exist. (T-7)

In the following quote, a master student describes how his observational skills benefit from explaining to peers what is observed, describing the transfer of domain-specific knowledge to the context of observing human movement.

For me, learning skills includes explanations to others. By explaining to others, I have to truly understand the theoretical backgrounds; only by understanding can I explain. (MS-4)

Assessment and feedback for learning

All participants mentioned the benefits of assessments both formative and summative to enhance observational skills learning. Teachers described the valuable impact of assessments, such as performance assessments, peer assessments, and case-based tests, on clinical reasoning. Assessment triggers discussion and critical reflection.

For a script concordance test [a written test on clinical reasoning using video recordings of real patients], a panel of experts provide the answer key. The discussion with students about differences of opinion between students and experts is interesting. You need to discuss and explain the test items and results with the students, helping them to set their personal goals. (T-6)

A student stated:

Hearing from a teacher what was correct and what was wrong in my observations of the patient helps me to understand whether I am on the right track or not. (S-5)

Moreover, the process of providing peer feedback, receiving peer feedback, and discussing feedback encourages students to express their thoughts and to develop a critical attitude toward their own and their peers’ performance.

Discussing the feedback gives me insights on my thoughts, my own capabilities, or shortcomings. (S-3)

Contextual learning

Teachers reported the importance of contextual learning in two ways. First, skills learned in the classroom setting should be transferred to the context of their future professional clinical practice. Observing human movement in a learning context, which simulates clinical practice as much as possible, was advocated. Second, teachers should facilitate the process of transferring knowledge and skills. Skills learned on one problem should be transferred and applied in other situations. Third, teachers emphasized that learning a single skill should be embedded in the body of knowledge of the profession and related to the whole process of diagnosis and intervention.

In the classroom, you have to refer constantly to related theoretic backgrounds … . Physical therapy is an applied science; it needs the incorporation of knowledge of a physical therapeutic problem of the patient, a health problem. As a teacher, you have to make sure to constantly refer to and explain the connections. (T-11)

Building complexity

Teachers mentioned that it is important to consider the degree of complexity of movement observations in the construction of education. The students’ achieved level of observational skills led to the decision to start with global or specific observations, with simple or complex cases, or with simulated or real patients.

First, students notice large differences in performances, they evolve in their observation, [and] they start noticing smaller, more defined differences. Observing healthy subjects is more complicated than [it is with] patients with real health problems. (T-8)

Basically, the complexity rises the moment the student gets less time to observe and interpret the motor skills performance. (T-6)

Educational tools

Teachers and students mentioned educational tools as beneficial for learning processes. Examples were the use of video recordings, Internet sources, real patients, or volunteers with minor health problems in classroom settings, and patient demonstrations in clinical settings.

Tools you always can use are the technical instruments in our motion lab, but also [a device] as simple as a Smartphone or tablet. You can record … replay and watch together with the students [and] use it for learning. (T-4)

Professional content: what needs to be mastered

Participants described observational skills as the ability to observe carefully and attentively – to notice, perceive, and register what has been observed as significant. To learn observational skills, students must be aware of three different ways to observe human movement, referring to their, respectively, different approaches and goals for observation. Furthermore, participants described it as necessary to master domain-specific knowledge, gaining body awareness and building a frame of reference and expertise to learn observational skills. Seven interrelated design principles were defined for this theme.

Implicit and explicit observations

Differences between implicit and explicit observations were often mentioned, emphasizing the importance of the ability to explicitly describe findings. Both students and teachers described how students basically start to ‘watch’ as human beings (implicit), providing them with rich information, though it is not yet related to the physical therapeutic domain. When they learn to name what they see, they learn to ‘observe’ (explicit).

I think one always has a certain frame of reference … . sometimes you have a gut feeling while observing; you think movements are deviant, but you can’t explicitly name what you actually see. (MS-3)

As a teacher, you can give structure for how and a language for what they observe; this enables them to explain what they observe. (T-5)

Physical therapeutic observations were related to knowing what to expect and recognizing what one sees based on domain-specific knowledge on human movement.

Whenever I describe what I see, this should result in recognition. I can only recognize something when I see what I know; otherwise, there is no recognition. Recognition means you did observe and not only watch. (T-1)

The observation of simple versus complex motor behavior

Participants reported differences between simple movements; a discrete local movement (e.g. the flexion of an elbow or the action of a knee while walking) and complex motor behavior (e.g. the behavior of any motor skill that is influenced by characteristics of the skill itself, the person performing the skill, and the environment in which the skill is performed). The observation of complex motor behavior was emphasized as significant for physical therapists.

A physical therapist incorporates the context in the movement performed [and] searches for the intention of the movement; what is the goal to achieve, and is this possible in the circumstances as they are performed in? … They are not just observing different parts of a movement. (P-10)

Objective versus subjective observations

All participants emphasized the importance of both objective observations free of prejudice and objective interpretation of the observations. They mentioned starting observations subjectively as a pitfall, resulting in early interpretations from predetermined assumptions about the quality of movement. In every patient encounter, a practitioner must have an open mind.

By interpreting in an early stage, you have pre-assumptions, causing a focus on certain parts of the skill or body. With that mind-set, you start filtering your observation right away; however, you possibly fail to observe certain other important aspects because they don’t fit in your predetermined picture. (T-9)

A barrier to objective observation might be a limited frame of reference based on expertise in other contexts and knowledge domains.

The students who already have observed in a different context [e.g., sports] possibly think they know what observing movement means. But also, they have to observe in the specific context of health problems, physical therapy, to start building a new frame of reference. Essential in this is the right state of mind, which enables this. (T-6)

Mastering domain-specific knowledge

Participants stated that domain-specific knowledge regarding human movement is necessary for physical therapeutic observations. They mentioned specific theoretical contents (e.g. anatomical, kinesiological, physiological, and biomechanical principles). However, they also cited specific theoretical knowledge of motor control, motor learning, and motor development through the life span.

Most important for me was to start with theoretical knowledge and backgrounds. I really think it all starts there; you have to know what to look for. Knowledge about what is normal, what is deviant, … what is normal for certain ages. Also, theoretical backgrounds such as. Fitts and Posner are essential while observing; what stages in motor performance are shown? (MS-1)

Furthermore, the participants differentiated between the qualitative and quantitative aspect of movement performances. The need for explicit descriptions on how to appraise either the quality or quantity of movement was clear. Participants gave extended explanations about how to judge the quality and quantity of movements.

Whenever you are going to observe and judge quality of movement, you have to know what it is. Ideally, there should be some kind of list with items and descriptions for qualitative movement. (S-4)

Movement assessment

The availability and use of standardized assessment tools, motor tests, norm, and reference values, and checklists for the observation of movement quality were discussed on several occasions. The necessity of using them while working with patients was evident.

Interpretations of observations occur based on reference values; a student has to know these. He has to recognize variances of performance regarding the norm. This enables the student to recognize deviant movement or when a performance is more or less, according to the norm, a typical movement. (T-1)

It is important that a student has a structure for observations. They have to learn reference values for typical movement. You could use measurement tools in the classroom for this goal. (T-5)

Body and movement awareness

Body awareness and an adequate sense of movement with the observed human movement were perceived as essential to recognizing and understanding problems in functioning.

Before observing, it is beneficial to perform certain activities yourself [student] and reflect on how you perform just to get a picture of the movement. (T-10)

Experience

Teachers explained that mastering observational skills and developing expertise requires deliberate practice and experience in a variety of clinical situations. They stated that experiences are critical for building a personal frame of reference and for mastering observational skills and interpreting the quality of human movement.

Knowing and recognizing is very important; the more you know, the more you recognize, and that is called expertise. (T-1)

Conditions for optimal learning

The observation of human movement requires optimal conditions, not only for learning as a student but also for coaching students as a teacher or observing human movement as a physical therapist. These conditions were described as follows: (1) accurate professional behavior, including creating mental space and tranquility; and (2) establishing a standard language. Two design principles were defined.

Professional behavior

A professional attitude was indicated as necessary to observe human movement. Willingness and ability to show certain behavior, determined by norms and values, motivation, and personal incentives, were mentioned as essential. Examples were given about learning readiness, perseverance, decisiveness, and willingness to collaborate.

As a physical therapist … you have to take time for your patient, and as a teacher, most importantly, you have to create the opportunity and structure for the student to develop this professional behavior. (T-8)

Standard language

Participants argued for the use of standard language when observing and interpreting human movement. They mentioned the current diversity in definitions and descriptions of movement quality. Establishing a standard language guided by therapeutic constructs was mentioned as a way to improve communication between physical therapists.

There is a need for clear definitions and descriptions. If definitions and descriptions are clear and mutually shared, then there will be less confusion in communication between health professionals. (P-7)

Proto-theory for educational program design

The interrelations between our three identified themes 1) didactics; 2) professional content; and 3) conditions for optimal learning and their design principles are shown in Figure 2. By discussing and reflecting on our analysis and results and by relating our results in existing literature, we developed a proto-theory for an educational program designed for learning observational skills (Table 2). Although the design principles for the didactic, professional content, and conditions for optimal learning themes are interrelated, our proto-theory focuses on the more generic didactic principles that facilitate observational skills learning. We considered those principles as generalizable to other professional content also, whereas conditions for optimal learning are prerequisites for learning. Our proto-theory shows the link between the identified design principles for the didactic theme, associated theories on learning and behavioral change, and recommendations for observational skills learning. Based on the design principles for the didactic themes, we identified six underlying educational theories for our proto-theory. These six theories, with a short description, are as follows: (1) Self-Regulated Learning Theory (Greene and Azevedo, 2007; Schön, 1983; Zimmerman, 1990, 2002): Students are self-regulated to the degree that they are metacognitively, motivationally, and behaviorally active participants in their learning. Learning goals are set by the student based on both past experiences and current learning experiences. The four areas of self-regulation are cognition, motivation, behavior, and context. (2) Self-Determination Theory (Ryan and Deci, 2000): Motivation for learning initiates activities and enhances the self to initiate behavior. Three basic needs for intrinsic motivation are (1) competence or efficacy; (2) relatedness (i.e. the need to feel a sense of belonging and connectedness with others [e.g. peers/teacher]); and (3) autonomy (i.e. the ability to perform activities on one’s own initiative, without experiencing external control or influences). (3) Social Constructivist Learning Theory (Norman, Bordage, Page, and Keane, 2006): Knowledge construction enhances attention to, storage of, and retrieval of knowledge from memory using contextual cues to facilitate the transfer of learning from the learning context to the application context. Teaching occurs in the zone of proximal development as described by Vygotskiĭ (Omrod, 2004) using scaffolding (Rasmussen, 2001). (4) Social Cognitive Learning Theory (Bandura, 1986; Bandura and Locke, 2003): Active participation and knowledge construction occur in collaboration with peers, enhancing the development of self-efficacy beliefs by performing the new behavior and experiencing the consequences of that behavior (i.e. mastery experience). (5) Variation Theory (Marton and Säljö, 1976; Marton and Trigwell, 2000): Individuals understand phenomena in the world differently because the experience is always partial. Learning takes place when the difference occurs against a background of sameness. Conditional for learning are the four patterns of variation: contrast, generalization, separation, and fusion. (6) Feedback Intervention Theory (Kluger and DeNisi, 1996; Kluger and van Dijk, 2010): Feedback changes the locus of attention among three general and hierarchically organized levels of control: task learning, task motivation, and meta-tasks processes (including metacognitive aspects of task learning).

Discussion

The overall aim of our study was to derive design principles for an educational program to develop observational skills for students in physical therapy. After our exploration of the needs, beliefs, and experiences of students, teachers, and practitioners regarding observational skills learning, we used the analyzed data to develop design principles and a proto-theory (Table 2). We based the design principles for the three identified themes: (1) didactics; (2) professional content; and (3) conditions for optimal learning on content-related data. However, the developed proto-theory focuses on more generic didactic principles that facilitate learning and is itself independent of the content.

The decision to focus the proto-theory specifically on the design principles belonging to the didactic theme was partly based on the poor description in the literature of specific educational strategies to support the development of observational skills learning (Al-Moteri, Plummer, Cooper, and Symmons, 2015). Another consideration was that design principles for the didactic theme and its associated principles could possibly be identified as generic didactic design principles. In contrast to professional content, which depends on the topics, skills, or goals students have to learn or achieve (Lubarsky et al., 2015; Monteiro and Norman, 2013; Norman and Eva, 2010; van de Wiel, Boshuizen, and Schmidt, 2000), and conditions for optimal learning which are prerequisites for learning and professional behavior (Grace and Trede, 2013). Including the design principles from all three identified themes in a proto-theory would have been possible; however, this would have led to the development of a more specific proto-theory for observational skills learning. While analyzing our data, we realized we had the opportunity to develop a generic proto-theory for learning by only including the design principles belonging to the didactic theme. The developed proto-theory shows the link between the design principles for the didactic theme and the associated theoretical concepts on learning and behavioral change, which could be considered generalizable and thus could be combined with other learning content in diverse professions. This proto-theory could also guide the ongoing development and refinement of educational design for skills learning.

One of the results of our study is the proto-theory that comprises two leading design principles: ‘student in the lead’ and ‘teacher as facilitator’. The other principles (i.e. peer learning, assessment and feedback for learning, contextual learning, building complexity, and educational tools) can be linked with these two principles. The first design principle is that students need to take the lead in their own learning to master observational skills. The importance of self-regulating learning processes based on personal learning goals is explained by the self-regulated learning theory (Greene and Azevedo, 2007; Schön, 1983; Zimmerman, 1990, 2002); and self-determination theory (Ryan and Deci, 2000). Supporting and allowing students to take the lead in their own learning and the responsibility for their own learning depends on both active involvement in learning and the students’ motivation to learn (Greene and Azevedo, 2007; Ryan and Deci, 2000; Schön, 1983; Zimmerman, 2002). Furthermore, the level of motivation is significantly related to the students’ self-efficacy beliefs (Bandura and Locke, 2003). For the students, this implies deliberate practice, critical reflection on the quality of the demonstrated observational skills, asking for feedback, and designing an improvement plan with personal learning goals.

The second leading principle is the teacher as facilitator. To facilitate student learning, the teacher helps to identify the personal learning needs of students, monitors their learning process, checks understanding, and adjusts instruction or coaching tailored to the actual level of students’ performance as supported by the social constructivist learning theory (Norman, Bordage, Page, and Keane, 2006; Omrod, 2004). Furthermore, the teacher enhances collaborative learning and critical performance appraisal among peers (Norman, Bordage, Page, and Keane, 2006). A dialogue with peers is important to build adequate self-perceptions as a tool to organize and give meaning and understanding to thoughts. Interaction with peers can help students to develop a critical attitude toward themselves and toward peers and to create opportunities to help each other in organizing thoughts by stimulating reasoning aloud (Chi, de Leeuw, Chiu, and LaVancher, 1994; Sluijsmans, Brand-Gruwel, van Merriënboer, and Bastiaens, 2003). In this collaborative learning process, the teacher coaches the students to develop adequate skills to support each other by providing constructive feedback. Active engagement in group work, and critical self-appraisal, will help students to optimize the distance between “what is known” and “what is to be learned” (Chi, de Leeuw, Chiu, and LaVancher, 1994; Seenan, Shanmugam, and Stewart, 2016). Additionally, the teacher, as a facilitator, is responsible for the choice in the complexity of cases and the choice of educational tools to facilitate gradual student development according to learning in Vygotskĭi’s zone of proximal development (Omrod, 2004).

Another design principle is the choice of educational tools. More specifically, the choice between enabling students to observe real patients, simulated or standardized patients, or students acting as standardized patients must be considered. Observing real patients adds the most to the learning process of students; however, the possibilities to observe real patients are often limited. Therefore, as an alternative, peer students are often used as standardized patients in the classroom setting. However, the use of same-year students acting as standardized patients may be perceived as less realistic than the use of trained standardized patients (Mounsey, Bovbjerg, White, and Gazewood, 2006). Training senior students as standardized patients for junior students could overcome these concerns, as the study of Mandrusiak et al. (2014) demonstrated. Senior students acting as standardized patients resulted in positive experiences for both junior and senior physical therapy students, with significant improvements in reported self-efficacy and satisfaction (Mandrusiak et al., 2014).

Although our developed proto-theory and the figure with interrelated design principles could be useful for teachers, it does not necessarily mean that all teachers can apply them in their daily practice of teaching. This implies a need to focus on training for teachers as well. Teachers must monitor and adjust the learning process and identify the learning needs of students. Doing so requires a sense of efficacy for teachers; they must have or develop positive beliefs about their ability to effectively teach, monitor, and assess all students well (Delale-O’Connor, Alvarez, Murray, and Milner, 2017). How to reflect critically on students’ performance and provide tailored feedback possibly based on an assessment must furthermore be addressed in teacher training. Not all feedback leads to performance improvement, as indicated by the feedback intervention theory (Kluger and DeNisi, 1996; Kluger and van Dijk, 2010). The type of feedback and the circumstances in which feedback is provided can affect students’ motivation and self-efficacy beliefs (Gielen et al., 2010; Govaerts, 2015). Creating opportunities for teachers to observe other teachers performing successfully in class (i.e. serving as role models), as well as experiencing their own success in classroom contexts, might enhance teachers’ sense of efficacy and help them to facilitate student learning (Bandura, 1986; Delale-O’Connor, Alvarez, Murray, and Milner, 2017).

Strengths and limitations

A strength of our study was the methodology of design-based research, which enabled us to triangulate multiple perspectives and sources of data collection in developing an educational program to teach observational skills (Anderson and Shattuck, 2012; Barab and Squire, 2009; Brown, 1992; Design-Based Research Collective, 2003). Using design-based research gave us the opportunity not only to invite students, lecturers, and practitioners but also for several individual rounds of data gathering using NGT meetings and interviews. We believe that expertise in learning strategies should be derived not only from professionals but from everyone who is involved in learning (Anderson and Shattuck, 2012; Barab and Squire, 2009). Moreover, a research group with a variety of backgrounds and expertise qualitative as well as design research, education, and physical therapy strengthens the review process and discussion of the themes and design principles. We ensured the inclusion of teacher participants with various theoretical and practical backgrounds (e.g. musculoskeletal physical therapy, sports physical therapy, and neurologic physical therapy) to reduce the influence that the researchers’ experiences and backgrounds (i.e. pediatric physical therapy) could have had on the outcome.

A challenge of design-based research is the difficulty of deciding when if ever a study is completed; because every cycle provides new information, it will be difficult to reach saturation (Anderson and Shattuck, 2012; Vanderhoven, Schellens, Vanderlinde, and Valcke, 2016). For our study, we modified the four sequential steps developed by Reeves (2006) according to our research aim to derive design principles for the development of educational programs to teach observational skills for physical therapy. A possibility was to conduct at least one more round of testing an education program, which would have improved the research design and its efficiency. However, we chose to conduct only one iterative cycle in step 3 and to continue the iterative process in step 4. By conducting NGT meetings and interviews and evaluating the draft educational program, we had the opportunity to go back to the participants in different rounds and listen to their opinions. After the last round, no new data emerged, so we decided to focus on the theoretical understanding of the derived design principles through the reflection in step 4.

Although we did not identify any differences in design principles and approaches for observational skill learning between bachelor and master students in either our study or the literature (Al-Moteri, Plummer, Cooper, and Symmons, 2015), we did not explore this explicitly; it would be interesting to study this in more depth in the future. A limitation was the decision in step 3 of our study to test the draft educational program during a postgraduate program for which learning outcomes were set. In the NGT meeting following this training, the participants possibly not only responded to the question about their perceived needs, beliefs, and experiences regarding observational skills learning, but also to the draft version of the manual and supplemental materials in relation to the learning outcomes, which may have contributed to the data and analysis. Another limitation is that although it is unlikely that observation of movement quality differs between countries, language differences, and differences in didactic approaches may influence the international transferability of the results. A recommendation for further research is to continue the iterative process of the development of educational materials with students, teachers, and practitioners to complete the implementation of materials in practice.

Conclusion

We conclude that to learn observational skills students must take the lead in their own learning process facilitated by an experienced teacher. This might imply a need for additional teacher training to strengthen their didactic skills. Another precondition for learning observational skills is a realistic context. This might imply the consideration to enable students to observe real or standardized patients to increase possibilities for observation in a clinical context. Although our proto-theory was developed for physical therapy students to learn observational skills, it might be interesting to study whether this proto-theory is applicable to other professions and in programs with different content related to learning observational skills, for example, interviewing patients. Regarding the research methodology used for designing educational programs, we can conclude that a design-based approach suited the project well. Its advantage is that all stakeholders fully participated in the design process, enhancing the applicability of the design in the context of its end users.

Declaration of Interest

The authors report no conflict of interests.

Acknowledgments

We thank Irene van der Glind for her support during the analyses process. The Dutch Research Council (NWO) supported Lieke Dekkers, under grant 023.004.037.

Additional information

Funding

This work was supported by the Dutch Research Council (NWO) [023.004.037].