Learning from 360-degree film in healthcare simulation: a mixed methods pilot

Abstract

Technology that delivers an immersive experience in education offers a viable alternative to in-person teaching. This study aims to compare learning from a clinical encounter viewed in a virtual reality 360-degree headset to that of a traditional monitor by quantifying the user experience and testing what was learnt. Furthermore, experiential learning is described as a key concept in simulation practice, and this is explored using transcripts of participants' experiences with 360-degree video. We could determine no statistical difference between median exam scores between groups (p = 0.25), and there was no correlation found between total immersion and motivational scores with exam performance (Rho = −0.14 p = 0.18, Rho = 0.08 p = 0.31). However, those viewing 360 media reported significantly higher immersion, motivation, and empathy scores (p < 0.05). Domains based upon Kolb’s learning cycle generated themes including engagement, communication, and self-efficacy. 360 video creates an immersive experience with an associated high-value motivational position; however, this could not be translated to an increase in exam scores. There are benefits to perceived learning and emotional content with 360 videos, although, pedagogical theory needs further understanding if educators are to embed new immersive technology in curriculums.

Keywords:

• Medical education
• simulation
• immersion
• technology enhanced learning

Background

Simulation is ubiquitous in the training of health care professionals (HCP) in their undergraduate and postgraduate studies. It is the process whereby a defined set of characteristics and environmental conditions are artificially created to mimic a real-world clinical scenario for others to experience (Al-Elq, 2010). Thus, simulation aims to increase clinical competency, reduce potential medical errors, and ultimately improve patient safety (Flanagan, Nestel, & Joseph, 2004) by bridging the gap between theory and the ‘doing’.

The resemblance to reality is described as ‘fidelity’ and classification of this from low to high fidelity can confer increasing costs (Al-Elq, 2010). This cost can impose a barrier to providing simulation and despite simulation enhancing HCP clinical skills access to simulation is not equitable (Park et al., 2020). Suspension of disbelief and a disconnect to the external environment contributes to the realism of the experience by those learning from simulation. A review of 113 articles on the utility of simulation found that repetition of conditions that mimic clinical scenarios has been shown to benefit learners in communication skills, reduced anxiety and preparation for assessment (Okuda et al., 2009).

Immersive technology offers a viable alternative to traditional in-person simulation with high levels of engagement (Jacobs & Rigby, 2022) offered by 360-degree (360) recordings of clinical scenarios. They grant users flexibility and upscaling of participants without compromising patient safety (Patel et al., 2020). 360 is a subset of virtual reality (VR) that compromises a passive (non-interactive) visual and auditory observation (Taubert, Webber, Hamilton, Carr, & Harvey, 2019). There have been limited studies in appraising the learning from 360 (Tang, Cheng, Mi, & Greenberg, 2020) and a need for more robust studies on immersive technology enhanced learning (TEL) and VR (Baniasadi, Ayyoubzadeh, & Mohammadzadeh, 2020; Pottle, 2019). One study, however, evidenced faster surgical knot tying by students learning from 360 compared with a standard recording of a teaching session (Yoganathan, Finch, Parkin, & Pollard, 2018). Other benefits have been shown in participant empathy with 360 content (Dyer, Swartzlander, & Gugliucci, 2018) and building communication skills (Drewett, Hann, Delacroix, Pan, & Fertleman, 2018).

There are numerous models of learning that have been evaluated in medical education (Swanwick, Forrest, & O'Brien, 2019). The process of participating in simulation presents the trainee with an opportunity for iterative cycles of learning through experience (Amod & Brysiewicz, 2019; Patel et al., 2020; Stocker, Burmester, & Allen, 2014). The experiential model of learning first theorised by Kolb (1984) asserted that learning starts with a concrete experience that follows with reflection and the integration of the observations forms the abstract conceptualisation. Lastly, learners actively experiment by transforming the knowledge into problem-solving activities. Summarised in Figure 1.

Figure 1. Kolb’s model of experiential learning was adapted by Stocker et al. (2014).

This study aimed to compare learning from 360 films of a simulated medical consultation versus a more traditional flat screen two-dimensional (2D) simulacrum. Furthermore, this work aims to qualitatively explore experiences of learning with 360 through open-ended survey responses, which will additionally provide new understanding and identify benefits and barriers to this TEL.

Methods

Study design rationale

Mixed method studies incorporate and integrate quantitative and qualitative philosophies with the objective to achieve a greater level of evidence than a single method. Pragmatism gained by this approach enables researchers to address multi-faceted constructs (Doyle, Brady, & Byrne, 2009; Lavelle, Vuk, & Barber, 2013). Post-positivism framework guided quantitative reasoning and this philosophical standpoint assumes that explanation can be reasoned by the phenomena explored. The interpretative approach, however, recognises that human nature and behaviour cannot always be explained by empirical data (Smith & Shinebourne, 2012). Hence, interview narratives are viewed by an inductive and deductive lens, whereby the learning theory seeks confirmation by observed patterns and iterative review of themes and theory (Barber et al., 2019). The hybrid of approaches employed incorporates data-driven, inductive analysis alongside a deductive analysis using an a priori template to create codes (Fereday & Muir-Cochrane, 2006). The theory-driven hypothesis of experiential learning helps guide thematic content analysis (Braun & Clarke, 2006).

Research questions aim to address if 360 has the potential to increase learning and associated positive subjective experience scores, including if increased empathy is felt by users of 360. Two conditions were tested to determine if differences were found. One group viewed a clinical consultation based on primary care in 360 allowing for greater immersion as there is the freedom to move the head. The second group viewed the same consultation on a traditional monitor.

The dependent variables were the measure scores in immersion, adapted immersive experience questionnaire (AIEQ), intrinsic motivation (adapted intrinsic motivation inventory, AIMI), and a multiple-choice question (MCQ). IEQ and IMI structure and content validity as instruments of measuring user experience have been extensively tested (Jennett et al., 2008; Ryan, Mims, & Koestner, 1983). Jacobs & Rigby (2022) further work tested concurrent validity in the context of the medical health setting.

Participants

In 2019, HCP trainees were recruited via non-consecutive sampling following teaching within a regional hospital in United Kindgom and a local university (n = 40), see Table 1. Eligibility criteria required participants to be over the age of 18 and currently in a form of healthcare training or qualified in an HCP field and undertaking further training.

Table 1. Demographic distribution of participants by HCP background.

Area of study or occupation	Frequency	Relative proportion
Doctor	5	12.5%
Medical student	17	42.5%
Physician associate	2	5%
Physician associate student	16	40%

Unblinded randomisation was utilised to allocate participants to either 360 or 2D arms with no study rewards gifted. Purposive sampling from the 360 groups for interview (n = 8) with a strategy to include any level of immersive technology previous experience and to have representation from each HCP subgroup recruited (Figure 2).

Figure 2. Study protocol overview. Key AIEQ = adapted immersive experience questionnaire, AIMI = abridged intrinsic motivation inventory, MCQ = multiple choice question.

Materials

The 360 and 2D versions of a primary care consultation were produced using a Samsung 360 camera. A medical encounter of abdominal pain was selected by a group of general practitioners (GP) to be scripted and then performed by a professional medical actor. This type of consultation would involve a GP needing to utilise the whole room, whilst the camera remained central in the room. This allowed for the entire 360 fields of view to be accessible by someone viewing in a Samsung virtual reality headset.

The consultation included posters depicting short sentences that relate to medicine, which were placed in different room viewpoints that would be seen as the GP walked across the room. They were printed in black and white for increased saliency and were in a large bold font that could be resolved in both formats. Additionally, key phrases were scripted, and important patient history was verbally audible to allow for a composite MCQ of both visual and auditory data.

CyberLink PowerDirect software version 16 enabled seamless stitching of camera footage (Figure 3) and placement of a title screen. Additionally, the 2D conventional video was created using the same 360 footage to minimise bias. Firstly, axis X, Y and Z were aligned to assume a view if looking directly at the doctor and patient and zoom was applied to remove some head and ground space. A direction of view was applied by selecting a target in the room that could be followed to mimic a camera pan. The GP was selected as they completed a full tour of the room during the consultation and the patient was viewable during the history, examination, and management. The consultation was concluded after 7 minutes had elapsed.

Figure 3. Stitched 360 still of room with GP in centre and patient on the couch. Posters to test knowledge seen above the GP to right and above the patient.

Procedure

The study was conducted in a simulation department at a regional hospital and participants were explained and taught how to use a Samsung Gear VR headset. The side of the headset had simple buttons to give users an opportunity to stop the video if motion sickness or other causes occurred. Likewise, a user watching the conventional 2D on a laptop could stop the video using universal video controls. The space was free from obstacles, which could allow headset users to move safely and unimpeded. Following media viewing the participants completed the AIEQ, AIMI, and MCQ.

The AIEQ is a 23-item measure with responses rated on a 1–5 Likert scale and the AIMI is a 14-item questionnaire with a 1–7 Likert response scale. The MCQ involved selecting the single best answer from 4 options for a total of 6 questions.

A researcher remained present throughout the AIEQ, AIMI, and MCQ data collection. 8 participants from the 360 arm were invited to complete an online survey with free-text responses to questions submitted by candidates through email participation.

Data analysis

For statistical analysis Stats Direct (version 3) was used. Following tests for normality, Wilcoxon’s signed rank was used to compare findings between groups. Pearson correlation analysis informed if an association between AIEQ and AIMI existed. Additionally, quantitative questions that related to research questions underwent further analysis. Likert scale was treated as continuous data. Written responses to the five qualitative questions in the open question survey were coded and categorised using Nvivo (version 1.0).

To ensure reflexivity two researchers independently coded the transcripts, which formed the primary analysis, and subsequently, met to discuss and refine codes iteratively. It is acknowledged that as the researchers are medical educators this will influence the process of interpretation of transcripts and iterative coding.

Quantitative results

There was no missing data. Randomisation occurred for every participant recruited. Seventeen participants were allocated 2D and 23 to the 360 versions. A test for normality and kurtosis indicated that the data was to be treated as nonparametric.

Those viewing the media on the 2D test of learning MCQ median score was 6 (IQR = 1) and also those on 360 medians scored 6 (IQR = 2). Wilcoxon’s signed ranks test determined no statistical difference between population medians (p = 0.25), and there was no correlation between immersion AIEQ total score and MCQ performance (Rho = −0.14 p = 0.18). The motivation total score in AIMI did not correlate with the MCQ score (Rho = 0.08 p = 0.31)

The total immersion score median for 2D was lower at 65 (IQR = 10) than 360 immersion total median score was 80 (IQR = 11). Additionally, the total score from the abridged intrinsic motivation measure for the 2D arm was 58 (IQR = 11) and for 360 the median was higher at 69 (IQR = 15) See Figure 4.

Figure 4. Box and whisker plot of total AIEQ and total AIMI scores for 360 and 2D.

Wilcoxon’s signed rank test on immersion measure question, ‘It was if I could interact with the environment as if I was in the real world’, median score difference of 1.5 favouring 360 groups (p = 0.004). This level of immersion with AIEQ question ‘How immersed did you feel’ identified a median score greater in the 360 participants by 3 (p = 0.001).

Within the AIMI a question asking about activity enjoyment, ‘I enjoyed doing this activity very much’. Identified increased enjoyment by a median score of 1 in the 360 groups. Additionally, the final question on AIMI asked on a user response to perceived learning, ‘I think I can learn from this activity’. The median difference between groups favoured 360 by 1 (p = 0.016). Questions relating to empathy experienced within the AIEQ had a median higher score of 1.5 in the 360 groups (p = 0.04), as seen in Figure 5.

Figure 5. Box and whisker plot comparing AIEQ questions on empathy 2D versus 360.

Qualitative data primary analysis

Both researchers coded entire transcripts using Nvivo (version 1.0). Emerging codes were shared and reviewed in virtual meetings and initial thematic analysis informed code creation and defining themes (Braun & Clarke, 2006).

Theoretical stance and application to analysis

Theoretical underpinning analysis with Kolb’s theory of experiential learning gave the researchers an initial source of codes (Figure 1), thus critical reasoning and pragmatism add to code development and theme emergence. Themes from the analysis are described under the four headings (major themes) that relate to Kolb’s theory. NVivo-assisted deductive and inductive thematic analysis of transcripts using Kolb’s theory of experiential learning was used to interpret data and form the theoretical stance.

The experiential theory is focussed on the learner’s internal cognitive state and involves the acquisition of abstract concepts that be then be applied in a learning cycle to develop new understanding. Learning becomes the process of creating knowledge from experience. This is described in four stages of a cycle and the researchers considered this as integrated with each stage during the conversation and exchange of data. By defining the four units of analysis: do, reflect, conclude, and apply. The narrative was translated using the inductive approach, as well as deductively considering the data with relevance to Kolb’s theory. Both researchers reviewed all four units and communicated interpretations to ensure codes were coherent and covered the narratives. This process used each unit as a point of reference to judiciously categorise the entire dataset in the application of 360 videos in medical education.

Results

Themes from Kolb and analysis of the dataset are presented under the headings of the four units they relate to. The key themes are underlined, and subthemes are in bold. Figure 6 pictorially represents this analysis in the form of a mind map.

Figure 6. Mind map pictorially indicating the relationship of major and subthemes.

Concrete experience

Scenario

360 affords the learner an opportunity to be present in consultations and moreover it is a learning resource that could be expanded to produce a compendium of different scenarios.

‘the scenario could influence the observer….360 video would be a great implementation to difficult scenarios’ Participant 2

‘a library of all of the different scenarios the student could experience’ Participant 3

‘Simulation of situations which are rare or difficult to teach in.’ Participant 9

A subtheme existed of Flexibility and this described the ability to watch a 360 video asynchronously to others and allows for repetition of material to enhance learning. Students felt that this would assist in training.

‘It offers more flexibility in terms of timescales, can be viewed in own time, at own pace’ Participant 2

‘entering this environments by allowing them to get comfortable with it prior to the actual, in-person placement.’ Participant 3

‘especially since these can be specifically created with teaching specific learning points in mind, or a student could select what sort of topics they would want to cover (whereas in real life you have to work with whatever the patient has come in with!)’ Participant 4

Engagement

The experiential model supposition that the learner is engaged was expressed by the group of students and they identified this in the context of how an individual is immersed in the environment. The detachment from real-world surroundings adds realism to the media they see and how involved they feel.

‘placing of an individual at the centre of an activity. Meaning that all senses and awareness are maximised and potentially different to what the person actually feels.’ Participant 2

‘Feeling like you are totally involved in an experience (physically, emotionally, mentally)’ Participant 7

‘A level of engagement with a task which leads to exclusion of unrelated thoughts and external sensations.’ Participant 9

Group

The individuality of the experience was reflected by one student as a barrier to utilising the 360 videos, whereby group interaction can influence learning. Group learning is not explicitly described in Kolb’s model, however, it could be seen as part of the ‘doing’ stage (concrete experience) whereby medical encounters often involve more than one HCP.

‘The device as far as I'm aware can only support one observer at a time so I think this could be a barrier in terms of groupwork and interaction as a group would be limiting’ Participant 2

‘Cannot be shown to a group in the same way a normal 2D video can – makes use for guided teaching more difficult’ Participant 9

Reflective observation

Challenge

Transforming experiences or feelings into understanding occurs as we internalise the observation and reflect on what happened. Students can find certain encounters challenging and 360 was described as a tool that could supplement their education by giving them opportunities to observe these.

‘Content aimed at a higher difficulty level like aforementioned difficult consultations and such that I would struggle with today would be very helpful’ Participant 3

‘situations with case based learning. Or for niche conditions/presentations’ Participant 4

‘improve attempts to relay chaotic, stressful or otherwise challenging environments….simulation of situations which are rare or difficult to teach in’ Participant 9

Communication

This is an integral skill to any HCP and students reflect this with their description of the observed consultation and process of how they are assimilating knowledge and judging complexity.

‘Can be useful to review and analyse consultations. Can also help to assist with training and maintaining that level of care and practice’ Participant 1

‘We often use examples of both good and bad practice in relation to communication skills. This element of immersing yourself within the scenario could influence the observer to give more in-depth feedback regarding the consultation’ Participant 2

‘I think as a method it could be incredibly useful on teaching how to conduct consultations regarding atmosphere and rapport in an engaging way’ Participant 3

‘you don't necessarily know the doctor's thought process throughout the consultation and how they are arriving at a specific diagnosis….I think it's so important to have patient contact frequently and early on’ participant 5

This can be regarded as a transition whereby the reflections on the experience and the observed interpretations mould the eventual conclusions we make. One participant commented on this transitionary phase by describing how students develop over their years of medical training.

‘However for lower year students, it could be an incredibly effective tool in the periods of transition to placement.’ Participant 3

Abstract conceptualisation

Constructivism occurs as learners form the knowledge as opposed to passively absorbing material. Thus, forming the new representations and revalidation or discounting of pre-existing knowledge to generate new schemas. Participants created concepts on 360 videos as an educational tool, which gave several barriers to implementation although it was valued from a humanistic perspective as they considered the emotional aspects of the role of HCP.

Barrier

The consideration of cost was perceived on two levels. Firstly, the financial cost to the individual or institution in setting up and delivering adequate teaching material. Reflecting this, students anticipated resistance from medical institutions to adopt TEL. Second, was the cost to students’ education, emphasising 360 is not a replacement for real-life patient encounters. These factors influence the accessibility of the devices to students and risk an uneven curriculum if the technology is not available to some.

‘The main barrier is probably its accessibility.’ Participant 2

‘Cost, getting ancient medical schools to adapt to modern technology’ Participant 6

‘You don't get patient contact which could be detrimental to medical training’ Participant 7

Emotion

Placements for students are often new environments and, with this, comes anxiety about what they might expect to see or do. As such, one participant felt 360 could be used to tour a setting prior to placement and this would help reduce anxiety levels.

‘I can also see it decreasing anxiety in students regarding entering this environment by allowing them to get comfortable with it prior to the actual, in-person placement.’ Participant 3

In addition, it was suggested that 360 could be utilised to develop confidence in clinical skills and procedures prior to translating learning to real-life scenarios. This similarly mitigates anxiety around attending clinical environments.

‘When we are out of practise it is beneficial and confidence boosting to watch first, before trying ourselves.’ Participant 5

Active experimentation

Applying the conclusions thus far, participants valued the repetition of videos to allow knowledge to be placed via practice. It was acknowledged that 360 could ultimately be utilised to assess accomplishment.

Self-efficacy

The system whereby we can assess our internal emotions and abilities creates an assessment on our capacity to execute behaviours to produce a specific performance.

‘Watching practical procedures can be useful, especially since on many placements there is less opportunity to do these on a regular basis’ Participant 5

‘Learning for and doing OSCEs, training in new surgical techniques or other practical skills, learning how to manage basic clinical situations (like simulation training)…. Likely to be a good educational tool throughout medical education and career. Good for ongoing assessments such as those in specialist training’ Participant 8

‘Could see it as a way of replacing OSCEs’ Participant 9

Discussion

This study aimed to answer the hypothesis that users of 360 educational material will learn more. The results did not find any statistical difference in the group's performance in the MCQ despite the AIEQ indicating an increase in perceived learning. A non-randomised study that compared obstetric learning by comparing 360 with traditional viewing formats additionally found no difference in measurements of knowledge following an MCQ test (Arents, de Groot, Struben, & van Stralen, 2021). Whilst a single institution study, the 89 participants were a suitably powered sample size. Anbro et al.(Anbro et al., 2020) focussed on communication skills as a form of assessment when comparing media formats and identified higher post-test scores, similarly, participants identified this as a theme of learning in our study. Qualitative data was not formally analysed alongside communication scores, however, they noted comments on increased engagement and desire to see more videos in this format. The AIEQ as a measure of engagement seen in this paper has significantly higher scores with 360 and this was correlated with increased intrinsic motivational scores on a value subset. Additionally, we found a significantly higher score of empathy towards the patient using 360, which forms an essential part of communicating. This was further commented on by a participant in how 360 immersion could assist in rapport-building skills. This paper adds to the other works and in design has both randomisation and two institutions to minimise bias.

The qualitative methods utilising Kolb as a theoretical stance to explore TEL have not been performed before and facilitated the narrative to help us review the value and potential barriers to immersive technology in medical education. There were several themes that related to the ability to experience more clinical procedures and in new scenarios. This would allow the learner to experiment and gain confidence so that they might improve performance, which could be ultimately assessed formally in exams, for example, objective structured clinical examinations (OSCE). A study that compared accuracy and speed of surgical knot tying found those watching 360 could perform the skill more accurately and faster, assessed using an OSCE (Yoganathan et al., 2018). It is important to consider the validity of the instrument of assessment as the MCQ in our study and OSCE in Yoganathan et al. work did not undergo the construct and content validity that the AIEQ in AIMI underwent (Jacobs & Rigby, 2022). Face validity was sought by doctors reviewing the MCQ, however, further construct validity work is required to whether conclusions can be drawn on how the test scores relate to the subject being studied (Heale & Twycross, 2015). A further limitation to their and our randomised controlled trials is both had 40 participants, which is underpowered as 48 participants were our original sample size goal. Large effect size with significant findings was seen with AIEQ and AIMI. However, to discriminate MCQ scores, a larger population would be advantageous. This may have helped work out if a ceiling effect had been reached with both groups.

Transportation and dissociation from the real world are concepts, that in part, informs the AIEQ construct. The 360 scores are higher, which is reflected in participant narratives that consider the emotional and situational awareness that this media can offer. Simulation enables the development of knowledge, skills, and behaviours. As an educational strategy, simulation provides the opportunity for learning that is both immersive and experiential (Aggarwal et al., 2010). The process of learning and testing this at a later stage as improved performance or reduced medical error is not straightforward to evidence. Learning is complex with over 10 theories in existence (Swanwick et al., 2019).

The benefit of analysing both quantitative and qualitative data sets affords a perspective of connecting the information to existing theory. The authors propose a unified learning model that encompasses elements of experiential learning, social constructivism and the exploratory work in this paper on immersion and motivation measures. The i-GEM (individual, group, engagement and motivation domains) theory of experiential learning (see Figure 7) in TEL. The process of learning is dynamic, and influences exist at different stages and depths of learning, whilst applying what is learnt. How we learn is individualistic, however, why we learn holds a dualism of intrinsic and extrinsic motivational systems. Also, the environmental context whereby multiple people that might be learning or engaged in clinical activities will affect the overall individual learning. Collective group cognitive functions become by-products of social interactions and individual self-efficacy affects almost all endeavours.

Figure 7. Visual representation of i-GEM (individual-group, engagement, and motivation) model of learning in TEL simulation.

Conclusion

This study did not find increased learning from 360, although, participants described many learning benefits. Those viewing 360 media reported higher immersion and motivation as evidenced by statistically higher AIEQ and AIMI scores. Additionally, increased empathy was experienced with those viewing the more immersive 360 videos. There is scope to explore immersive technology with a larger study that evaluates different modalities of learning. Further work to validate i-GEM as a measure and its relationship to learning could benefit users and educators as a means to evaluate and inform applications of new technology that offer immersive experiences in medical education.

Acknowledgements

I would like to thank Jacob Rigby and Daniel Finnegan who assisted with AIEQ and AIMI development.

Disclosure statement

No potential conflict of interest was reported by the author(s).