Appraising the use of smartphones and apps when conducting qualitative medical education research: AMEE Guide No. 130

Abstract

Smartphone use is rampant in everyday life and is increasing in: patient management, teaching and learning of medicine and health research. There is untapped potential to use smartphones as research tools in MER for a range of research approaches. Qualitative research is increasingly common in medical education research (MER). Smartphone use as a research tool has not been well explored in MER and this Guide will be useful to researchers considering integrating smartphones specifically in qualitative MER. First, we discuss the potential for smartphones in qualitative MER. Then, we discuss the opportunities and drawbacks for using smartphones in qualitative MER. We then provide three principles to consider when conducting smartphone MER: communication, ethics and reflection. Next we share ten lessons that emerged from the literature and our experiences. We end by looking to the future of smartphones in qualitative MER and hope this Guide provides evidence-based information to optimise smartphone use in qualitative MER. This Guide is important as there is an urgent need to redefine ethical boundaries to account for blurred lines between personal and professional use of smartphones.

Keywords:

• Smartphones
• medical education research
• qualitative research
• apps

Introduction

Smartphones have been used in patient management, teaching and learning of medicine and health research; less consideration has been given to smartphones in medical education research (MER). Smartphones are influential in many ways – our relationships with others, daily habits such as health care (Dorsey et al. 2017) and in education (Masters et al. 2016; Valle et al. 2017; Maudsley et al. 2019). As smartphones increase connectivity, there will be the urgent need to redefine ethical boundaries to account for the blurred lines between personal and professional use. Increases in smartphone use offer new opportunities in all aforementioned areas, including MER, but simultaneously bring challenges. This Guide will provide critical reflections on the opportunities and drawbacks for using smartphones in the medical education research arena. In this Guide we focus on smartphones, however the findings and suggestions that follow may apply to other mobile devices with overlapping capabilities (e.g. tablets, smart watches). Additionally, even though we discuss medical education research, the evidence and suggestions are likely relevant to other health professions education research.

Practice points

• Identify a secure, fast communication channel with participants to facilitate remote discussion of issues and concerns during data collection.

• Communicate risks to participants particularly if they are in the clinical setting regarding privacy risks to patients and others in the clinical environment.

• Complete a data management plan to submit alongside ethics applications.

• Consistently reflect and discuss the research process and concerns with peers.

Smartphones and their software applications (commonly called “apps”) have successfully penetrated the medical education environment; the vast majority of medical students and doctors own a smartphone (Ramesh et al. 2008; Koehler et al. 2012; Browne et al. 2015; Patel et al. 2015; Raiman et al. 2017). Smartphones are used every day in patient care to assist decision-making (Mosa et al. 2012; Patel et al. 2015; Valle et al. 2017), in medical education by facilitating how medical students and residents learn (Mosa et al. 2012; Browne et al. 2015; Maudsley et al. 2019), and in healthcare research by facilitating global participation in studies of asthma, breast cancer and Parkinson’s disease (Dorsey et al. 2017). Simultaneously, smartphones have advanced general research processes (García et al. 2016). Smartphones in quantitative studies can reduce data collection costs, improve data management and maintain participant interest (García et al. 2016). Recently, authors have also recognised the use of smartphones in qualitative research – in ethnography (Beddall-Hill et al. 2011), interview research (Beddall-Hill et al. 2011; Redlich-Amirav and Higginbottom 2014) and longitudinal data collection (García et al. 2016). In medical education research (MER), smartphone use is often implicit rather than explicit, even though there are important considerations for using them in our field.

The first author is a PhD student and this Guide was inspired by her experience in developing research protocols for her first empirical study, a longitudinal narrative inquiry using smartphones as a research tool. She and her co-authors realised that the topic had not been well explored in the medical education research space. Thus far, smartphones and apps have been used as interventions to supplement learning in medical education, but there is untapped potential to use smartphones as research tools in MER for a range of research approaches.

Smartphone use in research should be critically considered and we hope this Guide will provide direction to researchers choosing to use smartphones in qualitative MER. Smartphones can indeed be used in quantitative research (Dorsey et al. 2017). Smartphones facilitate in-the-moment data collection and capturing experiences as they happen, but overt exploration of their use in qualitative MER is rarely apparent. We therefore chose to focus on qualitative MER due to our ongoing experiences and the explosive potential of using smartphones in qualitative inquiries.

In this Guide, we aim to:

• Provide readers with a critical appraisal of using smartphones as a research tool within qualitative MER through highlighting the opportunities and drawbacks that using smartphones may provide in qualitative MER.

• Consider the way forward in using smartphones in qualitative MER and share ten lessons grounded in the literature and our experiences.

Qualitative research, smartphones and medical education research (MER)

Qualitative research is a humanistic, person-centred way of uncovering reality (Holloway and Biley 2011). It is important to consider how any research tool fits into the assumptions inherent in a study’s methods and theoretical background; this should be no different when using smartphones (Hein et al. 2011). Lingard reminds us that the importance of the tools in qualitative research lies in their critical purpose rather than the tools themselves (Lingard 2007). The just-in-time capabilities of smartphones in qualitative research may reduce recall bias as participants have access to their smartphones most times in the day. This constant access facilitates instant reaction to an experience that which may otherwise fail to make it in a written diary entry (García et al. 2016)

Smartphones can help ethnographers collect multiple forms of data in order to recreate events (Hein et al. 2011) facilitating interpretation and analysis. When using ethnography – an approach that uses observations, interviews and documents to explore social phenomena from multiple perspectives (Reeves et al. 2013) – we depend on the thick description of methods and events, reflexivity and triangulation of data obtained in different ways (Reeves et al. 2013). Hein notes that smartphones contribute to the epistemological assumptions in ethnography by supporting social constructionism (Burr 1998) through insight to multiple social realities(Hein et al. 2011). Ethnography as a methodology is increasing in MER (Atkinson and Pugsley 2005) having been used to explore bedside teaching (Atkinson 2018) and the implicit versus explicit curriculum in general paediatrics (Balmer et al. 2009). Smartphones complement this method of inquiry as medical education researchers continue to explore the practices and beliefs of medical trainees and professionals.

Smartphones, likewise, can help facilitate narrative inquiry research – an approach to inquiring into participants’ experiences through their stories which considers the larger world in which the experiences are lived (Clandinin et al. 2017). Narrative inquiry is also increasing in MER (Clandinin et al. 2017) having been used to explore professional identities (Monrouxe 2009). Clandinin and colleagues note the need to create and use methods that are congruent with narrative inquiry principles. Smartphones are likely one piece of the puzzle allowing participants to story their experiences (story-telling) in real-time through audio recordings (Andrews et al. 2013; García et al. 2016). Smartphones also create a remote space where participants may feel safe to tell their stories; this safety is critical to narrative inquiry (Clandinin et al. 2017).

Opportunity | smartphones can collect different types of data

Smartphones can unlock multiple data sources to researchers, which can help them answer research questions within qualitative research (Hein et al. 2011). Smartphones are capable of replacing the functions of a digital camera, video camera, voice recorder, note pad, drawing pad, and geo-tracking; they are like ‘electronic Swiss Army knives’ (Barkhuus and Polichar 2011). Outside of MER, in an ethnographic exploration of young males’ consumer experiences, smartphones were used to record field notes and interviews and also to take photographs (Hein et al. 2011). Within MER, smartphones can facilitate data collection within the nine observational dimensions for an ethnographic study described by Reeves and colleagues. Balmer and colleagues used observations and interviews during their ethnography (Balmer et al. 2009). Smartphones can facilitate such observations through digital drawings of the location of participants and objects and how they interact in space, to record informal interviews about participants’ feelings and general field notes. Diaries are a common data collection tool within narrative inquiry studies; diaries can be written or audio-recorded. Monrouxe used dictaphones for participant audiodiaries during narrative inquiry (Monrouxe 2009) while Gordon and colleagues (2017) used smartphones for audiodaries.

Opportunity | smartphones can make data collection more efficient

Using smartphones in research could save time through the real-time digitisation, management and backup of data (Redlich-Amirav and Higginbottom 2014; García et al. 2016). Garcia et al. described being able to remotely monitor data collected via an app, which facilitated automatic notifications as reminders for participants (García et al. 2016). Smartphones are portable, multifunctional devices and allow efficient data collection by minimising the need for participants to juggle multiple devices in order to participate (Beddall-Hill et al. 2011). In narrative inquiries, it has been recognised that audio diaries benefit some participants, as less time is taken to complete them, and reduce cognitive processing versus a written diary (Fisher and Noble 2004). Lastly, smartphones can maintain participants’ interest due to the everyday integration of the smartphone in everyday life (García et al. 2016). This however does not guarantee fewer dropouts during longitudinal research (García et al. 2016). Oppenheim quite accurately describes many researchers’ early experiences in setting up at the beginning of an interview, trying to hide frantic attempts to welcome the participant, gain consent while setting up the recording equipment and making sure it is optimally placed (Oppenheim 2000). Smartphones are used by most persons daily and this familiarity could facilitate a smoother entry into research interviews. Additionally, the smartphone can be used to digitally take notes; these can then be analysed alongside the interview transcript (Beddall-Hill et al. 2011). While we did not find examples of improved efficiency using smartphones within MER, Gordon and colleagues found lower attrition rates when using smartphones for a longitudinal audio diary study exploring the trainee to trained doctor transition (Gordon et al. 2017).

Drawback | smartphones can negatively impact data quantity and quality

Using smartphones could, however, negatively impact the quantity and quality of data collected. Reduced data quantity from dead smartphone batteries could lead to participants not being able to complete their research tasks on time. Smartphone apps installed for the purpose of research could introduce software viruses and contribute to dead batteries by using excess battery power. Additionally, having participants answer questions remotely using their smartphones is risky as the notifications may fall victim to the barrage of information continually popping up on their phones that is often ignored. Some authors found that response times when answering questions using a smartphones are longer compared with using a computer due to many reasons including increased distractions (Lynn and Kaminska 2013). Regarding data quality, some academics were mystified at the use of a smartphone to record an interview as it may not be seen as a ‘serious’ research tool due to its primarily social usage, raising queries about data security (Beddall-Hill et al. 2011). This could potentially influence the quality of data obtained as participants may not take the process seriously. All the aforementioned ways in which smartphones can affect data quantity and quality are relevant in medical education research (MER). Smartphones are used every day in healthcare and medical education and research prompts may be lost among competing reasons for smartphone use.

Drawback | smartphones threaten privacy

Qualitative MER is often done in clinical healthcare settings when exploring trajectories of medical students, residents and even the influence of patients on medical education. The importance of protecting privacy is increasing. General Data Protection Regulation (Official Journal of the European Union 2016) laws are being enforced in the European Union and worldwide. This has resulted in changes to many institutional policies. Within many health care institutions, strict policies on the use of smartphones in the clinical context are being implemented (John 2018). These policies stipulate security requirements for smartphones (The University of Alabama at Birmingham 2016), and the requirement of consent for taking photographs or identifiable information of anyone (Health Facilities Scotland 2008). These policies are equally, if not more, relevant to medical education researchers as we conduct research in these multifunctional environments.

Some participants may lose awareness that they are being observed and recorded (Beddall-Hill et al. 2011; Hein et al. 2011). Due to smartphones’ ubiquitous nature, using them in qualitative research could lead to the researcher becoming invisible in the environment leading to ‘covert surveillance’ (Beddall-Hill et al. 2011) – smartphones may not always be recognised to be a research tool. Even though this might be considered an advantage by reducing observer effect, it could mean that data are collected that participants would otherwise prefer to be private (Beddall-Hill et al. 2011). Additionally, apps could present a security risk to participants as not all apps may have standardised data security procedures and therefore may not reach standards required for healthcare mobile device policies for security.

Within MER, when collecting data in the clinical environment, participants may provide consent that allows us to observe and interview them informally. However, bystanders in the complex clinical environment may be unintentionally recorded; their individual consent may be logistically difficult to gain. As smartphones are so integral to everyday life, bystanders may not recognise that recording is occurring, which augments the risks. How can the voices in the background of researchers’ field audio-notes and persons in the background of photographs be protected? Anonymising photographs is more complicated than simply cropping and blurring images as digital photographs on smartphones often carry information such as date and exact location, which may make it possible to triangulate information, thus eroding anonymity (John 2018).

Drawback | smartphones introduce distractions

Smartphones introduce another source of interruption, multitasking and distraction into the hospital environment (Katz‐Sidlow et al. 2012). Smartphones enhanced observations in an ethnographic study within marketing (Hein et al. 2011), however, this benefit could be overshadowed when brought to the healthcare context due to the risks involved. Distraction of clinicians by smartphones occurs when one’s primary task is interrupted by any use of their smartphone (McBride 2015). The term ‘distracted doctoring’ has been coined by Papadakos highlighting the dangers of seductive mobile devices (Papadakos and Bertman 2017). These interruptions can have significant consequences on both patient care (Halamka 2011; Wu et al. 2013), and learning (Fox et al. 2009) as there is the potential for missing important information (Katz‐Sidlow et al. 2012). When using smartphones for MER, prompts may be used to stimulate participant responses. A review found that distraction by social connectivity could affect the impact mobile devices have when used in the clinical setting for learning (Maudsley et al. 2019). Using smartphones to conduct MER intensifies distraction in the clinical setting.

Principles of smartphone use in qualitative medical education research (MER)

Based on the aforementioned opportunities and drawbacks to using smartphones, we offer three principles that a medical education researcher using smartphones as a research tool should consider during protocol development, ethics application, data collection and storage.

Box 2 Ten lessons for using smartphones in qualitative medical education research.

1. Beyond innovation, smartphones fit the principles of qualitative MER.

2. Smartphones can collect different types of data (audio, video, notes, photos, location)

3. Smartphones can make data collection more efficient

4. Smartphones can negatively impact data quantity and quality

5. Smartphones threaten privacy

6. Smartphones can perpetuate distractions in the clinical environment

7. Constant communication of risks and benefits to participants via an open agreed upon communication channel is important.

8. Institutional review boards have a responsibility to consider risks when using smartphones as they provide approval

9. A data management plan is critical

10. Reflection, peer discussion and participant feedback will launch the way forward in using smartphones in MER.

Communication

Communication between researchers and participants should take place through previously agreed upon channels, which should include secure apps or institution emails. This facilitates fast resolutions of technical issues and an outlet for participants to voice concerns. Communication is key to reassure participants of their protection. One such opportunity for this is during consent where researchers can discuss the flow of data. In a case of academics who were fearful of using a smartphone to record interviews due to concerns about threatened data security (Beddall-Hill et al. 2011), reassurance and information could dispel these perceptions. It should be made clear when the device is recording and that it is not transmitting data during interviews. Sharing data with participants could function as member checking but also shows participants what data has been collected and facilitates dealing with any research use of data that participants are not comfortable with (Beddall-Hill et al. 2011). While in principle, participants should be free to withdraw part or all of their data at any time during the research process, it should be communicated that this can only be done if possible. For example, if their data has already been integrated through preliminary analysis, you can assure them of deletion of their raw data but it will not be possible to remove their existing input from aggregated growing analyses, although their privacy would be maintained. In the era of co-creation, including participants during protocol development could be potentially helpful to ensure procedures are acceptable for participants using their smartphones. Lastly, at an institutional level, similar to the proposed training for medical students and junior doctors on smartphone use (Maudsley et al. 2019), similar training for MER researchers would be beneficial.

Ethics

Institutional review boards (IRBs) have a role to play in policing protocols using smartphones to collect MER data. As smartphone use in research increases, it will be imperative that IRBs actively consider the implications for using smartphones to do MER. The consent process should be informed by a deep understanding of the implications of smartphone use and participants should be fully informed of the unique risks. Health care organisations and researchers should also consider whether participants should have institution-provided smartphones or if it is sufficient to use individual personal devices within institution policy. Research using mobile technology in education could consider moving from ‘permission-seeking’ modes of ethical approval toward iterative, incremental models of ethical approval when using technology like smartphones (Lally et al. 2012). Such a model would provide the flexibility necessary when conducting qualitative MER. Iterative ethical approval would allow researchers to efficiently gain further access as necessary as their research unfolds.

Since the launch of data protection protocols such as the European Data Protection Regulation, complete data management plans (European Research Council 2017) (see Table 1 and Box 1 for examples) should be critically considered, documented and submitted alongside ethical applications. This is especially important when designing a study using smartphones as a research tool. Storing data on internal smartphone storage or using native apps threatens participant privacy. As such, all data should be uploaded promptly and stored on secure app servers and those regulated by institutions. Regarding protecting ‘unintentional participants’, researchers and participants should avoid the voices, faces or identifiable attributes in video, audio and photos of persons and places that have not provided permission to be included in a study. Where their inclusion is inevitable, as much as is possible, their informed consent should be obtained at that point.

Box 1 Sample data management data flow system.

1. Sanne will record audio diaries securely from September 2018 until March 2019 via her smartphone.

2. Sanne will take part in two participatory interviews with AA which will be audio recorded and will produce a concentric circles network map. The work area will be videotaped as well.

3. Sanne’s audio recordings will be obtained by AA who upon downloading will assign Sanne a code- 001.

4. Sanne’s audio recordings and interviews will be saved using her code e.g. 001-diaries.mp3; 001-interview1.mp3; 001-interview2.mp3; 001-interviewvideo.mp4; 001-ccmap.jpg.

5. Sanne‘s audio recordings and interviews will be sent to a professional transcription service (as 001-diaries or 001-interview1). The transcription service will sign a confidentiality form. The transcript names will follow the naming system for the recordings (e.g. 001-diaries.docx)

6. AA will de-identify the transcript and delete the identifiable version. This will be saved as 001-diaries_anon.docx.

7. AA will analyse Sanne‘s data alongside all participants.

8. AA will delete the audio recordings, video recordings (of the workspace) and interview recordings following analysis of all participants.

Table 1. Sample Data management plan summary.

	Description	Origin	Format	Size	Data utility
Audio files	Audio files from interviews and audiodiaries	Audiodiaries and interviews	.mp3	216 MB	Will be deleted after analysis
Video files	Video files from participatory interviews	participatory interviews	.mp4	40 Gb	Will be deleted after analysis
Interview & Audio diary transcripts	Interview transcripts from interviews and audiodiaries	Audiodiaries and interviews	.docx	6 Gb	Will be kept on an encrypted hard drive and at the end of the study uploaded to approved institutional database or research direct for duration as determined by ethical board
Participant-produced diagrams	Participant-produced diagrams from participatory interviews	Participatory interviews	.jpeg	480 MB	Will be stored on approved institutional database once anonymised for duration as determined by ethical board

Burning questions regarding the ethics of using smartphones in qualitative MER remain. (1) When should raw data be deleted when collected with smartphones? Whatever the decision, this should be disclosed in the ethics protocol and communicated to participants. (2) Is there a risk introducing a third party (e.g. apps) whereby we depend on them to delete the data from their servers? Just as many transcription companies sign confidentiality agreements with institutions, app developers should be required to do the same. (3) Do IRB blanket statements such as ‘locking data in a locked cabinet or password protected computer’ need to be reconsidered in the era of using smartphones as research tools? (4) Are there any legal concerns with data that were taken for research purposes but may be necessary for legal proceedings as evidence (e.g. tracking and location data)? Lastly, amidst global participation in health research through smartphone use (Dorsey et al. 2017), (5) is a global IRB system possible, as this could ultimately extend to MER using smartphones?

Reflection

Reflection is important in medical education practice (Mann et al. 2009) and research (Ng et al. 2015) and has the ability to change practice and understanding (Mann et al. 2009). Qualitative research often includes maintaining audit trails and research diaries. Through reflection, medical education researchers should also be flexible in their decisions for data collection to minimse risks and challenges. It may be beneficial to discuss with peers and potential participants from your sampling frame to help inform your research design choices. Published reflective papers report on the process choices researchers make when using smartphones in research, with explicit examples critical decisions made by researchers and questions still to consider (Beddall-Hill et al. 2011; García et al. 2016). These papers are examples of how reflection could impact future research practices and protocols when using smartphones (Beddall-Hill et al. 2011; García et al. 2016). Lastly, asking participants to discuss their experiences with the research process (Crozier and Cassell 2016) could be another way to improve future MER using smartphone research. Ultimately, when conducting MER, such flexible, intentional data collection could increase participant adherence while protecting both participants and patients. In addition to these principles, Box 2 describes ten lessons when using smartphones in qualitative MER grounded in the literature and our experiences.

The future of smartphones in MER

In these days of internationalisation of medical education, MER could follow suit. Barring challenges and ethical considerations, could technology provide automatic translation that minimises the need for participants and researchers to speak the same language? Can medical students in the United States, the Caribbean, the UK, Europe, Australia, and Asia potentially take part in the same study, regardless of language, providing they meet the inclusion criteria? Just as apps have erased the geographical boundaries placed on health research and allowed global participation, especially for diseases with lower prevalence, could smartphones and apps be the start of true multi-institutional globalised MER?

While researchers can use existing smartphone apps for their research, often the cost attached to existing apps is quite high and could reduce use in those with limited research budgets. Many apps that can facilitate research data collection are created for market research; commercial companies have high budgets. There is therefore an opportunity for the creation of low-cost research apps tailored for efficient and secure collection of audio, images, video, survey data while allowing for geo-tracking data in educational contexts.

It is rare to find meta-research on the impact of using smartphones or technology in research in medical education. McLeod and colleagues adopted socio-materiality – how the human and non-human relate to organise, allow and constrain social interactions – to explain that materials (e.g. mannequins, stethoscopes and technology) are not a neutral elements to human interaction in learning but impact agency and meaning making (MacLeod et al.; MacLeod et al. 2015). Future research could further explore socio-materiality theories to examine how smartphones and other mobile devices influence process and outcomes in MER.

Conclusion

The aim of this Guide was to provide guidance to design, conduct and reflect on the use of smartphones in medical education research. It is critical for researchers to share experiences in the academic literature to facilitate global progression of research processes. Smartphones can enhance the research purpose and fits the principles of qualitative MER. Researchers must however be mindful of using smartphones in MER as they can affect the research data, threaten privacy and perpetuate distractions. We suggest constant communication with participants, being reflective as a researcher and challenge institutional review boards to recognise their responsibility in monitoring the use of smartphones in MER. We look forward to seeing intentional, reflective use of smartphones in medical education research in the future.

Acknowledgements

We would like to thank the reviewers for stimulating comments on an earlier version of this Guide.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

Anique Atherley is supported by a scholarship through the Western Sydney University as part of a joint PhD collaboration between Western Sydney University and Maastricht University.

Notes on contributors

Anique Atherley

Anique Atherley, MPH, is a public health physician and joint PhD candidate at the School of Health Professions Education (SHE) Maastricht University and Western Sydney University. Her research interests are in trainee support and wellbeing and she is currently exploring the social aspects influencing the transition from pre-clinical to clinical training.

Wendy Hu

Wendy Hu, PhD, trained as a family physician before becoming a medical educator. Her research interests include educational change and innovation, research and career development, qualitative and participatory action research methods.

Pim W. Teunissen

Pim W. Teunissen, PhD, is a gynaecologist at the Amsterdam University Medical Centers and a professor of medical education at Maastricht University. His research focuses on how healthcare professionals learn through work and as part of his research he has undertaken longitudinal qualitative research using smartphones.

Iman Hegazi

Dr. Iman Hegazi, PhD, is director of Medical Education at Western Sydney University. She has expertise in quantitative and mixed methods research, and her areas of research interests include research that supports medical students’ learning and professional development, as well as research in curricular design and pedagogy.

Diana Dolmans

Diana Dolmans, PhD, is educational scientist and a professor at the Department of Educational Development and Research and School of Health Professions Education (SHE), Faculty of Health, Medicine and Life Sciences, Maastricht University, The Netherlands, with a special interest in innovative learning arrangements.