Series: Practical guidance to qualitative research. Part 5: Co-creative qualitative approaches for emerging themes in primary care research: Experience-based co-design, user-centred design and community-based participatory research

Abstract

This article, the fifth in a series aiming to provide practical guidance for qualitative research in primary care, introduces three qualitative approaches with co-creative characteristics for addressing emerging themes in primary care research: experience-based co-design, user-centred design and community-based participatory research. Co-creation aims to define the (research) problem, develop and implement interventions and evaluate and define (research and practice) outcomes in partnership with patients, family carers, researchers, care professionals and other relevant stakeholders. Experience-based co-design seeks to understand how people experience a health care process or service. User-centred design is an approach to assess, design and develop technological and organisational systems, for example, eHealth, involving end-users in the design and decision-making processes. Community-based participatory research is a collaborative approach addressing a locally relevant health issue. It is often directed at hard-to-reach and vulnerable people. We address the context, what, why, when and how of these co-creative approaches, and their main practical and methodological challenges. We provide examples of empirical studies using these approaches and sources for further reading.

Keywords:

• Primary care
• co-creation
• qualitative research
• patient and public involvement
• eHealth

KEY POINTS ON CO-CREATIVE QUALITATIVE APPROACHES

• Experience-based co-design seeks to understand how people experience a health care process or service.

• User-centred design is an approach to assess, design and develop technological and organisational systems, for example, eHealth, involving end-users in the design and decision-making processes.

• Community-based participatory research is a collaborative approach addressing a locally relevant health issue. It is often directed at hard-to-reach and vulnerable people.

Box 5. Sources for further reading on stakeholder analysis and management, patient and public involvement and three co-creative qualitative approaches.

Web sources on stakeholder analysis and management

• NHS England and NHS Improvement www.england.nhs.uk/wp-content/uploads/2021/03/qsir-stakeholder-analysis.pdf.

• State of Victoria, Department of Health and Human Services, State of Victoria, Australia www.dhhs.vic.gov.au/publications/stakeholder-engagement-and-public-participation-framework-and-toolkit.

Web sources on patient and public involvement

• INVOLVE www.involve.org.uk.

• James Lind Alliance www.jla.nihr.ac.uk.

• Strategy for Patient-Oriented Research (SPOR) www.cihr-irsc.gc.ca/e/41204.html.

• Gordon and Betty Moore Foundation and the American Institutes for Research www.air.org/project/roadmap-guides-patient-and-family-engagement-healthcare.

• Patient‐Centered Outcomes Research Institute https://www.pcori.org/.

• Health Issues Centre https://healthissuescentre.org.au/health-services/consumer-engagement-guide.

• National Framework of Consumer Involvement in Cancer Control https://canceraustralia.gov.au/sites/default/files/publications/national_consumer_framework_web_504af020f2184.pdf.

Web sources on budgeting involvement

• The SPOR Networks in Chronic Disease and their Patient Partners https://diabetesaction.ca/wp-content/uploads/2018/07/TASK-FORCE-IN-PATIENT-ENGAGEMENT-COMPENSATION-REPORT_FINAL-1.pdf.

• INVOLVE https://www.invo.org.uk/wp-content/uploads/2013/07/INVOLVEMHRNBudgeting09Jul2013.pdf.

• National Institute for Health Research https://www.nihr.ac.uk/documents/payment-guidance-for-researchers-and-professionals/27392.

Experience-based co-design

• Bate B, Roberts G. Bringing user experience to health care improvement. Oxford: Radcliffe Publishing; 2007.

• King’s Fund www.kingsfund.org.uk/projects/ebcd.

• Boyd H, McKernon S, Old A. Health Service Co-design Working with patients to improve healthcare services guide and toolkit https://www.wdhb.org.nz/assets/Uploads/Documents/45f463d911/rttc_health-service-co-design-by-h-boyd.pdf.

• Melchior I, Moser A, Stoffers S, et al. EBCD Methodiek: Versterken van de participatie van oudere patiënten en naasten ter verbetering van de kwaliteit van zorg http://www.innovatiesindezorg.eu/files/4914/8853/9220/EBCD_toolkit_methodiek_DEF.pdf.

User-centred design

• Neuhauser L. Integrating participatory design and health literacy to improve research and interventions. Inf Serv Use. 2017;37(2):153–76.

• Shah SGS, Robinson I. User involvement in healthcare technology development and assessment: structured literature review. Int J Health Care Qual Assur. 2006;19(6):498–513.

• Tripp SD, Bichelmeyer B. Rapid prototyping: an alternative instructional design strategy. Educ Technol Res Dev. 1990;38(1):31–44.

Community-based participatory research

• Cargo M, Mercer SL. The value and challenges of participatory research: strengthening its practice. Annu Rev Public Health. 2008;29(1):325–50.

• Israel BA, Eng E, Schulz AJ, et al. Methods for community-based participatory research for health. 2nd ed. San Francisco (CA): Jossey-Bass; 2012.

• Viswanathan M, Ammerman A, Eng E, et al. Community-based participatory research. Evidence Based Practice Centre Contract No. 290 – 02 – 0016, Agency for Healthcare Research and Quality; 2004. Available from: http://www.ahrq.gov/clinic/epcsums/cbprsum.htm.

• Wallerstein N, Duran B, Oetzel J-G, et al. Community-based participatory action research for health. San Francisco (CA): John Wiley; 2018.

Introduction

Over the years, in our supervisory work, we have noticed that qualitative research tends to evoke many questions and challenges. This article, the fifth in a series aiming to provide practical guidance for qualitative research [1–4], introduces three co-creative (and mostly) qualitative approaches for addressing emerging themes in primary care research: experience-based co-design to improve the quality of care, user-centred design to develop and evaluate eHealth resources and community-based participatory research to improve local health collaboratively.

A changing primary care

Primary care faces a changing context, including the increasing provision of chronic care and elderly care, shared decision-making and proactive care planning, e- and mHealth, preventive and community care, and interprofessional collaboration with nurses, paramedics and relevant services [5–8]. These changes have consequences for primary care research. By nature, general practitioners are co-creators in working with their patients and other professionals on seeking solutions for complex health issues in daily practice. However, the ‘explicit’ idea of co-creation may not be very familiar to general practitioners, depending on their national policy context [9].

Co-creative approaches

In research, co-creation means an iterative and non-linear process throughout the research continuum and the collaborative generation of knowledge by academics working alongside stakeholders [10]. We use the term co-creative qualitative approaches as an umbrella concept. The three co-creative approaches have different origins, core principles, goals and stakeholders (Box 1) but they share common ground. They start from solving a problem in practice, supporting stakeholder involvement and equal partnerships among the stakeholders, empowering vulnerable people/communities and bridging the gap between practice and research. They complement the scientific knowledge and expertise of general practitioners and other primary care professionals. They provide insights into needs, experiences, aspirations, stakes and changes from a multi-perspective. Co-creative approaches are relatively novel to primary care but they are rather familiar in other settings such as hospitals, psychiatric care or social care and to disciplines such as nursing, sociology or developmental research.

A growing body of literature suggests that co-creation can ultimately result in improved efficiencies and outcomes, increased patient satisfaction and trust and greater capacity for research [11]. It is a bottom-up approach to improve health services and the population’s health that general practitioners and primary care professionals serve [12]. We are aware that in scientific literature many different terms are used that fit our notion of co-creation such as co-design, co-production, partnership approaches, stakeholder engagement, patient and public involvement, and participatory research [13].

Stakeholders

Co-creation aims to define the (research) problem, develop and implement interventions and evaluate and define (research and practice) outcomes in a partnership with those who have a stake. For this article, we define stakeholders as those who have an explicit interest in a particular practice, process, decision and/or health outcome and the supporting evidence. Common stakeholders in primary care research are patients, family carers, researchers, care professionals (including managers), advocacy organisations and other relevant stakeholders (e.g. local policymakers, insurance companies). However, every research project using co-creation requires a stakeholder analysis at the stage of defining the research problem. The initial project members start with a brainstorm of all possible stakeholders and then prioritise them according to their power over, influence on, and their interest in the problem and the project. They explore their motivations, interests, positions, expectations and expected benefits [14].

Target audience and content of this article

This paper is relevant for researchers who want to use these co-creative designs and general practitioners who will increasingly read articles using this methodology. They might consider our introduction a ‘first date’. We address possible questions about the context and the what, why, when, and how of these approaches and their main practical and methodological challenges. We provide examples of published empirical studies in primary care and other health care domains and sources for further reading.

Experience-based co-design to improve the quality of care

Context

Providing high-quality care services is the goal of every primary care professional. Classic ways for improving quality of care are based on evaluating biomedical and psychosocial outcomes, functioning and cost-effectiveness [15]. In recent years, there has been a shift towards quality of care improvement based on patient experiences by actively involving patients, family carers and the public in the design process of health services. An innovative approach to improving the quality of care services is experience-based co-design [16]. Published empirical studies using this approach include:

• Empowering people to help speak up about safety in primary care: using co-design to involve patients and professionals in developing new interventions for patients with multimorbidity [17].

• Improving the experience of older people with colorectal and breast cancer in patient-centred cancer care pathways using experience-based co-design [18].

• A road less travelled: using experience-based co-design to map children’s and families’ emotional journey following burn injury and identify service improvements [19].

What?

The goal of experience-based co-design is to facilitate collaborative work between patients, family carers and professionals towards a common goal – to improve the quality of care. This approach is a form of action research that seeks to capture and understand how people experience a process or service [16]. An experience-based co-design approach deliberately draws out the subjective, personal feelings of patients, family carers, the public and professionals to identify touchpoints – key moments that shape a person’s overall experience. Experience-based co-design enables patients, family carers, the public and professionals – as partners – to co-design services or care pathways to improve the quality of care based on experiences.

Why and when?

Health care professionals often think they have the unique expert knowledge to improve care processes and create value for patients [16]. Berwick [20] proposed shifting away from professional dominance to a greater focus on co-creation. There is a growing interest in patient and public involvement, often triggered by health policy initiatives and support for co-creating value across health care.

Patient and public involvement entail the active participation of patients, family carers and the public in planning, delivering and evaluating health care services. It involves the ongoing process of patient and service user initiation, building reciprocal relationships, co-learning and re-assessment and feedback [21]. Involving patients can happen at the individual level – in decisions about individual care and treatment – and at the collective level – in decisions about the delivery of care services [22].

Based on an adapted version of Arnstein’s [23] participation ladder, we distinguish five levels of involvement: information, consultation, advising, partnership and citizen control (Box 2). In the primary care context, by the active involvement of patients, family carers, the public and professionals, co-design connects the knowledge of stakeholders to address quality of care priority concerns.

How?

Experience-based co-design projects to improve the quality of care typically last 12 months [24], and the process contains discovery and co-design phases [25] (Figure 1). The start involves setting up a core group that runs the project and recruiting a researcher.

Figure 1. Phases and teams in experience-based co-design. Based on Bate and Roberts [16].

The discovery phase begins with observations by the researcher that provide valuable insights into how the service to be improved works. These insights are helpful to sensitise researchers for the subsequent interviews. The discovery phase proceeds with discovery interviews, which aim to explore and learn from the impact of illness on patients’ and family carers’ everyday lives. Discovery interviews – conducted with patients, family carers and professionals about their experiences with a health service – produce knowledge about needs that may significantly impact care, recovery and wellbeing. The touchpoints are identified based on the experiences of participants. Interviews are filmed to develop a video to trigger a dialogue between patients, family carers and professionals.

In editing the video, researchers identify areas for quality improvement, often following a certain chronology, for example, diagnosis, treatment and follow-up. The video is an important catalyst in the co-design process as the visualisation of patient experiences helps (re)connect people with similar experiences and stories and offers an emotionally and cognitively powerful starting point for the co-design process [26].

Next, the various areas for improvements are prioritised in the co-design team: separately within each group (mostly patients, family carers and professionals) and then with all the groups together. Each meeting starts with presenting the various areas for improvement, illustrated by videotaped quotes from the previous phase. Then, the participants jointly choose three or four areas as the key priority for quality improvement.

In the co-design phase, co-design quality improvement groups – small working groups of patients, family carers and professionals – design and implement quality improvement activities to target the key priority issues highlighted at the co-design meetings.

Finally, the improvements are evaluated in a closing event, and the gains are communicated and celebrated by the co-design team. Experience-based co-design is a change approach and process that improves health care and scientific insights into change processes.

User-centred design to develop and evaluate eHealth resources

Context

eHealth is the use of (digital) information and communication technology (ICT), in particular internet technology, to support or improve health and health care [27]. It offers a comprehensive promise for a better quality of primary care and high-quality data for quality assurance, education and research [27]. Innovative but valid research methodology is a prerequisite for the ongoing success and sustainability of eHealth [28]. End-users need to be involved in the development and implementation of eHealth via co-creation processes, and design should be mindful of vulnerable groups and eHealth illiteracy. An appropriate approach is user-centred design.

Published empirical studies using this approach include:

• User-centred design of a tablet waiting room tool for complex patients to prioritise discussion topics for primary care visits [29].

• Development of a mobile clinical prediction tool to estimate future depression severity and guide treatment in primary care: user-centred design [30].

• Creating guardians of physiologic birth: the development of an educational initiative for student midwives in the Netherlands [31].

What?

The goal of user-centred design, stemming from social and technological design sciences, is to develop eHealth technologies with very high usability. It is a method to assess, design and develop technological and organisational systems, which involves end-users in design and decision-making processes [32]. Its key features are rapid cycles of problem identification and solution creation, in-depth understanding of end-user characteristics, the influence of end-users on how a design takes shape, iterative evaluation during the entire development process, and accounting for the implementation conditions from the beginning [33]. Ideally, the user-centred design considers all potential stakeholders, for example, patients, family carers, professionals and staff, ICT designers, representatives of the health care system and researchers responsible for the content of the technology. However, the end-users are mostly patients, family carers, professionals and staff.

Why and when?

Developing eHealth often uses new technologies and services for users experiencing complex health problems. User-centred design supports developing eHealth by understanding and solving the problem simultaneously and iteratively [33]. If end users are engaged to create and implement interventions themselves, the interventions will incorporate nuanced factors and consider social, structural and environmental determinants of health that affect the end-users. Without this input, these elements would not have been evident to researchers or professionals [33]. Apps developed with user-centred design have reported improved user acceptance, face validity, user-friendliness and uptake [30]. Critical for eHealth’s uptake and continuous use is that it is user-friendly, meets end users’ motives, values, needs and abilities and fits into the organisation of care.

How?

User-centred design uses mostly qualitative or mixed methods [33]. The problem development cycle involves gathering and analysing data from users and other sources to define problems and needs. The solution development cycle involves the generation of ideas to build and test prototypes with end-users. Within and between these cycles, there are iterative feedback loops. Researchers and developers finalise and deploy an eHealth solution when it meets the end users’ key requirements.

A specific type of user-centred design is rapid prototyping, which is often used for developing educational (e-learning) programmes [34]. It involves overlapping stages of needs assessment, input and feedback from key stakeholders in designing subsequent prototypes to reach a final prototype for implementation and evaluation.

Various descriptions exist of the stages within each of the problem and solution development cycles in the user-centred design process, for example, a five-stage process consists of concept, design, testing and trials, production and deployment stages with end-users participating in all stages, except production [33,35] (Box 3). The methods most used for involving end-users are usability tests, interviews and questionnaire surveys. Since capturing end users’ perspectives at various stages depends on the method applied, selecting an appropriate method is important [36]. This also requires in-depth consideration of all users and their activities, their actual daily environment and their functional limitations, innumeracy and skills [35]. For example, considering midwives’ high workloads, researchers chose individual interviews and written feedback rather than focus groups [31].

For less available end-users, including elderly people and people with disabilities and/or special needs, substitutes called ‘user surrogates’ might be involved [35]. A user surrogate is a user who has the knowledge or authority to perform tasks on behalf of another user. User surrogates report on what they know about the user or by role-playing how the user would behave.

Community-based participatory research to improve local health collaboratively

Context

Primary care professionals often provide care to vulnerable groups, such as cultural minorities and deprived communities. They care for patients who struggle with health problems affected by their lifestyle choices, biography, life events, educational level, socioeconomic situation and social and physical environment. A research approach to address health disparities is community-based participatory research. It has often been used for hard-to-reach or very vulnerable communities. We define community as a group of people with common interests – such as shared values, culture, customs or identity or as all people living in a particular geographical area – such as a neighbourhood, district or local area, or as groups of people with a common interest living in a geographical area.

Published empirical studies using this approach include:

• Participatory development and pilot testing of the Makasi intervention: a community-based outreach intervention to improve sub-Saharan and Caribbean immigrants’ empowerment in sexual health in France [37].

• Implementing community-based participatory research in the study of substance use and service utilisation in Eastern European and Turkish communities in Belgium [38].

• A community-based participatory research on improving the integration of health and social in the Netherlands [39].

What?

The goal of community-based participatory research is to educate, improve practice or bring about social change. It is a collaborative approach to research, which seeks to address a locally relevant health issue [40]. What is unique to community-based participatory research is its emphasis on the diverse community partners involved and on striving for equal participation and ownership, reciprocity, co-learning and change [41]. This approach engages researchers and community members in all aspects of the research process, including needs assessment and agenda-setting, decision-making, capacity building, knowledge generation and the implementation and dissemination of findings [42,43]. Because of its focus on community engagement, community-based participatory research allows community partners working with academic partners to identify and address health problems affecting their communities (Box 4). It fosters social connections that can lead to change and produces knowledge that can lead to action [44].

Why and when?

Community participation in primary care has its origins in the Alma-Ata Declaration of 1978 [45], which stated that people have the right and duty to participate individually and collectively in the planning and implementation of their health care. The benefits include the following: ensuring that the research topic reflects a major issue identified by the community; improving the quality, validity and sensitivity of the research by drawing upon community wisdom, thus promoting trust between communities and researchers; improving the translation of research findings into policy and practice; and enhancing uptake of the research findings by community members [42]. Researchers together with the local community might help address the well-described issue of ‘ivory tower’ research and have a social impact in the ‘real world’ [46].

How?

Community-based participatory research can employ diverse methodologies, study designs and data collection methods, for example, qualitative case studies, environmental assessments, mixed methods research and randomised controlled trials. In general, there are seven phases [44] (Figure 2). Researchers and the local community work together as partners.

Figure 2. Phases of community-based participatory research. Based on Israel et al. [44].

The first phase is forming a community-based action research partnership involving activities to identify potential non-academic partners. Partners might include the following: patients; interpersonal support networks, including family members, mentors and friends; members of the general public who are not patients but who support or believe in the issue; those who interface directly with patients and/or patients’ interpersonal networks, including practitioners, health professionals and administrators; and others, such as service providers and policymakers. The activities aim to build trust and relationships, establish operating norms and community-based action research principles to ensure equity and power-sharing and create an infrastructure for the research [43].

The second phase entails assessing community strengths and dynamics. This involves activities such as discovering and assessing the strengths and resources in the community, key cultural and historical dimensions, influential organisations, power relationships in the community and partners to be involved to ensure that the community voice is heard [45].

The third phase is identifying priority local health concerns and research questions. Key activities are to identify the major health problems that community partners experience as affecting the community and that need to be addressed and prioritise health concerns and their contributing factors. Finally, the researchers and community partners formulate the key research questions for the study.

The fourth phase involves collaboratively designing and conducting interventions and/or policy research. This involves prioritising the research questions and goal, selecting the research design and data collection methods and deciding the most appropriate intervention. In addition, it involves determining how to carry out the research design and the intervention selected and, finally, agreeing on the evaluation.

The fifth phase is feeding back and interpreting the findings within the community. This involves data analysis: sharing (preliminary) findings from surveys, in-depth interviews, focus group discussions, etc. and engaging the community partners to make sense of the findings.

The sixth phase is disseminating and translating the research findings. This involves identifying the most important findings for sharing with the community, the community’s role in communicating and translating the findings, disseminating the findings into broader interventions and policy changes and publishing the research results. This might lead to the formation of a community-based action research partnership.

All phases share an underlying continuous process of maintaining, sustaining and evaluating the community partnerships. The researchers and community partners are reflective about their working relationships and shared long-term goals and capacities. All these approaches might integrate some mixed-methods research such as pilot testing or proof-of-concept [47].

Challenges in applying co-creative approaches

Drawing on our experience with co-creative research projects and based on the methodological and empirical papers we referenced, we provide a brief overview of practical and methodological challenges that such research projects may face.

Practical challenges

 

Unclear purpose and expectation

Stakeholders and researchers need to understand what the project goal is and why the process of co-creation is essential [40,46]. It helps to work step-by-step and establish a shared starting point in each phase of the co-creative approach. At the time, researchers, especially the principal investigator, need to keep track of the scope and expected outcomes of the project [44].

Skills, capacities and financial resources

Some stakeholders might lack the skills to adopt a view beyond their personal stakes [18]. Researchers need to make optimal use of the various stakeholders’ capacities in different project phases or research activities. Some stakeholders, especially patients and vulnerable community members, might lack the resources to participate in meetings [43,48], for example, affording public transport or self-confidence to speak up. Researchers, especially principal investigators, need to balance preferred ways of engagement in co-creation, meaningful activities to stakeholders and the available time, enabling measures, time demands and financial resources [44]. It is important for principal investigators to budget for stakeholder involvement in their research projects adequately. When applying for research grants, stakeholder involvement, especially patient and public involvement should be explicitly budgeted (Box 5). Funders often check to ensure budgets have been thoughtfully allocated to promote meaningful participation.

Multiple perspectives and conflicts

Various data types are collected during in-depth interviews, focus-group discussions, workshops etc., from different sources, for example, patients, professionals, and managers. The integration and prioritisation of these perceptions and concerns are challenges for stakeholders and researchers [49]. Conflicts may occur due to different decision-making styles, values, priorities, use of language, engagement history, perceived power imbalance, competition or lack of feedback on stakeholders’ input [40]. Researchers need to foster a democratic process of dialogue, shared responsibility and positive relationships [38,39,41,46].

Methodological challenges

Methodological quality

Most stakeholders are primarily interested in how the project will address their perceived health issues, whereas researchers also strive for generating valid scientific knowledge. Researchers need to be flexible in all research steps in balancing practical relevance, methodological quality, and timing [40,44,49].

Research team

Co-creation requires various competencies in the research team. Usually, individual researchers in multidisciplinary teams bring in their specific expertise in certain research phases or steps. Researchers need to compose a research team that integrates competencies from different health disciplines, methodological competencies and social competencies in guiding all stakeholders through the co-creation process [41,49]. The flexible, time-consuming and sometimes unexpected nature of co-creation might cause time pressure [43]. Researchers need to balance getting things done and reflecting on the research process, methodological quality, stakeholder relationships and their own role [18].

Digital research

Other qualitative approaches, such as netnography [50], use of various formal and informal online data sources, digital data collection methods and interactive digital tools are fully in development. Digital research might support efficient data collection and management but might also bring inequality risk, for example, exclusion of people lacking digital skills [51]. Researchers need to consider ethical and methodological issues in digitalisation in qualitative research because it might be a promising way forward in co-creative approaches.

Further reading

We hope that our introduction to co-creative approaches in qualitative research functions as an appetiser for researchers facing emerging themes in primary care. A deeper understanding is necessary to apply these comprehensive approaches in research projects. Therefore, we provide sources for further reading (Box 5).

Acknowledgements

The authors thank the junior researchers who have participated for the last 10 years in the ‘Think tank on qualitative research’ project, a collaborative project between Zuyd University of Applied Sciences and Maastricht University, for their pertinent questions. The authors are grateful to Darcy Ummels for proofreading our manuscript and providing valuable feedback from the novice perspective.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

Box 1. Summary of the origin, core principles, goals and key stakeholders of three co-creative qualitative research approaches.

Box 1. Summary of the origin, core principles, goals and key stakeholders of three co-creative qualitative research approaches

	Experience-based co-design	User-centred design	Community-based participatory research
Origin	Participatory designs, ethnography, phenomenology, design science, management science	Computer science and sociology	Psychology, sociology and pedagogy
Core principles	Understanding of patient experience for improving and redesigning health services Stakeholders collaborate on the redesign of process	In-depth knowledge of what matters to the intended users and their abilities. Support services must be designed to support eHealth	Perceived power imbalances, mostly the very vulnerable or minorities are affected Sustainable change depends on mutual trust, power-sharing and continuous engagement
Goals	Improved patient experience of health services	eHealth that is acceptable, usable and fits the intended users. eHealth that is feasible and supports work processes to deliver efficient care	Local and scientific learning Integrated knowledge transfer Change that reduces inequalities
Key stakeholder	Patients, family carers, professionals, managers, quality officers, change facilitators, researchers	Patients, family carers, professionals ICT designers, support staff, representatives of the health care system, researchers	(Vulnerable) members of the community, health and social care providers, advocate groups, policymakers, researchers

Box 2. Five levels of patient involvement. Based on Arnstein’s participation ladder [23].

Box 2. Five levels of patient involvement. Based on Arnstein's participation ladder [23].

Information	Researchers provide information to patients and the public.
Consultation	Researchers seek views of the patients and the public.
Advising	Researchers selectively include patients and the public in decisions and selectively adopt the advice given.
Partnership	Research teams share responsibilities in decisions and research activities as equal partners throughout the research process.
Citizen control	Patients and the public have complete control over the design, execution and dissemination of research, and researchers are involved on request.

Box 3. User-centred design stages: methods for capturing end users’ perspectives. Based on Shah et al. [35].

Box 3. User-centred design stages: methods for capturing end users' perspectives. Based on Shah and Robinson [35].

	Concept stage	Design stage	Test and trials stage	Deployment stage
Interviews	X	X	X	X
Usability tests	X	X	X	X
Questionnaires	X	X	X	X
User and producer seminars	X	X	X	X
Task analysis	X	X	X
Observations	X	X	X
Simulations	X	X	X
Discussion	X	X
Video recording			X	X
Human factors approach		X	X
Use experiment				X
Focus groups (Delphi)				X
Users’ feedback	X
Design sessions		X