Systems thinking and designerly tools for medical device design in engineering curricula

ABSTRACT

Background

In this paper we focus on medical device development (MDD) in Industrial Design Engineering (IDE) academia. We want to find which methods our MDD-students currently use, where our guidance has shortcomings and where it brings added value.

Methods

We have analysed 19 master and 3 doctoral MDD-theses in our IDE curriculum. The evaluation focusses around four main themes: 1) regulatory 2) testing 3) patient-centricity and 4) systemic design.

Results

Regulatory aspects and medical testing procedures seem to be disregarded frequently. We assume this is because of a lack of MDD experience and the small thesis timeframe. Furthermore, many students applied medical-oriented systemic tools, which enhances multiperspectivism. However, we found an important lack in the translation to the List of Specifications and to business models of these medical devices. Finally, students introduced various participatory techniques, but seem to struggle with implementing this in the setting of evidence-based medicine.

KEYWORDS:

• Medical device design
• new product development
• human factors engineering
• co-design
• systems thinking
• participatory techniques

1. Introduction

Traditional new product development (NPD) approaches inherently apply to a broad set of industries. This general nature may turn them insensitive to the specificities of demanding contexts. For healthcare, with its high level of regulations, complex economics and a very diverse set of stakeholders, the need for detailed additions to general NPD approaches has been recognised in literature (Medina et al., 2012; Ocampo & Kaminski, 2019). Accordingly, different NPD models specifically for medical device design (MDD) have been proposed, often elaborating on the complex regulatory context (Kaplan et al., 2004; Ogrodnik, 2019; Pietzsch et al., 2009). These MDD models mostly apply to industrial companies, whose products are heavily influenced by economic and regulatory conditions. For academia, specific MDD models and programmes have been proposed, such as Stanford’s BioDesign Process (Yock et al., 2011) and TU Delft’s Medisign programme (Goossens et al., 2004). These models usually show the development process, but do not specifically cover the practical tools and methods to apply in MDD.

In the Industrial Design Engineering (IDE) curriculum at our university, we noticed an increasing popularity of MSc theses in MDD in recent years. Here, students develop a medical device for (and together with) a hospital, care institution or medical device company. The end result mostly is a proof-of-concept prototype and a written report. Our MSc students are coached by their university promotor and the external institution. Furthermore, they can fall back on general design and engineering courses from earlier in the curriculum. Our IDE programme offers a broad polytechnical education with a specialisation in industrial design engineering; specifics can be found in the Addendum. However, we currently do not offer specific courses on MDD in our IDE curriculum, nor is there a specialisation track. We wish to improve the thesis guidance of our MDD students and wish to be more efficient as promotors. Therefore, we have analysed the MDD theses from the past 8 years in this study. By investigating which tools our MDD students currently use, we may find where our guidance and curriculum show shortcomings and where they bring added value.

Our teaching curriculum, research and thesis supervision are influenced by the fields of Industrial Design Engineering and Systems Thinking. Both fields have a strong implementation in the healthcare sector. Nevertheless, their individual approaches cannot fully serve every problem. We hypothesise that a synergy arises when combining them. In this study, we quantified the implementations of these fields and their tools in MDD-related theses to get a better understanding of their added values.

1.1. Systems thinking in healthcare

Healthcare is a classic example of a complex sociotechnical system, as a whole of interactions between interdependent parts (Jones, 2013). Systems thinking approaches are widespread in healthcare and medical device management. In risk and quality management particularly, different models have been introduced that investigate the different levels of a system and the relationships between its components. Frameworks such as Systems Engineering Initiative for Patient Safety (SEIPS) have contributed in enhancing patient safety in a holistic manner (Carayon et al., 2006). Many of these models stem from systems and Human Factors Engineering (HFE), such as the Cognitive Work Analysis (Naikar, 2017). Here, the human factor is not analysed standalone but in its systemic context.

1.2. Industrial design engineering in healthcare

In IDE, user-centred approaches (e.g., Human Centred Design) often take a central role in the development process, making sure the user desires the product and is able to use it properly. In most MDD, user research is also structurally embedded via HFE, but this mostly is for a different reason. Here, usability tends to be investigated for patient safety and risk mitigation reasons, pushed by the high level of regulations (Benker, 2015). Although these are valid motives, the user acceptance aspect may be overlooked this way (Karsh, 2004). This is unfortunate, as it is crucial to detect and mitigate use errors and user rejection as early as possible, in order to reduce development costs. On the other hand, user-centred design often lacks a systemic perspective (Santos & Wauben, 2014; Sevaldson, 2010).

It is therefore suggested that the user-centred, empathic and pragmatic approaches of IDE may overcome the shortcomings of HFE in MDD (Barbero & Pallaro, 2017; Privitera, 2017; Privitera et al., 2015). Similarly, the HFE methods may tackle the lack of a systemic approach of user-centred approaches. Therefore, we believe it is useful to investigate how students implement these various approaches in MDD.

2. Materials and methods

We have collected and analysed all MDD-related theses from the Industrial Design Engineering specialisation, submitted between 2014 and 2021 at our local university. These 19 MSc and 3 PhD theses are listed in Table 1 and are available throughout the institutional repository. Projects were identified as dealing with MDD if they met the EU Medical Device Regulations’ (REGULATION (EU) 2017/745, 2017) definition of a medical device. In brief, a medical purpose such as prevention, diagnosis, monitoring or treatment was required for the device. Projects that fell into the broader healthcare spectrum without medical purpose (e.g., commercial health wearables) were withdrawn from this selection.

Table 1. List of the analysed theses. In the second column, a short description of the thesis is given. The third column describes whether this was a MSc or a PhD thesis. In the fourth column, the type of client is given. We divided these into 3 main categories: medical device manufacturers, universities or university hospitals, and care institutions. The fifth column shows the academic year in which the thesis was submitted. In the sixth column, the anonymised code of the promotor is given. In the seventh column, the reference to the university website is given to find more details on the thesis.

N°	Thesis subject	MSc or PhD thesis	Client type	Year	Promotor	Reference
1	Implementation of a patient-nursing staff call system	MSc	Medical device manufacturer	2014	A	https://lib.ugent.be/catalog/rug01:002154080
2	Redesign of a patellar tendon bearing (heel brace)	MSc	Medical device manufacturer	2015	B,E	https://lib.ugent.be/catalog/rug01:002224363
3	Advanced out-of-bed detection system to relief nursing staff	MSc	Medical device manufacturer	2015	C	https://lib.ugent.be/catalog/rug01:002224703
4	Non-stigmatising assistive device for people with reduced mobility	MSc	Medical device manufacturer	2018	A,C	https://lib.ugent.be/catalog/rug01:002494780
5	3D printed rehabilitative knee orthosis to reduce stigma	MSc	Medical device manufacturer	2019	C	https://lib.ugent.be/catalog/rug01:002786165
6	Personalised 3D printed thumb orthosis	MSc	Medical device manufacturer	2020	C	https://lib.ugent.be/catalog/rug01:002946090
7	Innovative hinge for a sitting orthosis as dyskinetic support	MSc	Medical device manufacturer	2020	B	https://lib.ugent.be/catalog/rug01:002946061
8	Custom-made elastic orthosis in additive manufacturing for patients with spasticity in the forearm	MSc	Medical device manufacturer	2020	C	https://lib.ugent.be/catalog/rug01:002946095
9	Smart adjustable ligament knee orthosis	MSc	Medical device manufacturer	2021	B,D	https://lib.ugent.be/catalog/rug01:003014993
10	Optimisation of the prone position in breast radiotherapy	MSc	University hospital	2014	F	https://lib.ugent.be/catalog/rug01:002154077
11	Bistable composite connection for a radiotherapy breast couch	MSc	University hospital	2016	B,E	https://lib.ugent.be/catalog/rug01:002300852
12	Ergonomic solution for the steep Trendelenburg position in surgery	MSc	University hospital	2019	B,D	https://lib.ugent.be/catalog/rug01:002786156
13	Feedback system for breath holding during breast cancer radiotherapy	MSc	University hospital	2019	B,D	https://lib.ugent.be/catalog/rug01:002786157
14	Pivoting breast couch to increase the beam range in radiotherapy	MSc	University hospital	2021	B,D	https://lib.ugent.be/catalog/rug01:003015176
15	Redesign of a lower limb exoskeleton	MSc	University	2015	A,B	https://lib.ugent.be/catalog/rug01:002224360
16	Autonomous foot exoskeleton with preservation of natural ankle joint freedom	MSc	University	2016	A,B	https://lib.ugent.be/catalog/rug01:002300467
17	Special seat for people with a severe mental disability	MSc	Care institution	2014	F	https://lib.ugent.be/catalog/rug01:002154075
18	Measuring tool for assistive seats for children with cerebral palsy	MSc	Care institution	2016	B	https://lib.ugent.be/catalog/rug01:002300684
19	Home monitoring device for early detection of lymphoedema	MSc	Care institution	2021	B,D	https://lib.ugent.be/catalog/rug01:003014934
20	Prone support device for radiotherapy of breast and regional lymph nodes	PhD	University hospital	2019	B	https://lib.ugent.be/catalog/pug01:8610240
21	Exploration of the advanced capabilities of the Open Access Breast Couch	PhD	University hospital	2022	B	(to be published online)
22	Adaptation by product hacking for assistive technology	PhD	Care institution	2016	B	https://lib.ugent.be/catalog/rug01:003014934

For this qualitative study we followed the QUAGOL guideline, derived from Grounded Theory Approach (Dierckx de Casterle et al., 2012) In the first part of QUAGOL, documents are analysed iteratively and rich information is clustered into concepts. In this case, a list of MDD-related theses was composed. The four researchers then individually scanned the theses for tools, methods and guidelines where medical specificities were indispensable. The full methodological list was shared and discussed amongst the investigators. To structure the coding into concepts, the researchers together came up with 4 common themes after the first iteration: 1) regulatory aspects, 2) testing methods, 3) patient-centricity and 4) systemic design. In the second part of QUAGOL, the concepts are critically analysed by the group of reviewers. With the final concepts structured, documents are analysed again for conclusions. In this case, the MDD-related theses were divided amongst the investigators and read thoroughly. The investigators coded the information and finally discussed the outcomes for conclusions.

Table 2 shows three general methodologies that we teach to students. Although they are general, this combination of methodologies is not taught in every IDE curriculum and is thus typical to our institution. We embedded them in Table 2 as they presumably influence the work of our students, and as they are used to further structure the tools and methods in the Results section.

Table 2. Overview of general methodology in our IDE programme. Our curriculum and research combine three typical methodologies: research-through-design, co-design and systems thinking. In the third column, we linked each of these general methodologies to the main themes that were identified in this analysis: 1) regulatory aspects, 2) testing methods, 3) patient-centricity and 4) systemic design. This can also be found in Figure 1.

General methodology	Summary	Main themes
Research through Design	Practice-based research approach that employs designerly methods and processes to gain knowledge (Zimmerman et al., 2010). For the design of medical products and product-service systems (PSS) especially, the iterative development and evaluation of prototypes may serve as a stimulus for knowledge generation between a broad set of stakeholders (Stappers, 2014).	3.Patient-centricity 2.Testing methods
Co-design	Co-design refers to a user-centred participatory approach where stakeholders, without a design or engineering background, actively contribute to the development process (E. B.-N. Sanders & Stappers, 2008). This may go beyond a mere passive form of user testing, instead seeking active input from patients to caregivers. This is considered as a critical step, as medical products often stem from a technology-push and fail in the market when lacking user acceptance (Benker, 2015; Shah & Robinson, 2007). Furthermore, co-design -where all stakeholders have ownership of the solution- is described as a key success factor in complex challenges (Norman & Stappers, 2015)	3.Patient-centricity
Systems thinking	Contrary to the pragmatic and solution-oriented nature of designerly tools, systems thinking tends to be oriented more towards mindsets and competencies in dealing with complexity and wicked problems. Nevertheless, systemic methods such as giga-mapping may be both holistic and solution-oriented. Systemic design tools enhance multiperspectivism by unravelling the dynamics between actors in the system.	3.Patient-centricity 4.Systemic design

3. Results

The quantitative results can be found in Table 4, supported by qualitative information in Figure 1 and Table 3. Figure 1 provides a schematic overview of the methodologies, tools and guidelines that were discussed in this analysis. This figure serves as a guide to see the relationships between the examined methods and tools. Table 3 provides a structured summary of the different methods and tools that appeared (at least once) in the analysis. Table 4 shows the frequency of the different methods and tools. The data of Table 4 is analysed below.

Figure 1. Overview of methodologies and tools used in MDD at the Industrial Design Engineering specialisation at our university. The first row shows a simplified version of Pahl and Beitz’ Systematic Approach. This is our design workflow of choice at Ghent University, but as shown by Ocampo (Ocampo & Kaminski, 2019) different general design workflows are applicable in MDD. The second row shows the general approaches we actively teach that also may show added value to MDD. The 3rd row shows the 4 main aspects we focused on. Next to these aspects, the initials of the general approaches from the 2nd row are placed, showing which approach is typically used in the given aspect. In the 4th row, we placed the different tools and guidelines that were found in the analysis, sorted underneath the proper main aspect.

Table 3. Summary of the methods and tools. Each of these appeared at least once in the analysed theses. Column 1 shows the main theme to which the tool or method can be linked. This allocation was discussed amongst the researchers and is not exhaustive. Column 2 shows the specific tool, method (or guideline) that was analysed. A structured overview of the themes and their tools can be found in Figure 1. Column 3 offers a short description. Column 4 refers to related methods and tools, that were not found in the student theses but are also relevant to MDD. This non-exhaustive list was composed by researchers with experience in MDD.

Main theme	Specific tool, method or guidelines for MDD	Description	Related methods & tools
1. Regulatory aspects	1.1 Device Classification	The European medical device classification starts from the vulnerability of the human body in relation to the potential risks (low-medium-high) associated with the device. Different criteria can determine the class, such as duration of bodily contact, degree of invasiveness and local-versus-systemic effects. The class defines the regulatory obligations, which directly influence the engineering (materialisation, production, …) and testing phases.	ISO13485 Medical Device Regulation (EU) 2017/745 FDA section 510 (k)
	1.2 FMEA	A Failure Mode and Effects Analysis (FMEA) is often embedded as a part of the risk management assessment. In a FMEA potential failures are identified, their causes determined and effects ranked. (Greig et al., 2014). Different subtypes of FMEA are applicable in MDD, as they apply to different aspects such as design, usability or manufacturing.	STAMP-analysis SEIPS model Fault tree analysis Root cause analysis Cognitive Work Analysis
2. Testing	2.1 Validation & Verification	Part of Quality management, structurally embedded in the FDA’s Design Control and ISO13485. Verification means checking whether the device complies with the specified requirements (‘making the things right’), whereas validation checks if the intended use has been fulfilled (‘making the right things’). Figure 2 shows an adapted version of the Design Control scheme, with different designerly tools pinned upon.	ISO13485 Medical Device Regulation (EU) 2017/745 FDA section 510 (k)
	2.2 Testing methods in MDD	Three testing schemes from Stanford’s BioDesign (Yock et al., 2011) give the students a clear take on MDD testing specifics: ‘Levels of evidence in healthcare design’ – ‘Testing continuum’ – ‘Key R&D milestones’, This is shown in Figure 3.	Stanford BioDesign Process
	2.3 Ethics committee & Compassionate Use	An ethics committee then ensures whether patient testing may be carried out in an ethical manner, in accordance to legislation. This evaluation process, from writing a submission to a final decision is very time-consuming and resource-intensive. Compassionate use is a treatment option that allows the use of an unauthorised medicine. Late-stage products may use this option to enable patient access, however this is only possible under very strict conditions.
3. Patient centricity	3.1 Patient Journey Map	Patient Journey Mapping, a specific variant of (customer) journey mapping, describes the full treatment path from a patient’s perspective. All encounters with medical actors, places, devices can be noted, as well as the in- and outputs in each touchpoint (McCarthy et al., 2016). Both the physical and the emotional journey of the patient may be listed, which serves multiple goals.	Patient Synthesis Maps (Jones et al., 2017). Learning Histories method (Kleinsmann et al., 2018) Multi Level Design Model (Boru et al., 2015) Medical Metroline (Griffioen et al., 2021)
	3.2 Participatory design research methods	Different authors have listed participatory techniques that stem from user-centred design and their application in healthcare. These lists are not exhaustive and the majority of techniques is often shared as they are common participatory methods.	Peter Jones’ Integrated Service Design Framework (Jones, 2013) Liz Stappers’ Map of Design Research (E. B. N. Sanders, 2008) Storyboards Workflow analysis …
4. Systems thinking	4.1 Treatment Journey Map	A Treatment Journey Map is a systemic map where all actors of a treatment are mapped and all underlying processes are determined with their respective in- and outputs.	Learning histories method (Kleinsmann et al., 2018) Patient Synthesis Maps (Jones et al., 2017).
	4.2 Stakeholder Map	Visual process of mapping all stakeholder (not limited to healthcare). Stimulates a systemic view on complex challenges. For healthcare, stakeholder mapping is useful as there usually is a very broad set of direct and indirect stakeholders.	Learning Histories method (Kleinsmann et al., 2018) Patient Synthesis Maps (Jones et al., 2017).
	4.3 Business models for medical devices	Medical devices often have specific business models, as they are part of a complex socio-technical system. For many of them, their economic success is highly dependent of reimbursement policies and the methods through which a product is prescribed, purchased, and/or billed (Durfee & Iaizzo, 2018). For students, it is important to understand the specificities of the value exchange between different stakeholders.

Table 4. Count of the different methods and tools. We divided both the MSc and the PhD theses in 3 subgroups, differentiated by the client type. For the methodologies, tools and guidelines, we used the same structure as found in Figure 1.

		MSc theses: n = 19			PhD theses: n= 3
		Client type			Client type
		Medical device company	University (Hospital)	Careinstitution	Medical device company	University (Hospital)	Careinstitution
General methodology	Co-design	5/9	2/7	3/3	0/0	2/2	1/1
	Research-through-Design	7/9	7/7	3/3	0/0	2/2	1/1
1. Regulatory aspects	Device Classification	0/9	0/7	0/3	0/0	2/2	0/1
	ISO 13485	0/9	0/7	0/3	0/0	2/2	0/1
	FMEA	0/9	2/7	0/3	0/0	1/2	0/1
	Validation & Verification	0/9	0/7	0/3	0/0	2/2	0/1
2. Testing	Ethical Committee & compassionate use	0/9	0/7	1/3	0/0	2/2	0/1
	Testing Hierarchy	1/9	4/7	1/3	0/0	2/2	1/1
	Business models for MDD	5/9	0/7	1/3	0/0	0/1	1/1
3. Patient-centricity	Patient Journey Map	5/9	2/7	1/3	0/0	1/2	0/1
	Participatory Design Methods	6/9	4/7	3/3	0/0	2/2	1/1
4. Systemic design	Causal Loops	0/9	0/7	0/3	0/0	1/1	1/1
	Function-Structure Diagram	2/9	1/7	0/3	0/0	1/1	1/1
	Stakeholder map	7/9	6/7	2/3	0/0	1/1	1/1
	Treatment Journey Map	3/9	3/7	1/3	0/0	1/1	0/1

3.1. Analysis

Medical regulatory aspects (device classification, MDR, FMEA, …) are often disregarded in the analysed MSc-theses. If the project gets picked up further (e.g., PhD-thesis or industrialisation), this may result in important and costly problems later on, as explained in the design paradox. Similar tendencies were found for medical testing. Little medical testing protocols were applied in the analysed cases.

Both Patient Journey Maps, Treatment Journey Maps and Stakeholder Maps were commonly used in the theses. We consider this as systemic MDD tools that support multiperspectivism. Nonetheless, we noticed that the wishes and demands of stakeholders, uncovered in these tools, were regularly overlooked in the List of Specifications.

Rather classic systemic tools such as causal loops and function-structure diagrams were used in a minority of theses. As for business models, we found a large frequency in theses for medical device companies whilst business models were underexposed in theses for universities, hospitals and care institutions.

3.2. Examples

Figure 4 shows a selection of valid examples of different tools that were used in the analysed theses. Table 5 gives further details on these examples and explains their added value.

Table 5. Thesis examples of different methods and tools for MDD. In this table, we highlighted some good examples of tools used in the analysed MDD theses. In the 4^th column we explain the added value of this method or tool for the end result of this specific thesis, addressed by the student (in the thesis) or the reviewers. In the 5^th column we explain the potential pitfalls of these tools, addressed by the reviewers. The examples are visualised in Figure 4.

example in Figure 4	Thesis subject (thesis number)	Short description	Method or tool used	Added value to end result	Possible pitfalls
A	Exploration of the advanced capabilities of the Open Access Breast Couch (21)	A novel prone patient position was introduced in breast cancer radiotherapy. After a proof-of-concept it was time to explore the medical benefits of this device.	Treatment Journey Map (TJM)	The TJM was useful to bridge different stakeholders and their needs, as some were somewhat isolated from others. This guided a proper and complete problem statement to start with.	The TJM offers a holistic view on the treatment, but it does not showcase the flows of information and dynamics in the system. It is also portrayed rather linear.
B			FMEA	The FMEA was performed to obtain a CE-certification. This also triggered critical thinking towards risks and provided an overview of potential use situations.	The FMEA was a theoretical exercise, estimating the risks and consequences. It is not fully backed by real tests.
C	Feedback system for breath holding during breast cancer radiotherapy (13)	The head region of a positioning device had to be adjusted so that patients could hear and read instructions for breath holding. Special attention for ergonomics and material use in a radiation environment	Function-Structure diagram	Radiotherapy requires consultation with multiple specialists and multi-disciplinary review boards. There is a strict protocol of material, information and energy flows. Mapped out, these flows provided insights for the various stakeholders, as well did it untangle the complex workflow for the designer.	This diagram is highly informative for the student, but takes a lot of effort to make. It is difficult for the student to estimate whether this effort will proof useful. Furthermore, it is not straightforward to read this diagram as a stakeholder or outsider.
D	Implementation of a patient-nursing staff call system (1)	Nurses are confronted with heavy workload and stress. Inefficient communication between patient and nursing staff may add to the problem. Therefore, a novel nurse-patient call system was developed to enhance intervention efficiency.	Causal loop diagram	The causal loop diagram was used to redefine the problem statement. The dynamics between different managerial aspects & interpersonal relationships were unveiled, showing where the potential benefit of innovation was most efficient.	The student only made a causal loop diagram of the current situation, and not of the situation with the developed medical device. Without realistic testing, it may be difficult to estimate the change in dynamics.
E	Home monitoring device for early detection of lymphoedema (19)	Lymphoedema is an incurable condition, often discovered too late because of a deficient check-up. A home monitoring device could detect the symptoms sooner, halting the disease. It was important to design a technological functioning device with respect to cognitive and physical ergonomics.	Stakeholder Map	The Stakeholder Map unveiled all stakeholders related to both the device and the patient. This proved useful as some stakeholders were unaware about other stakeholders and their relation to the device.	The stakeholders are mapped, but their demands and wishes towards the product are not portrayed. These requirements were often not translated into the List of Specifications.
F	Smart adjustable ligament knee orthosis (9)	Postoperative swelling occurs often after knee surgery. An orthosis needed to be developed to adapt precisely to the swelling and the patients physique. Additive manufacturing and mechatronics were most suited to tackle this challenge.	Patient Journey Map (PJM)	The PJM provided a patient perspective to the different stakeholders to enhance empathy. Furthermore, new challenges were uncovered that were not part of the initial problem statement. It was also used as a template to properly compare the initial state, the possible scenarios and the end state.	The PJM is quite linear and only uncovers one perspective of the treatment. The information flows between other stakeholders are not portrayed here.

4. Discussion

By quantifying the tools and methods that our students use in MDD-related theses, we wanted to find trends and opportunities that could enhance the guidance of thesis students. With the results listed, we can now explain the observed trends and make recommendations for MDD-related theses.

In many theses (n° 1–13, 15, 18–19), we encountered an important lack in anticipating regulatory requirements. We assume that the short timeframe (10 months at 40% of their total workload) and limited experience of students may evoke other priorities. A big regulatory lack was found in theses for hospitals, presumably as they’re not primarily focussed on marketable products. Students often referred to the slow nature of clinical trials, their large administrative workload and the difference in priorities between the student and the medical researchers. In theses with medical device manufacturers however, the regulatory side wasn’t highlighted that often either. We presume that this is because these companies have in-house knowledge to further tackle regulatory issues. Altogether, we would suggest a short introduction of the most important regulatory aspects to thesis students and a structural integration in their List of Specifications. This is important, as projects may suffer from regulatory flaws in further development stages.

As for the use of systemic design, we noticed that some medical-oriented systemic tools (e.g., Stakeholder Mapping) were popular (n° 1, 3–5, 8–15, 17, 19, 21–22), but that general systems thinking tools (e.g., function-structure diagrams) were not widely applied (n° 1–2, 8, 13, 21–22). Systemic tools may be useful in complex settings for different applications such as problem finding, problem reframing, future forecasting and value communication (Peters, 2014). However, understanding and interacting with systems can take a lot of time and effort. For our master theses, the timeline may be too limited to elaborate on systemic interactions. We should therefore aim for time-efficient systemic tools in MDD, like Treatment Journey Mapping.

Furthermore, we found that participatory design methods were applied often (n° 1–2, 4–6, 9–10, 12–13, 16–22), but classic medical testing procedures were applied less (n° 2, 10–12, 14, 18, 20–22) . For industrial design engineers, these different research approaches may be difficult to cope with (Groeneveld et al., 2018). Whilst medical testing relies on strict protocols and quantitative data, participatory techniques are less defined and open to the researcher. It may be difficult to apply these two opposed ways of thinking, and also to convince medical staff of their added value as they’re not used to these rather subjective methods. Nevertheless, both research methods are of important value in MDD. Participatory techniques support in obtaining end user acceptance, while traditional medical testing (“evidence-based medicine”) is vital to obtain an effective, safe end product that also surpasses the regulatory requirements. For the students, it can be useful to anticipate on further medical testing in the future work section of the thesis. Figure 2 is an adapted model of the Verification and Validation model, that originates from the FDA’s Design Control. Here, it is shown how this regulatory framework can serve as a template for designerly tools, bridging both approaches. Figure 3 shows a list of MDD testing methods, adapted from Stanford’s Biodesign process (Yock et al., 2015). This summary may help IDE students to efficiently overcome differences in both approaches.

Figure 2. Validation & Verification (Design Control) as a template for designerly tools in MDD (adapted from (Privitera et al., 2015).The work of professor Privitera shows the possibility of using the classical Design Control figure as a template to map different design techniques upon. It is noteworthy that next to the clinical and technical requirements, participatory techniques can play a useful role in uncovering user needs. This may lead to a usable and desirable product, as checked by the Validation process. Furthermore, the active testing of prototypes is an important step to valorise the overall functioning of the product, as checked by the Verification process. For this process, we have described the benefits of using a Research through Design approach in a co-design trajectory.

Figure 3. Testing methods in MDD (adapted from (Yock et al., 2015)). The first row shows the simplified “Levels of evidence in healthcare design”, which give students notice of the ways healthcare professionals determine evidence. The second row shows the “Testing continuum”, a schematic overview of the different testing procedures in MDD and their respective order. The “Testing continuum” is not limited to medical devices, whereas the “Key R&D” milestones (3rd row) provides a chronological order of important testing milestones that relate to medical devices.

Figure 4. Visual examples of tools used in the examined theses. The letters (A-F) 406 correspond with the detailed explanation given in Table 3.

By implementing co-design and participatory design methods, we believe students facilitate multiperspectivism in a medical context with different types of stakeholders. This may support the user acceptance and usability. However, we also found that the requirements of these various stakeholders are not all translated into the List of Specifications. This is a point of attention, as not only the patient and other main stakeholders have a large influence on the medical product and its underlying processes. Furthermore, we found that only students at medical device companies paid attention to the business model behind the product. This seems evident, as these companies focus on marketable products whereas hospitals and care institutions do not. Also, it can be difficult to envision the financial mechanisms as a lot depends on the reimbursement of the device, which is not yet clear in the development phase. Nevertheless, we think it’s unfortunate that students take the effort of portraying all stakeholders, but do not put them together in a business model. This not only involves the financial aspects but also other forms of value exchange: who handles and owns the medical data, can used products be remanufactured, …

We could not distinguish any clear trends over time, except for the Patient Journey Map that became more widespread, whilst the Stakeholder Map was the overall most frequent tool. Furthermore we also examined the influence of the promotor on the use of tools. We did not find any clear differences, but we found that some promotors guided increasingly more MDD-related theses over time. This experience presumably adds to the efficiency of the guidance. A last comparison can be made between MSc and PhD theses, although the sample size of the PhD theses is too low for generalisations. We found that the PhD theses used a larger variety of tools, which is presumably facilitated by the larger timeframe (>4 years versus 1 year). The impact of design implementations could also be better evaluated as there is a larger timespan. PhD students are also expected to spend more time on scientific work, which is noticeable in the theses. Here, the experiments were better performed, based with less subjective and more scientific outcomes. The creative R&D-process was more pronounced in the MSc-theses.

Finally, we want to discuss the shortcomings of this analysis. The sample size (n = 22) was rather small, and the analysis may be prone to bias as our curriculum logically influences the theses results and ourselves as researchers. As for the future, it would be useful to incorporate the findings of this manuscript in guiding new thesis students. Furthermore, we would like to continue this evaluation to obtain bigger sample sizes, and to compare it to analyses of other (design) engineering curricula. Also, we would like to include interviews with students, to uncover why certain tools were (not) applied.

5. Conclusions

In this case-study based analysis, we screened 22 MDD-related theses of industrial design engineers for the methods and techniques that were applied. In the end, we wanted to find the added value and pitfalls of different methods that demand extra attention from our part as promotors.

Regulatory aspects and medical testing procedures seem to be disregarded frequently in theses, although these are of great importance to a sound MDD. We assume the small thesis timeframe plays an important role, next to limited experience with MDD. Furthermore, many students applied medical-oriented systemic tools such as treatment journey mapping, which enhance multiperspectivism. However, we found an important lack in the translation of these multidisciplinary insights to the list of specifications and business models for medical devices. Finally, students introduced various participatory techniques, which enhances the usability aspect of different stakeholders. However, it seems that students working for hospitals struggled to fully implement these participatory techniques in settings of evidence-based medicine. The insights of this paper may be of interest to other academic curricula, including but not limited to: mechanical engineering, biomedical engineering and medicine curricula.

Disclosure statement

No potential conflict of interest was reported by the author(s).