Combining Soft Systems Methodology, ethnographic observation, and discrete-event simulation: A case study in cancer care

Abstract

We describe a project where discrete-event simulation (DES) is combined with Soft Systems Methodology (SSM) and ethnographic observation to support the improvement of patient flows in an outpatient chemotherapy unit and a chemotherapy preparation pharmacy unit. The project combines three interesting characteristics. First, we rely on ethnographic observation to explore the problem situation and frame the problem, a technique rarely reported in operational research (OR). Second, this leads us to define several root definitions, an aspect that remains overlooked in studies combining SSM and DES. Finally, the project stands at the interface between two departments (outpatient oncology unit and chemotherapy preparation pharmacy unit), a situation seldom explored in healthcare OR but where SSM has arguably a lot to offer to help identify and accommodate diverging objectives. We framed the problem, developed a simulation model, explored scenarios and refined a preferred one into a new business process. However, the proposed changes were not implemented. We analyse the outcomes of the project, the contribution of each method and their integration into an intervention. The paper provides insights on how SSM and DES can be combined in practice, and how the potential of ethnographic observation can be leveraged in this context.

Keywords:

• Simulation
• ethnographic observation
• soft systems methodology
• healthcare OR
• problem structuring methods
• soft OR

Introduction

Discrete-event simulation (DES) is a computer simulation approach that models the operation of a system as a series of events in time. It has been extensively used to analyse industrial flows in various industries, and it has arguably a lot to offer to health services (Pitt, Monks, Crowe, & Vasilakis, 2016; Rutberg, Wenczel, Devaney, Goldlust, & Day, 2015). However, its uptake in this sector remains low (Brailsford & Vissers, 2011; Fone et al., 2003). To improve this situation, some have proposed to combine DES with problem structuring methods from Soft OR to help manage conflicting views on managerial issues. Using Soft OR methods could help better account for the variety of perspectives in healthcare organisations. In particular, Soft Systems Methodology(SSM) provides a structured approach to the elicitation of different worldviews on a problem situation. Therefore, it has the potential to support the investigation of the expectations of a variety of stakeholders, leading to more relevant models (Kotiadis, 2010; Pidd, 2010). However, we still have much to understand about how to combine SSM and simulation (Brooks & Wang, 2015; Van der Zee, Brooks, Robinson, & Kotiadis, 2010).

In this paper, we present a case study where SSM and DES are combined to improve waiting times in an outpatient chemotherapy unit. This project contributes to the literature on combining Problem Structuring Methods (PSMs) with simulation in three different ways. First, the case study describes how we used a simulation model at the interface between two departments of the same hospital, an outpatient unit and a pharmaceutical unit. Multi-department studies are rarely reported in healthcare OR (Gunal & Pidd, 2010; Vanberkel, Boucherie, Hans, Hurink, & Litvak, 2009), and we show how SSM helped identify interests and issues in both departments. Second, another distinctive aspect of this study is its reliance on extensive ethnographic observation in the early stages, supported by SSM’s tools, to explore the problem situation. Ethnographic observation is rarely reported in OR papers, despite its promises and historic place in OR practice (Horlick-Jones & Rosenhead, 2007). Finally, we report on how our ethnographic observations led to the definition of several root definitions in the project, which only a few studies combining SSM and DES report, despite the fact that exploring multiple root definitions is a major aspect of SSM. Our analysis shows that these root definitions provided useful aide-memoires along the project, although a primary task root definition needed to be used to develop the simulation model.

In the next Section, we review the literature on combining SSM and DES, and on using ethnographic observation in OR. We then report the case study itself. Finally, we discuss the results of this project and its contribution to the literature.

Background

We next give a background on how SSM and DES have been combined in past studies in various contexts. We then describe the principles of ethnography and how ethnographic observation has been used in OR.

Combining SSM and DES

SSM is an organised way of thinking about problematical situations in order to take action to improve them (Checkland, 1981; Checkland & Scholes, 1990). In its most recent form, SSM works through four activities (Checkland, 2000; Checkland & Poulter, 2010):

1. Finding out about a situation which is seen as problematical, including culturally/politically.

2. Formulating some relevant purposeful activity models, based on different worldviews.

3. Debating the situation, using the models to structure the discussion, in order to identify desirable and (culturally) feasible changes that could improve the situation, while accommodating conflicting interests in presence.

4. Defining and taking action to bring about improvement.

In each step, SSM offers a set of tools, such as rich pictures (visual representations of the problematic situation), root definitions (which define systems relevant to the investigation from particular viewpoints) or Analyses One, Two and Three (which structure reflection about the intervention and the cultural and the political context around it, respectively). Applications of SSM can be very methodology-driven, and follow the steps and tools as an “external recipe,” as described in the original seven-step model of SSM (Checkland, 1981), or the modeller can use SSM as an internalised model to guide his actions in a situation-driven fashion (Checkland, 2000). These two ideal-typical approaches have been called the Mode 1 and Mode 2 of SSM, respectively.

It has been argued that SSM could form a synergetic pair with DES in the investigation of complex organisational issues. For instance, SSM could support the initial qualitative investigation in conceptual modelling (Montevechi & Friend, 2012) through structuring data collection, abstracting this knowledge into conceptual models, and setting objectives for the simulation study (Kotiadis, 2010; Pidd, 2010). SSM is also based on the idea that reality is socially constructed through different people holding different worldviews on any situation (Checkland, 1981), and it offers a systemic approach to investigate these worldviews. Therefore, SSM could help integrate the perspectives, interests and expectations of different stakeholders into simulation projects (Pidd, 2010). This way, it is hoped that projects using SSM would be more likely to see their results implemented (Lehaney & Hlupic, 1995).

There have been various examples of studies combining SSM and DES in the OR literature, most of them in healthcare, with only one example to our knowledge outside health services, in manufacturing (Pereira, Montevechi, Miranda, & Friend, 2015). Most of these studies focus on the use of SSM to support the conceptual modelling stage of the simulation process (Holm & Dahl, 2011; Kotiadis, 2007; Kotiadis, Tako, & Vasilakis, 2014; Lehaney & Paul, 1994, 1996; Pereira et al., 2015). Only a few papers report on SSM being used in the full process (Holm, Bjornenak, Kjaeserud, & Noddeland, 2017; Holm, Dahl, & Barra, 2013; Lehaney, Clarke, & Paul, 1999; Tako & Kotiadis, 2015).

A key element in SSM is establishing “root definitions” of the problem situation. Root definitions are structured statements describing the situation to be modelled from a specific perspective. They can often be expressed using the mnemonic CATWOE (Customers, Actors, Transformation process, Weltanschauung/Worldview, Owner, Environmental constraints). Defining multiple root definitions in a project allows to take into account different, complementary perspectives on the problem situation. However, some studies combining SSM and DES only develop one root definition (Kotiadis, 2007; Kotiadis et al., 2014). Others mention developing multiple root definitions (Holm et al., 2017; Holm et al., 2013; Lehaney et al., 1999; Lehaney & Paul, 1994, 1996; Pereira et al., 2015), but it is not always clear how they are used. The use of multiple root definitions is also proposed by Pidd (2010), but the application is a thought experiment in a post-hoc analysis of a real simulation project. The most explicit account of developing multiple root definitions is by Holm and Dahl (2011). The authors develop multiple root definitions in a facilitated workshop with healthcare workers. They then split each root definition into its “What?” “How?” and “Why?” components, following SSM’s PQR technique (Checkland & Poulter, 2010). These elements are then merged into one overarching root definition which is used in the conceptual modelling stage of the simulation process.

To summarise, combining SSM and DES is not a new thing, especially in a healthcare context. However, most reports focus on the conceptual modelling stage, and rely on a single root definition or remain vague when they use several root definitions. We aim to tackle these two points in our case study, by describing how SSM and DES can be combined all along the project, and exploring the use of several root definitions.

Ethnographic observation in OR

Ethnography is a research method originally developed in social anthropology. In this article, we draw on the broad definition of ethnography provided by Hammersley and Atkinson:

“In its most characteristic form it involves the ethnographer participating, overtly or covertly, in people’s daily lives for an extended period of time, watching what happens, listening to what is said, asking questions – in fact, collecting whatever data are available to throw light on the issues that are the focus of the research.” (Hammersley & Atkinson, 1995, p. 1)

Participant observation is a distinctive feature of ethnography. It refers to a wide range of practices, going from “hanging out” to more structured observation processes (Dixon-Woods, 2003), and from long-term involvement to shorter observation periods (Leslie, Paradis, Gropper, Reeves, & Kitto, 2014). With this approach, ethnography “can capture the winks, sighs, head shaking, and gossip that may be exceptionally powerful in explaining why mistakes happen, but which more formal methods will miss.” (Dixon-Woods, 2003, pp. 326–327) By investigating practice, observation allows to capture work “as done” rather than “as imagined” (Catchpole et al., 2017).

Besides observation, various types of data collection methods are possible in ethnography, such as interviews or document analysis (Dixon-Woods, 2003; Hammersley and Atkinson, 1995; Huby, Hart, McKevitt, & Sobo, 2007; Leslie et al., 2014). Therefore, the data collected in ethnography is very rich. Because of the tension between the insider and outsider perspectives of the researcher, ethnography forces one to see the situation from different angles, thereby allowing to explore important relationships and to highlight the dynamics of a particular situation (Huby et al., 2007). As such, ethnography is particularly well suited to exploring the influence of contextual factors in complex improvement interventions (Leslie et al., 2014).

Ethnographic observation has been used in OR practice since the early days of the discipline, but there are only few explicit accounts of its use in combination with other OR approaches (Horlick-Jones & Rosenhead, 2007). A few studies combining SSM and DES mention the use of observation to inform OR modelling (Holm et al., 2013, 2017; Kotiadis et al., 2014), but they do not describe what was observed, in which conditions, and how it informed the process or content of the intervention. In the only published explicit account of ethnography used inside an OR project that we are aware of, Horlick-Jones and Rosenhead (2007) describe two cases where they used ethnography—defined broadly as “observation of real-world aspects of operational practice, conducted in the field”—in combination with Problem Structuring Methods (PSMs). They used different data collection methods: shadowing staff, taking notes during everyday activity, having informal conversations, attending meetings, and carrying semi-structured interviews. They insist that ethnography and PSM intervention were not used sequentially but rather in parallel, in an “intertwined” way (Kotiadis & Mingers, 2006). They conclude that the insights gained from ethnographic work helped them greatly in deploying their PSM interventions. It helped them to negotiate the problem focus, to establish their roles as facilitators, to improvise sensibly during their PSM workshops, and to evaluate the interventions. The main difference they see with the traditional use of PSMs is that participants’ accounts are not used straightforwardly, but constitute topics for further investigation, where they are interpreted in relationship with observed concrete actions and practical behaviour.

This example and the use of ethnography in other contexts suggests that ethnographic observation can be promising for OR. However, it remains little studied, and only in combination with PSMs. In this paper, we aim to explain how we used ethnographic observation to support OR modelling using DES and SSM.

Case study

We report the results of a project combining SSM and DES in the outpatient chemotherapy unit of a French hospital. The project was part of a long-term involvement of the first author (GL) as an on-site OR analyst and researcher in this hospital, close to the researcher-in-residence model (Marshall et al., 2014). The analyst engaged with stakeholders continuously and used the “Mode 2” of SSM, iterative and interactive, issue- rather than methodology-focused (Checkland, 2000), to support this process. This arrangement allowed for a lot of ethnographic observations and discussions to take place to frame, inform and support the OR work, discuss issues, monitor progress and debate next steps (Horlick-Jones & Rosenhead, 2007). We now report the project as it unfolded, starting with a messy situation being declared problematic, from which clarity progressively increased on the stakes, issues and possible actions to take.

Setting and initial demand

This project took place in a large public academic hospital in France (1,300 beds, 4,000 employees). This hospital provides the full range of healthcare services for adults. An initial contact was made with the head of the oncology department. He explained that demand in the outpatient oncology unit was rising, and efficiency was deemed to be low. The unit had 30 treatment chairs, whereas days with more than 25 patients were exceptional, and most patients came only for a few hours of treatment. Therefore it should have been possible to reduce the number of chairs in order to give more space to each patient, through increasing rotations on each chair. However, patients seemed to experience very long waiting times that prevented this change.

The situation was described as complex, with different intertwined processes and information systems affecting the delivery and injection of drugs. Waiting times at different stages of the care process were of particular concern. In terms of SSM, an unstructured “problematical situation” (Checkland & Poulter, 2010) had been identified, and it was decided that the OR analyst (GL) would investigate the situation.

Understanding the problem situation

Data collection

The analyst spent extensive time in the field, observing, interviewing and collecting data, including quantitative data on patient waiting times and process durations. The study started in the outpatient chemotherapy unit. The primary objective was to carry a preliminary analysis to map the care process and the patient flow, to quantify waiting times, and to identify potential causes for this phenomenon. However, the observations and conversations quickly broadened in scope, as new issues and topics emerged. Multiple methods were used:

• Observation and informal conversations, to understand the processes and get a sense of the experience of patients and workers in the system.

• Short semi-structured interviews with various staff members, to discuss and confirm the information gathered in observations, and to capture the different perspectives on the situation.

• Time-studies, to evaluate waiting times and process durations.

The time study showed that patients waited on average 1 h and 11 min in the treatment room, for a total time in the system of 3 h and 22 min. Overall, the cumulated waiting time during the observation period represented 35% of the cumulated time-in-system (ie, patients spent on average a third of their time waiting). All sources identified the delivery of chemotherapy drugs as the main blocking point. Patients accumulated in the treatment room without being treated, until the first drug delivery in the morning. Later in the day, some patients would wait hours before getting their treatment, whereas for others the drugs would be ready before they arrived. The chemotherapy unit of the pharmacy department prepared some doses in advance, based on the results of a blood test that patients received two days before their appointment at independent test labs. If the blood tests for a patient were good enough, and if the drugs were stable enough and not too expensive (the cost of chemotherapy doses ranges from a few euros to a few thousands per dose), the pharmacy department would prepare the drugs in advance. However, nurses sometimes had to chase the external lab for late blood test results. Besides, both patients and nurses got very frustrated when they had sent all the required information to the pharmacy, and they still had to wait hours for the drugs to arrive, with little information on the progress of their order. That frustration occasionally resulted in blaming the pharmacy department for the waiting and the congestion in the unit. Nurses often called the pharmacy to enquire about their prescriptions, but overall the drug preparation process remained a black box to them. To better understand the situation, it was decided that the processes in the pharmacy department needed to be investigated.

In the pharmacy department, the analyst observed a complicated situation. The unit in charge of preparing chemotherapies worked for many different departments: oncology, haematology, dermatology… overall, sixteen units had ordered more than a hundred doses in the past eighteen months, with the various units in the oncology and haematology departments accounting for almost 90% of the orders. The pharmacy department had to ensure fair treatment of all units, and to prioritise its production based on a variety of criteria: patient scheduling in all the downstream units, the order of the drugs in each patient’s protocol, giving priority to outpatients over inpatients, and the day of administration of the drugs. To prepare chemotherapy doses, all components need to be sterilised and mixed in a sterile isolator following precise guidelines. Doses are then controlled by a dedicated controller, before they are packed. The whole process involves incompressible sterilisation and control times, which favours batch production over one-piece-flow. Production scheduling was done manually and was considered a complex and demanding task, left to experienced pharmacy assistants. The phone was often ringing, with nurses enquiring about a preparation, cancelling an order or confirming the arrival of a patient. Because of this constant need for coordinating last-minute orders, pharmacists had to postpone tasks that were of lower immediate priority, while still important. For instance, they often could not be present in multidisciplinary cancer team meetings, where all cancer cases are discussed by a multidisciplinary group before treatment is agreed.

For the drugs prepared in advance, ie, before patients had actually seen a physician on the day of their appointment, a major issue was that in 10% of the cases the results of the blood tests were positive but ultimately the patients were not in sufficiently good condition to receive their treatment. If the drugs had been prepared, they were returned to the pharmacy department to be either processed for a reallocation to another patient, or disposed of. Overall, 7% of the drugs prepared were returned to the pharmacy department, half of which could not be reallocated and were ultimately wasted. This was a major concern for the pharmacy, as it incurred a cost for each wasted chemotherapy dose and work on processing all returned doses. Although the cost was ultimately supported by the hospital as a whole, it was the pharmacy and its managers who were responsible for managing this process and ensuring that it worked efficiently. The whole process had also been identified as a potential safety threat, as storing drugs to reallocate them later made the traceability more complicated and incurred a risk of contamination. However, the pharmacy department felt that it could not afford to either stop advanced preparation or throw away all returned drugs.

Analyses One, Two and Three

After the first phase of data collection, the analyst had developed a mental “rich picture” of the situation, and he had explored SSM’s three streams of analysis: analysis of the intervention, “social system” analysis and “political system” analysis (Checkland & Scholes, 1990). This information needed to be somehow formalised in order to be communicated and acted upon. The analyst synthesised his data, and confirmed his interpretations through conversations with stakeholders.

We defined project roles according to Analysis One of SSM:

• Clients: the head of the oncology department, the head of the cancer division.

• Practitioner: first author.

• Owners: oncology staff (represented by nurse leader and head of department), staff of the chemotherapy unit of the pharmacy department (represented by unit manager), head of cancer division, head of pharmacy department, research team (authors). Patients and hospital administrators were not represented in the discussions on the project, but were identified as essential stakeholders in the issue at hand.

We then proceeded to Analyses Two and Three, where the data from ethnographic observations proved most useful. In Analysis Two (“social systems” analysis), we noted the different roles and perceptions of nurses, oncologists and pharmacists in the system. Elements of the care/cure dichotomy between nursing and medicine (Treiber & Jones, 2015), the tension between the aspiration to care and the pressure to ensure efficiency and “flow” (Emes, Smith, Ward, Smith, & Ming, 2017) and the struggling pharmaceutical identity (being often reduced to a “production unit” because of pressure to deliver drugs, but wanting to better support patient care with clinical pharmaceutical expertise) were identified as major points. The view of economic constraints as externally imposed and unanticipated, coming in the way of better practice, was also important.

In Analysis Three (“political systems” analysis), the complex power structure of public academic hospitals was acknowledged (Jarzabkowski & Fenton, 2006). In our situation, oncologists seemed to be more influential than nurses and pharmacists when decisions had to be made. However, nurses remained relatively autonomous in the organisation of daily work in the oncology unit. All decisions involving investment were discussed at the level of the clinical division, before being brought to the hospital administration. Once they had negotiated their resources in yearly reviews, heads of departments were autonomous to organise their departments, and administration would normally not question their choices. Finally, although the relationship between the oncology and the chemotherapy unit of the pharmacy department was very good, a more distanced analysis showed that the two units had potentially conflicting interests:

• The pharmacy department needed to reduce the value of wasted drugs, which was caused by advanced preparation of drugs for patients who were ultimately too weak to receive them.

• The oncology department needed to reduce waiting time, and therefore benefited from advanced preparation since drug delivery was the main source of waiting times.

Following this first analysis, we moved on to conceptual modelling of the problematic situation.

Root definitions and measures of performance

Having spent extensive time in the field, the analyst investigated different perspectives on the situation based on his observations and interviews. Since we were going to build a DES model to analyse patient flows, we first developed a primary task root definition to be used as a basis for the conceptual model of the DES, using the CATWOE mnemonic:

• Customer: patients.

• Actors: pharmacy staff (pharmacists, pharmacy assistants), outpatient oncology staff (nurses, nurse assistants, oncologists), patients, hospital’s internal logistics.

• Transformation: Patient with prescribed chemotherapy and planned treatment date → Patients having received chemotherapy.

• Weltanschauung (worldview): an efficient running of the outpatient oncology process, with a balanced approach between minimal wastage of resources and minimal waiting times, is beneficial to all.

• Owner: the hospital’s director.

• Environment: other independent organisations involved (taxis, ambulances, biomedical laboratories), regulations, budget, IT system.

This definition focuses on a system aimed at delivering a pharmaceutical treatment to patients in a timely fashion. Other, complementary, perspectives were also explored and refined into different CATWOE root definitions of relevant systems, to express the views of particular groups. An illustrative sample is provided in Figure 1. These root definitions emerged from long-term observation and engagement with practitioners, and were developed through observations, conversations and triangulation between sources and methods rather than by directly sitting down with stakeholders to write them. Two root definitions express the views of the pharmacists, first as a support unit preparing drugs for all departments prescribing chemotherapies (not only oncology), and second as a stakeholder in patient care, ensuring that drugs were prepared safely and were adequate for patients, without any adverse effect being generated by inappropriate drug combinations. One expresses a nursing perspective of providing patients with medical treatment while also providing compassionate care.

Figure 1. Sample of CATWOE definitions produced.

These definitions provided useful aide-memoires of the different perspectives that needed to be taken into account for the rest of the project. These root definitions were here to remind us in the next steps of what could be feasible and desirable changes, given the different worldviews in the system. At this particular stage, they helped defining the goals of the project, reflected in the measures of performance that were chosen based on SSM’s Three E’s:

• Efficacy: patients receive the prescribed treatment on the due date.

• Efficiency: a balance is achieved where waiting times and drug wastage are both reduced to acceptable levels.

• Effectiveness: patients get the benefits of their treatment without spending excessive time in the stressful environment of the hospital, nurses can spend more time on nursing care activities, pharmacists get to attend multidisciplinary cancer team meetings more often, the outpatient oncology unit manages to accommodate the increasing inflow of patients.

One can notice how some elements of the CATWOE definitions in Figure 1 are reflected in the “Efficiency” and “Effectiveness” criteria. In particular, the balance between the pharmacy’s objectives and a smooth patient flow is included in the “Efficiency” criterion, while “Effectiveness” accounts for nurses’ aspirations to spend more time on care and less on administrative tasks, and the aspirations of pharmacists to participate more in the clinical processes.

Simulation modelling and scenario analysis

Having defined a primary task root definition and performance criteria for the project, we moved on to build a conceptual model for the simulation (Robinson, 2008). Conceptual modelling started with the definition of a patient flow diagram, which most other studies combining SSM and DES use (Holm et al., 2013; Kotiadis et al., 2014; Lehaney et al., 1999). Our model combined the activities of all participants in the outpatient chemotherapy process, focusing on the flow of drugs and patients. To build it, we first mapped all activities and the associated departments, following the patient and her/his prescription through the system. The result is presented in Figure 2. Some elements do not appear in this graphical model but were also decided upon at this stage and later included in the computer model:

Figure 2. Activity model.

• First, this model only shows the flow of prescriptions for the outpatient oncology unit. However, these prescriptions account for only 37% of the activity of the pharmacy: we needed to represent orders from other departments, since the pharmacy needs to support all departments. We decided to represent the orders of departments other than the outpatient oncology clinic based on empirical distribution of arrivals, and to use a simplified system of priorities with only two levels, “normal” and “priority.”

• Second, this model includes two actors whose activities were not really well understood: the external blood laboratories, which were completely independent from the hospital, and the logistics service that dispatched drugs from the pharmacy to the clinical departments. We decided to model the reception of lab results (in time or late) as an exogenous stochastic variable, and to exclude the possibility of influencing the labs. The logistics service also appeared somehow as a black box in the system, and it was unclear who was responsible for its management. Realistically, this process needed to be considered exogenous rather than endogenous in order to reflect that we had no control over it. Therefore, we decided to model the logistics service as a stochastic process with a distribution of service time based on observed performance but with no associated resources.

The model would include all other resources: nurses, oncologists, pharmacy technicians, nurse assistants. Based on our understanding of the problem situation, on the project’s objectives and on the conceptual model, we built a DES model, using ARENA (Rockwell Automation). In order to validate the model, we compared simulated results to historical values, and we discussed the model and its animation to stakeholders (Sargent, 2013).

Having developed a simulation model, we needed to identify scenarios that could be evaluated. We used a mixed strategy to develop these scenarios: internal ideas from project stakeholders, literature review on the management of outpatient chemotherapy processes, and benchmarking other hospitals to identify interesting ideas.

Internal ideas worked mainly by reorganising existing resources through modified work schedules, or adding resources at some points of the process. The literature was mostly used in combination with benchmarks from other hospitals. The analyst met with six people from four different hospitals which had been identified as “good performers” or as having developed innovative practices, which were then also checked in the scientific literature. Three solution principles emerged from this study:

• Getting information on the patient’s status before she comes for her chemotherapy, so that drugs can be prepared in advance (Bonan et al., 2010; Scotté, Oudard, Aboudagga, Elaidi, & Bonan, 2013).

• Preparing in advance a range of standard, rounded doses of chemotherapy for each drug, rather than adjusting the dose to every individual patient’s body surface area. This practice is called “dose banding,” and equates to using a Make-to-Stock (MTS) rather than Make-to-Order (MTO) policy (Chatelut et al., 2012; Plumridge & Sewell, 2001).

• Automate stages of the process, eg, chemotherapy preparation by purchasing a drug preparation robot, or transportation by using a pneumatic system (Palma & Bufarini, 2012).

Before testing these principles in the simulation, they were first qualitatively evaluated. After a first feasibility evaluation, we retained only some of the scenarios. For instance, the “make to stock” policy was deemed unsuitable at the time of the study, for lack of storage space. Transporting drugs in pneumatic tubes was seen as too risky, with other hospitals reporting experiences of drugs leaking in the tubes. Robotised preparation would have operationally the same impact on preparation time as adding human operators to the drug preparation stage, so we tested the latter option. Figure 3 shows the impact of various scenarios on waiting times. It shows that the most promising principles are to extend the list of products eligible to advanced preparation, and to make more advanced preparation possible. Additional testing on these two parameters confirmed their impact.

Figure 3. Simulated impact of change scenarios on patient waiting times.

Two solution principles allow to act on these parameters: getting advanced information on patient status, and enlarging the list of products eligible to advanced preparation. These two principles are also very complementary: by acquiring better information on patient status, the pharmacy reduces the risk that a patient will in fact not be fit for treatment, despite her blood test results. Therefore, the pharmacy can afford to enlarge the list of products eligible to advanced preparation, without wasting more drugs. Additional information on patient status can be acquired by calling patients with a specific questionnaire, or having them fill the questionnaire in an online application. Either way, we estimated that the system would need a full-time nurse to run.

Adding a nurse would cost around 50,000 euros per year. We sought to evaluate if the cost of this position could be compensated by an increase in the number of patients that could be accommodated in the system, thanks to reduced waiting times. Figure 4 shows the results of simulations of the system with +20% patients compared to baseline, to assess the impact of increasing advanced preparation thanks to the information collected by the new nurse role, first by preparing more of the drugs currently eligible to advanced preparation (from 25% to 70% in Figure 4) and then by extending the list of drugs eligible to advanced preparation (from 84% to 95% in Figure 4). The simulation results show that if the first step can be achieved (increasing advanced preparation from 25% to 70% of eligible drugs), the system can accommodate 20% patients more while still reducing waiting times, which represents an additional 413,000 euros in revenue. If the second step is achieved (increasing the list of drugs eligible to advanced preparation from 84% to 95% of all preparations), waiting times could be reduced, even with an increase of 20% in the number of patients treated.

Figure 4. Simulated impact of increased advanced preparation under increased workload.

To summarise, by adding one nurse dedicated to contacting patients prior to their chemotherapy, the hospital could increase its revenue (investing 50,000 euros for the nurse but gaining 413,000 euros from additional patients) while reducing waiting times. As the number of patients had been on the rise for the past years (at the time of the analysis, +18% compared to the first semester of the previous year), this option was seen as reasonable and economically viable. Even if a 20% increase in patients could not be achieved, an increase of only 2.6% in the number of sessions would break even the cost of the additional nurse. Therefore, we chose this solution principle.

Detailed solution design

Once a solution concept had been identified, the project stalled for a while. It became clear that to move further, the solution needed to be detailed further and refined to a more operational level. The team decided to organise workshops, which the analyst would lead, using the “Service Blueprinting” method of service modelling (Bitner, Ostrom, & Morgan, 2008; Shostack, 1984).

Three one-hour workshops were organised with nurses, oncologists and pharmacists. The workshops systematically explored the proposed process, and the changes that needed to occur to implement it: IT systems, working schedules, costs etc. Some issues were identified and quickly tackled, for instance, by re-communicating to all patients the dates for their blood tests, or by shifting some of the premedication taken before chemotherapy from intravenous to oral, so that patients could take them as pills before coming to the hospital. In this process, the analyst acted both as a facilitator and as a knowledge provider, based on his observations and time spent in the outpatient unit and the pharmacy. The knowledge he had acquired helped steer debates and challenge ideas, by injecting information in the discussion in the form of anecdotes or figures.

The workshops allowed for the definition of a service blueprint for the new organisation of chemotherapy prescription, mixing and delivery. The final blueprint is presented in Figure 5.

Figure 5. Blueprint for the redesigned chemotherapy administration process.

Working towards implementation

Once a detailed new process had been defined, all the elements were present to make a case for this project. Executives in the hospital had been aware of the existence of the project and of its objectives, but not of its specifics. We produced several documents to present our proposal and the evidence supporting it. A meeting was organised with senior executives, who supported the project and proposed to trial the new system for a limited evaluation period.

However, it seems that various elements then interfered with the implementation of the new process. The main enabler to this change should have been the recruitment of a nurse. We had shown that the costs could be met by a small increase in activity. Yet at that time, the activity in outpatient chemotherapy seemed to reach a plateau. This project also conflicted for attention with a major development project, aimed at building new facilities and renovating others. Finally, strikes in public hospitals at that period probably also made it difficult to put this project on the agenda of hospital administrators. External funding was secured to develop an online solution that would allow to acquire patient information before chemotherapies, but the contractual arrangements happened to be more difficult than expected. Ultimately, although this last development was still ongoing, 18 months after the end of the project, the new process had not been implemented.

Discussion

We analyse this case study on three dimensions. First, we investigate what “worked” and what did not in the project, using OR evaluation frameworks. Second, we discuss the role of each method in the intervention. Finally, we analyse how they were combined and integrated.

Did the intervention “work”?

Given the non-implementation of the recommendations at the end of the study, the first question that arises when looking back on this project is: what “worked” and what did not in this project, why, and how? To answer, we apply two evaluation frameworks to the intervention, one for PSMs and one for simulation studies.

First, we use the framework proposed by Connell (2001) for evaluating SSM interventions, which separates between success in structuring the problem and in implementing change. Regarding the problem structuring dimension, there is evidence that the intervention did help shape the issue and identify the main elements of the problem. The issue of waiting times had been present for at least six months, and data on this problem had been collected a few months before we started the project, but it did not yield any insights at that time. The integration of “Hard” and “Soft” OR and the application of a “process management” perspective (Silver, 2004) that crossed departmental boundaries helped in structuring data and presenting it in ways that helped stakeholders make sense of it (Crowe, Turner, Utley, & Fulop, 2017). With this project, stakeholders realised interconnections between departments, and the project allowed the first structured meetings dedicated to operational issues between the pharmacy and the oncology unit in a long time. DES allowed to evaluate proposed changes quickly before implementing them, and to filter ideas. A senior clinician told us at the end of the project that a major product of the study was that it had proved that it was possible to model hospital activities, which suggests that the modelling activities were deemed useful. Therefore, we consider that the intervention was successful in helping structure the project. When it comes to implementation, ultimately, the problem situation was not successfully resolved. The intervention provided all the elements to move forward, should funding be allocated, but the newly designed process was not put in place.

We can also assess the project along the four stages of success in simulation projects defined by Robinson and Pidd (1998). The results are as follows:

1. “The study achieves its objectives and/or shows a benefit”: yes. The project showed benefits in helping to structure the problem situation, generating and appraising options for change, detailing the proposed change and synthesising recommendations.

2. “The results of the study are accepted”: yes. All project members accepted that the recommendations resulting from the study pictured a desirable change.

3. "The results of the study are implemented”: no. The change suggested by the study seemed unfeasible in the prevalent budgetary and political context at that time.

4. “Implementation proved the results of the study to be correct”: could not be assessed.

Taking into account the two assessments, we may conclude that the project generated insights and learning, without resulting in immediate practical changes. Therefore, although the results were not implemented, the project “served a purpose: it allowed ideas to be generated and evaluated” and “it successfully engaged a lot of relevant people in this endeavour” (Ormerod, 2001, p. 1178). The project also demonstrated how other possible changes (eg, adding nurses or drug preparation staff, or changing working schedules) were inferior to the solution proposed, and may have prevented these inferior solutions from being implemented.

Few studies combining SSM and DES report organisational changes made as a consequence of the intervention. Lehaney et al. (1999) and Holm et al. (2013) briefly describe the results of their study being implemented, while Tako and Kotiadis (2015) and Holm et al. (2017) mention decisions being taken. However, these authors do not provide much detail on implementation. Therefore, it is difficult to know to what extent these projects led to the implementation of the suggested changes.

What did each method contribute to the intervention?

Several methods were combined in this project: ethnographic observation, SSM, DES and service blueprinting. Although methods were integrated (we describe how later), it is worth discussing each method’s distinctive contributions to the intervention.

Soft systems methodology

Our use of SSM mirrored that of other studies combining SSM and DES, where SSM provided both a framework to guide the project (Holm et al., 2013; Lehaney et al., 1999; Tako & Kotiadis, 2015), and specific tools to infuse a “Soft” perspective in the intervention (Holm et al., 2017; Kotiadis et al., 2014; Pereira et al., 2015; Tako & Kotiadis, 2015). We explore how SSM provided a framework that supported the integration of methods in the next sub-section (“How were methods combined?”). Aside from this, applying SSM led us to see the problem situation as socially constructed, and to explore different worldviews. To do so, we relied on the four-activity process of SSM as a guideline, on Analyses I, II, and III and on root-definitions as practical tools for collecting, classifying and analysing data.

Through Analyses II and III, SSM helped identify power imbalances and cultural differences between stakeholders, even before we framed the problem as a multidepartment issue where these dimensions are crucial. These two devices are very simple but also very flexible, and they helped us to maintain our eyes open for elements of cultural or political relevance while conducting, and then when analysing data.

Then, by explicitly writing multiple root definitions, the analyst clearly established for himself different worldviews as legitimate for the project, and explored differences in perspectives and interests at all levels, from the very operational (the difference in interests of the oncology department and the pharmacy regarding advanced preparation) to the more philosophical (the tension between operational efficiency and care). We believe that defining multiple root definitions was helpful in surfacing the pharmacy’s perspective on the issue at hand, and in explicitly integrating the nurses’ interests in the project goals. It also helped identify some of the issues encountered by nurses, in particular the frustration of having a room full of patients waiting for their drugs, without nurses being able to help. Holm and Dahl (2011) went as far as to discuss “taboo perspectives,” including a commercial viewpoint where patients are the source of revenue of the hospital. We did not include this viewpoint at this stage of the project, but it could have been interesting.

The results of Analyses I, II, and III, and the root definitions were used at various points of the project; they constituted a “knowledge base” (Clarkson et al., 2004) to frame and analyse subsequent observations, and aide-memoires on what seemed desirable to the various stakeholders. These analyses and the staged approach of SSM helped us to focus on high-level objectives (what we ultimately want to achieve, what “good” would look like in our context), and kept us focused on the issue rather than on the technique of DES and its underlying logic of looking at a situation as a network of interconnected flows of entities processed by shared resources. This appears in the way we defined the “effectiveness” criterion for the project, which incorporates the aspirations of the pharmacy and the nursing team to dedicate more time to tasks that they currently had to leave aside in order to keep the process working. Structuring the approach and being sensitive to cultural and political aspects also prevented us from developing early on a simulation model of the outpatient clinic alone, which most studies of outpatient chemotherapy delivery have done (Lamé, Jouini, & Stal-Le Cardinal, 2016).

Ethnographic observation

Ethnographic observation was used to collect data all along the project. To summarise, observations were performed in 16 sessions of one full day of observation of one activity—2 in the consultation rooms, 2 in waiting rooms, 4 in the pharmacy, 8 in the treatment room. The cumulative observation time for these “structured” observations is approximately 112 h, with multiple informal exchanges outside of these observation periods. During the rest of the study, the process was less structured but we continued to analyse observations, impressions and interesting conversations gathered in interviews and meetings or through “hanging around” in the units.

The main benefits we see in this approach are its capacity to investigate everyday practice rather than the image stakeholders have formed of it, and the way it helps establish trust and the position of the analyst. We observed people experience problems and frustrating events, which they sometimes do not mentioned in interviews because they had internalised them as “the way the system works.” In interviews and formal conversations, people can tend to mention what they think the interviewer is interested in, or refrain from mentioning certain topics. In everyday life, they more naturally talk about what matters to them “here and now,” which helps to capture “work as done” rather than “as imagined” (Catchpole et al., 2017). This happened in the pharmacy, where the phone rang every few minutes at some points during the day, and last-minute orders frequently disrupted production management. In the treatment room, the contrast between relatively quiet days and busier ones showed how nurses could take advantage of the additional time available to provide a support to patients through informal discussions and questioning about treatment and daily life. Observations also revealed how tense relationships could get between the pharmacy and the nursing team on busy days. Observing practice and getting insights into people’s daily activities is important in the context of healthcare, where organisations are particularly complex and politically pluralist (Jarzabkowski & Fenton, 2006; Klein & Young, 2015), disciplinary and professional boundaries are strong (Liberati, Gorli, & Scaratti, 2016), and the importance of hierarchies and the fear of speaking up are well-identified (Morrow, Gustavson, & Jones, 2016; Palanisamy & Jenkins, 2015; Walton, 2006).

Gathering observations also allowed the analyst to challenge the assumptions behind people’s narratives (Hammersley & Atkinson, 1995). This took the form of informal questioning, relating their accounts to observed practice: “When I observed this part of the unit, I have noticed this happening. How do you interpret it? Can you tell me more?” or “What you describe makes me think of this observation, is it the same thing? Is it the kind of situation you were referring to?” Together, these elements contributed to a cycle of observing and having people react to data. This was especially the case when discussing the processes of one unit with members of another, or during meetings. In the beginning of the project, both units largely ignored the other’s processes, issues and ideas, for lack of time and communication available to discuss these topics. Nurses sometimes blamed pharmacists for drugs being late, while in the pharmacy staff would get frustrated of being chased on the phone for products they were already doing their best to deliver in time. The knowledge accumulated by the analyst on both units involved in the situation (the pharmacy and the outpatient unit) helped elicit assumptions, question hypotheses or challenge ideas. This also helped in assessing the feasibility of change scenarios, some of which were discarded before even being simulated because they would have been very challenging to implement.

An additional benefit of carrying observation was to establish a trust relationship between the analyst and the various categories of problem owners involved in the project (oncology and pharmacy staff). The presence of a stranger taking notes (especially during time studies, when the analyst noted process durations) was first looked at suspiciously. However, showing the notes, sticking to observation, spending long periods “where the work is done” seemed to help to overcome initial reluctance. Our interpretation is that the time spent in observation helped build a legitimacy as a “field person” for the analyst, who had initially been introduced as “an industrial engineering researcher,” combining the negative perception of academia (as an office-based, theoretical profession remote from practical concerns) and industrial engineering (as a Taylorist, mechanistic discipline oriented towards cost cutting). This role-building dimension of participant observation was also noted by Horlick-Jones and Rosenhead (2007). Long-term involvement could also help to attenuate the “observer effect” (sometimes called “Hawthorne effect”), by which people modify aspects of their behaviour because they are being observed.

Finally, getting to know the way people worked and talked about their activity was useful when presenting findings at various stages of the project. The choice of formats, the presentation of data, the vocabulary and the associated comments were all informed by the observations. Communication gaps are important issues in healthcare OR (Jahangirian, Taylor, Eatock, Stergioulas, & Taylor, 2015), but OR analysts can also play a role of knowledge mediation in this context (Crowe et al., 2017). In this project, ethnography clearly supported this role. This phenomenon has also been noted in “embedded research” more generally (Vindrola-Padros, Pape, Utley, & Fulop, 2016). It suggests that beyond the “methods” used in interventions, we may want to also pay attention to the way OR analysts engage with groups, including the timescale of engagement, the type of links with the group and the types of interactions developed outside core OR activities.

Simulation

Simulation was the most “visible” part of the analyst’s work. DES provided a safe testbed for evaluating scenarios, dispelling ineffective proposed solutions, and assessing the economic impact of the proposed re-organisation. The quantitative analysis performed with DES was the core element of our final argumentation for implementing the proposed changes.

Aside from encompassing two hospital departments, a rare feature in healthcare simulation studies (Gunal & Pidd, 2010), the simulation modelling part of the intervention was classic. We were confronted with the usual problems for acquiring data in order to calibrate and validate the model (Tako & Robinson, 2015), such as missing data and limited timespans, which we addressed by confirming our data with expert opinions from stakeholders. Our long-term involvement in the hospital helped us mitigate other frequent issues, such as lack of stakeholder engagement or clients’ shortage of time (Jahangirian et al., 2015; Tako & Robinson, 2015).

Service blueprinting

Finally, service blueprinting provided a format for creating a shared representation around which to focus group discussions. The final blueprint synthesised our proposition, and allowed it to be stored and communicated easily. Flowcharts are well-established in healthcare, and generally appreciated by practitioners (Jun, Ward, Morris, & Clarkson, 2009), so the method was well-accepted. However, we do not think that the graphic formalism of service blueprinting was particularly influential here. It is quite likely that any activity-based process modelling format (eg, Business Process Model and Notation (Milton & Johnson, 2012)) would have worked just as well.

Few studies combining SSM and DES report on post-modelling activities (Holm et al., 2013; Lehaney et al., 1999; Tako & Kotiadis, 2015). Tako and Kotiadis (2015) introduce a facilitated implementation workshop, where possible changes are discussed and assessed for feasibility and desirability. In our case, we needed to detail the solution to systematically identify all the practical changes that were needed to implement the chosen concept. This may be because the three aforementioned projects dealt with resources allocation or scheduling, whereas in the present project, the process needed to be re-engineered, which required some more thoughts and detailing.

How were methods combined?

In this project, we used ethnographic observations, SSM, DES and Service Blueprinting to support an improvement project in an outpatient chemotherapy unit. Although this could be described as “a simulation project,” as the simulation model was the most “visible” part of the project, SSM really formed the backbone of the intervention and was used as a framework to integrate other methods. Table 1 shows how the different stages of the project mapped onto the four activities of SSM, as defined by Checkland and Poulter (2010).

Table 1. Mapping the case study to the four activities of SSM (Checkland & Poulter, 2010).

SSM stages	Project stages
I. Finding Out	1. Observation and interviews, time-study	4. Visits and interviews in other hospitals
II. Making Purposeful Activity Models	2. Multiple root definitions, conceptual modelling, DES modelling	5. Defining scenarios and simulating them through DES	8. Design of new service blueprint
III. Using Models to Structure Discussion About the Situation and Its Improvement	3. DES validation	6. Debate on scenarios	9. Discussion of impact of new process and actions needed to get there
IV. Defining “Action to Improve”		7. Choice of solution principle	10. Building the case for the proposed change and bringing it to executives

This analysis shows DES as embedded in a broader intervention, supported by SSM. From this, it could be deduced that this project was another case of a “soft with a hard centre” OR intervention (Robinson, 2001), where the Hard OR methods are subdued by the Soft OR perspective. However, DES was only one type of model used in the project, and not the only instance of “hard” thinking, as we also performed time studies and developed a service blueprint. When looking closely, at each stage, the collection of qualitative data through observations and interviews was complemented by the collection of quantitative data, and the analysis alternated between an interpretive approach and a more objectivist approach. Even the definition and analysis of the scenario simulated involved both a numeric analysis and a notion of feasibility and desirability. Rather than one dominating the other, “Hard” and “Soft” thinking were intertwined all along the project (Kotiadis & Mingers, 2006).

In some other studies combining SSM and DES, the Soft and Hard methods are used sequentially, with SSM used mostly to prepare the simulation work (Holm et al., 2017; Lehaney and Paul, 1996; Pereira et al., 2015). Our approach to combining methods is closer to that of Tako and Kotiadis (2015) and Holm et al. (2013), where elements of the Soft and Hard paradigm are continuously balanced. Our main difference with the latter studies is that we did not use facilitation to the same extent. Indeed, we organised workshops only late in the process, to develop a service blueprint for the new process. In the early stages, we relied on ethnographic observation to identify worldviews, values and interests. This is different from the deliberative ideals close to Soft OR (Mingers, 2011). However, research has shown the prevalence of strong hierarchies and difficulties to “speak-up” in healthcare organisations (Morrow et al., 2016; Palanisamy & Jenkins, 2015; Walton, 2006). Deliberative methods such as SSM may not be enough to tackle such situations (Mingers, 1980, 2011; Rouwette & Smeets, 2016), whereas ethnography has been used to identify and analyse worldviews and power struggles in such contexts (Conn et al., 2016; Liberati et al., 2016). This moves away from “orthodox PSM practice,” as noted by Horlick-Jones and Rosenhead (2007, p. 599), but it seems suitable to support a process which could be hijacked if only managed through facilitated workshops in a tense political context. Ultimately, we do not argue that ethnographic observation should replace facilitation, but rather that it could come as a useful complement by providing the benefits we identified. The future may be in combining ethnographic approaches with facilitation, rather than favouring one over the other.

Conclusion

In this article, we reported a case study where we combined SSM and DES to help improve waiting times in an outpatient chemotherapy unit. This project contributes to the accumulating evidence on combining DES and SSM. It shows how SSM can support in-depth investigation of the problem situation through framing ethnographic approaches early in the project. Through the investigation streams of Analyses I, II, and III and the definition of several root definitions, these early observations constituted a knowledge base which was influential for the rest of the project. Much value was derived from combining an ethnographic approach with SSM and DES.

As others have noticed when combining SSM with DES, the cost can be high (Lehaney et al., 1999). This is even more so when the project also involves extensive data collection through ethnographic observations. The analyst dedicated a significant part of his time to the project, which lasted 18 months. This approach is probably best suited when internal improvement teams are leading improvement projects, or within researcher-in-residence arrangements (Marshall et al., 2014).

This case study shows a promising combination of Hard and Soft OR methods with an ethnographic approach drawn from social sciences. The combination of Hard and Soft OR methods is now well-established (Howick & Ackermann, 2011; Munro & Mingers, 2002), and this paper adds to a specific literature on combining SSM and DES. However, the combination of OR techniques with methods and theories from social sciences is much less common. The interface between OR and social sciences was explored in the OR community in the past (Jackson, Keys, & Cropper, 1989; Lawrence, 1966), and recent papers suggest a renewal of interest in the methods and theories of social science to study or explain OR practice (Becker, 2016; Brocklesby, 2016; Franco & Greiffenhagen, 2018; Ormerod, 2014). In contrast, the use of social science methods and theories inside OR projects is much less developed, with the notable exception of the series of papers by Horlick-Jones et al. (2001) and Horlick-Jones and Rosenhead (2002, 2007), who use Soft OR approaches in combination with ethnography. We argue that this constitutes a complementary and exciting perspective for new developments in OR, and that future studies could investigate the use of methods for data collection and collation and of theories from the social sciences inside OR interventions, in combination with our traditional OR approaches.

Acknowledgments

The authors thank Dr. Kathy Kotiadis for her very useful comments on previous versions of this paper. All potential mistakes remain our own. Elements of this work were presented at the 2016 Winter Simulation Conference in Washington, DC, and at the 2017 International Conference on Engineering Design in Vancouver.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

GL is supported by the Health Foundation’s grant to the University of Cambridge for The Healthcare Improvement Studies Institute.