Reconsidering critical appraisal in social work: choice, care and organization in real-time treatment decisions

Abstract

This paper seeks to provide an empirically grounded discussion of the critical appraisal model of Evidence-based practice (EBP) in social work practice. Studying real-time treatment decisions, the paper looks ethnographically at an attempt to implement critical appraisal in everyday social work practice, and problematizes some of the assumptions underlying this idea. Whereas critical appraisal tends to view treatment decisions as clear-cut events emanating from autonomous social workers, participant observation shows that decisions emerge over time and are ‘organizational’ rather than individual. Drawing on the notion of ‘logic of care’ and findings from studies of organizational decision-making, a more practice oriented understanding of treatment decision-making is outlined.

keywords:

• Evidence-based practice
• decision making
• critical appraisal
• ethnography

Introduction

Evidence-based practice (EBP) was launched in order to improve professionals’ clinical decision-making regarding patients and clients (EBM Working Group 1992). Originating in medicine, this idea has proliferated to the field of social work where it has been embraced by many scholars (see Gambrill 1999; Sheldon and MacDonald 1999; Gibbs and Gambrill 2002), although not univocally (see Webb 2001). As EBP has been disseminated, it has also been subjected to a wide range of reinterpretations, which sometimes has created confusion as to what this popular acronym really refers to. Attempting to fight this confusion, several scholars (e.g. Shlonsky and Gibbs 2004; Gambrill 2006; Thyer and Myers 2011) have defended what they see as the original interpretation, which presents EBP as a decision-making process in which practitioners shall integrate the ‘best research evidence with clinical expertise and patient values’ (Sackett et al. 2000, 1). This interpretation, which can be described as the current dominant model of EBP in the social work literature, will in the following be referred to as the critical appraisal model of EBP and is the focus of this article.

There are different ways of describing the critical appraisal model, but the original idea is that the social worker shall, based on the client’s problems, integrate three factors in decisions about the care to be offered: research evidence, the client’s values, and clinical expertise. The decision procedure has been described as the following sequence of steps (Sackett et al. 2000):

1. Define an answerable practice question.

2. Search for evidence to answer this question.

3. Critically appraise the relevant evidence found.

4. Integrate this with the professional’s clinical expertise and the client’s values in deciding on an appropriate intervention.

5. Evaluate the outcomes of this intervention.

Later, in a much influential modification of this model, Haynes, Devereaux, and Guyatt (2002) have added a fourth factor, ‘clinical state and circumstances’, to the original three. This is exemplified as the availability of treatment options in a clinical setting.

In the critical appraisal model, the idea is that the social worker has a relatively autonomous role in searching for and critically appraising evidence and subsequently making decisions. This can be compared with the ‘guideline model of EBP’ (Bergmark, Bergmark, and Lundström 2012) in which the social worker has a less autonomous role in decision-making and relies on reviews and clinical practice guidelines produced by experts (cf. Howard and Jenson 1999; Guyatt et al. 2000).

A growing body of research is concerned with social worker attitudes, skills and knowledge relating to various aspects of the critical appraisal model of EBP. Survey studies suggest that a majority of social workers in countries where the evidence movement has gained a foothold support the basic idea of EBP, but that they rarely search for or apply research findings in their clinical decision-making (Bergmark and Lundström 2002; Morago 2010; Pope et al. 2011; Gray et al. 2013). Studies exploring attempts to implement a critical appraisal model of EBP have shown several barriers to implementation (Bellamy et al. 2008; Gray et al. 2012). Among the most frequently cited are inadequate organizational support dedicated to EBP, deficient skills and knowledge on the part of the social workers, and insufficient evidence. In sum, then, it seems that the critical appraisal model is a highly regarded idea, but that it is difficult to implement in social work practice. This has led several scholars to appreciate the complexities of implementing EBP and to argue for more multifaceted approaches to supporting EBP in practice (Manuel et al. 2009; Gray et al. 2012).

In the literature, there are two peculiar omissions. First, there is a lack of research examining actual practice (Plath 2012; Smith 2014a). Most studies use surveys and interviews as a way of investigating social worker attitudes, skills and knowledge of EBP, but not real-time decision-making practices. Second, the critical appraisal model is often taken for granted as a desirable idea, despite bourgeoning findings showing the difficulty of implementing it.

While critiques and reformulations of EBP mainly have been informed by general theoretical insights about knowledge and clinical practice (cf. Webb 2001; van de Luitgaarden 2009; Nevo and Nevo-Slonim 2011; Petersén and Olsson 2014), this paper seeks to provide an empirically grounded discussion about the applicability and desirability of the critical appraisal model of EBP in social work practice. In order to fully appreciate the challenges of being ‘evidence-based’, we need to have an empirically informed conception of how decisions actually are arrived at in social work practice. This paper looks at real-time decision-making processes in a social services agency determined to work in line with the critical appraisal model. Taking seriously the empirical reality of treatment decision-making, this paper asks some deceptively simple but fundamental questions about treatment decisions in real-time practice: When are decisions made? Who makes decisions, based on what? By turning attention to decision-making as it actually occurs in practice, it is possible to rethink the current idealized and somewhat unrealistic demands that the critical appraisal model puts on practitioners and on the social services.

In the next section, I will outline the theoretical concepts used in the article to understand decisions in real-time practice and their relation to the critical appraisal model. This is followed by a description of the case I have studied and the methods and materials I have used to analyse the agency’s decision-making practices.

Considering different logics in social services treatment decisions

In this paper, I use the concept of logic to analyze how treatment decisions actually happen in practice and to compare this with how critical appraisal describes decisions. Mol (2008) describes the ‘logic of choice’ as a widely celebrated ideal that informs how many ‘solutions’ to the problems of professional decision-making are framed (see White and Stancombe 2003; Smith 2014a). The logic of choice assumes an autonomous rational actor with stable preferences and knowledge of decision alternatives and their consequences. Critical appraisal is a clear exponent of this logic in that finding the best treatment for a given client is described as a matter of individual rational choice between a set of clear alternatives (e.g. Haynes, Devereaux, and Guyatt 2002). Moreover, the decision steps of critical appraisal construct this choice as being clearly defined in time. That is, the steps are followed, and the choice is made and executed.

As an alternative to the logic of choice, Mol (2008) has outlined the ‘logic of care’ as a more appropriate way of understanding how treatment decisions are made in healthcare practice. The ultimate goal of this logic is to make daily life more bearable for clients. Compared with the logic of choice, which assumes decision-making to be clearly defined in time, the logic of care views decision-making as an ongoing process in which professional activities attend to the often unexpected events in the clients’ treatment trajectory. The logic of care, primarily focused on the professional-patient relationship, is useful for analyzing aspects of the client-social worker relationship in making treatment decisions.

However, in a bureaucratic organization such as the social services, there is also a need to consider organizational aspects, which is not addressed in the logic of care. Therefore, I introduce the concept of ‘organizational logic’ in order to fully be able to grasp the decision-making processes within the agency. This concept is derived from studies of organizational decision-making, which similarly describe these decisions as open ended processes, famously depicted as ‘muddling through’ (Lindblom 1959).

The organizational and care logics differ from the logic of choice in two crucial respects. First, they do not assume a simple connection between means and ends, that is, between treatment and the goals they should further. The logic of choice suggests that a rational treatment decision should be made by assessing treatment alternatives according to a stable set of preferences, for example client preferences and research evidence. However, the organizational and care logics highlight that clients’ preferences are ambiguous and changeable, as unexpected events happen during the treatment trajectory. Moreover, conflicting and ambiguous organizational rationales also shape how clients’ needs and preferences are interpreted during this process. This implies that there is no single moment when all relevant facts and preferences are available. In fact, what counts as relevant evidence, preferences and organizational rationales in a treatment decision is not external but internal to the decision-making process; since essentially defined along the way (Sjögren 2006; Mol 2008).

Second, the logic of choice assumes an individual autonomous actor making a choice, but in line with an organizational logic a social worker cannot make treatment decisions entirely on her own since there are laws and regulations that shape what can be done. These rules express different organizational rationales that, apart from the clients’ needs and preferences, must come together in a treatment decision. Thus, the social worker cannot be seen as an autonomous decision-maker, but neither is s/he completely constrained, since these organizational rationales are ambiguous and often defined relationally within each decision process (Lindblom 1959; March 1988).

The concepts of logic of care and of organization are used in this paper to facilitate other ways of seeing rational treatment decision-making, other than rational choice. Whereas the critical appraisal model suggests a universal rationality that seeks to formulate prefixed rules, the logic of care and the organizational logic suggest another kind of rationality based on a situational judgement of each case. Seeing the organizational and care logics as more appropriate ways of understanding treatment decisions does not mean that rational choices are altogether impossible, but rather that the limits of this logic need to be considered in order to improve decision-making practices.

The case and methods

In order to study decision-making in practice, I conducted ethnographic fieldwork in a large Swedish social services agency providing a wide range of services to adults with substance use problems. This particular agency was selected because it for several years have worked extensively with EBP implementation in their routine work. Their ‘evidence-based’ commitment started in 2007 when they initiated a two-year project together with the Swedish National Board of Health and Welfare (NBHW) with the objective of implementing EBP in different respects. Since then, the agency has dedicated a great deal of attention and resources to EBP, among other things to implementing evidence-based decision-making in line with the critical appraisal model of Haynes, Devereaux, and Guyatt (2002). This agency can thus be said to be a ‘critical case’ (Flyvbjerg 2001), which means it is of strategic importance with regard to the idea of critical appraisal in social work. Since the agency has worked with this extensively over a long period of time, implementation difficulties are not likely to be the result of poor effort, which allows for a more general discussion of the possibilities, limitations and preconditions for critical appraisal as a viable idea in social work.

The agency consists of four main units: (1) intake, (2) investigation and treatment planning, (3) psychosocial outpatient treatment, and (4) reinforced aftercare. All professionals within the agency, including managers, are qualified social workers with at least a bachelor’s degree. The clients of the agency, who are a relatively marginalized group of persons with substance use problems, typically go through a chain of care. During the first meetings, an intake social worker assesses the client’s problems as well as his or her motivation to receive treatment. Possible interventions are also presented and discussed. But it is the investigative social workers who are responsible for drawing up a treatment plan together with the client, something which is developed during the course of a couple of meetings. Most clients end up in the outpatient unit receiving psychosocial treatment such as Cognitive Behavioral Therapy CBT), twelve step treatment, and other kinds of counselling, often in combination with different kinds of housing services. Clients with more severe problems are sometimes provided with external inpatient or residential treatment, where they receive accommodation and food as well as treatment.

The empirical material for the present analysis draws from ethnographic fieldwork in the social services agency, conducted between April 2011 and December 2012. Divided into three ‘rounds’ of focused fieldwork, I spent about 400 h in the field. During fieldwork, I observed and wrote fieldnotes about the agency’s daily work, interviewed social workers and managers in all units, and analyzed local documents used in the agency’s work. Given the focus on frontline decision-making, I followed social workers in different situations: in informal discussions, in client conferences where social workers and managers discussed cases, and in meetings with clients. However, as became clear during this fieldwork, social workers’ treatment decisions are dependent on the organizational context. For example, there is a politically agreed upon ‘delegation of decision-making’ that regulates who is allowed to make certain kinds of decisions. The social workers are formally allowed to make most decisions by themselves, but costly interventions such as inpatient treatment must be granted by the agency manager, and compulsory treatment can only be decided on by the ‘Social Welfare Board’, a political committee within the social services. Further, there are also local guidelines regulating which housing arrangements the clients are entitled to. To account for decision-making within the agency it was therefore necessary to analyze such documents. All participant observations were conducted following informants’ verbal consent. Further, names and biographical information about the informants have been changed so as to ensure anonymity.

During fieldwork I employed different strategies of participant observation. Some days I ‘shadowed’ (Czarniawska 2007) social workers, a technique for doing fieldwork with people on the move. I followed different kinds of social workers in the agency during an ordinary workday, as they made home calls, went to meetings, talked with colleagues and managers, opened mail or collected documents from the fax machine, or sat in their office talking on the phone or working on the computer (not always very eventful, but still informative). Some days I attended specific meetings that I previously had identified as especially interesting. Meetings with clients and client conferences in which social workers discuss cases have proved to be the most fruitful meetings for the purposes of this study.

In addition to participant observation, I also conducted eight formal interviews with social workers in different roles. Although this paper centers on decision-making practices, these have served the purpose of articulating the informants’ perspectives, and validating or refining my preliminary understandings of their decision-making.

Fieldnotes and interview transcripts were analyzed through rough thematic coding with the aid of NVivo. Analyzing the material, I tried to single out events where decisions were made (which were actually very hard to find) as well as discussions and deliberations about clients and possible treatment alternatives. Following this, I sought to understand what the social workers were actually deciding on as well as the factors in play that were necessary to observe for arriving at a decision.

The research was funded and ethically approved by the Social Sciences Faculty at Stockholm University.

Results

During fieldwork at the social services agency, I often came across the Haynes, Devereaux, and Guyatt’s (2002) figure that describes the elements of critical appraisal. Knowing that this somehow reflected a commitment to EBP, I showed the figure to one of the managers, who has been a driving force in the agency’s work with these questions, and asked her what it meant. ‘Every co-worker should know about this! It’s like the foundation of evidence-based practice’, was her response. Thus, this figure represents managerial pressure on the social workers to make decisions according to the critical appraisal model. In line with this managerial ambition, several measures have been taken to make sure that this is realized in practice. An investigation template has been developed that specifies a set of headings that should be included in the written investigation report. At the very end of the template is the heading ‘assessment according to evidence-based practice’, under which a series of sub-headings are formulated that capture the elements of critical appraisal. New social workers at the agency get an instruction in EBP by a senior colleague who talks about how to incorporate EBP in investigation of cases and treatment planning. During this introduction the investigation template is also presented. Further, the social workers are expected to read a book about EBP in the social services (Oscarsson 2009), which has been bought in several copies for the purpose of increasing understanding of this decision model.

This illustrates the top-down structure of the agency’s work with EBP. It is a commitment that is managerially driven. In fact, the entire EBP movement in the Swedish social services is characterized by a similar pattern, in which the central government via the Swedish National Board of Health and Welfare (NBHW) forcefully has been pursuing this issue (see Bergmark, Bergmark, and Lundström 2012). This is thus an idea that has been taken up at the highest political level and then been translated down to the social services managers of the present agency. However, when it comes to the actual practices of the social workers at the agency, there are virtually no traces left of critical appraisal. In fact, during my fieldwork in this agency I did not observe a single case where the critical appraisal steps were followed; something, however, which does not seem unique to social work (see Gabbay and le May 2004).

In an agency so committed to EBP and worked with it systematically for several years, why are almost no decisions made in accordance with the critical appraisal model? As important explanations previous research has suggested a lack of organizational resources to support EBP implementation, or lacking social worker skills, along with negative attitudes toward EBP (Manuel et al. 2009; Gray et al. 2012). But based on participant observation of decision-making practices in this agency, my answer is rather that critical appraisal builds on a poor understanding of how treatment decisions are actually arrived at in social work practice. In the not always straightforward logic of care it is difficult to follow linear decision models such as critical appraisal.

When are decisions made?

In this section we will look at social worker decisions as they appear in real-time practice. While critical appraisal constructs treatment decision-making as a linear, stepwise process, I will argue that decisions are not always clear-cut but emerge gradually and tend to be reformulated over time.

Trying to understand decision-making during fieldwork, I was often confused about the elusiveness of the decisions being made. Although this was my main focus, I was often surprised that I did not capture any clear-cut decisions in my fieldnotes. After a long day shadowing a social worker, we had a conversation in her office about critical appraisal and treatment decisions in her work. She was one of the more ambitious social workers at the agency and was trying to make sense of critical appraisal. She showed me a NBHW flyer describing the five steps of critical appraisal:

‘It’s so much more than these steps’, she says. ‘You jump back and forth. The investigation is merely one small part; you make so many decisions along the way’. She says further that it is impossible to base every decision on evidence. She takes an example from earlier that day when she spoke briefly with her manager in the hallway about a client who risks being evicted because of her drinking and hashish smoking. The manager argued that it was important that she should not solve the client’s housing situation at once, since this may be a factor that might motivate her to quit drinking and smoking. ‘This is one way to look at it’, the social worker says ‘But is it scientific?’

Acknowledging the difficulties of realizing critical appraisal in her work, the social worker addresses the central point that I make in this section, namely that clinical decisions are not made once and for all but are made in small chunks that eventually result in the clients getting treatment and other support. In the case that she describes, the decision not to solve the client’s precarious housing situation is made in passing, in a chance meeting with the manager. Thus, in real-time practice, decisions emerge through a series of interactions with clients, managers, and other professionals.

Through the chain of care, a sense of the client’s problems and what to do about them emerges, which is an inseparable part of the final treatment decision (cf. White and Stancombe 2003; Smith 2014a). As support in the decision-making process, the investigative social workers regularly have client conferences in which they discuss possible treatment alternatives together with a manager. But decisions need not be made there either. As we saw in the excerpt above, a decision, or at least a part of it, may be made in passing in more informal situations.

Another aspect of the temporal structure of decisions is when things do not turn out as planned. A decision may have been made and suddenly everything may be turned upside down, as in this case discussed at a client conference:

S (social worker) needs help to think what to do with her client who was sup-posed to enter a residential treatment center the day before, but who did not make it there. When S came to the client’s apartment to drive him there, he just stood confused in the hall and had not packed his belongings. S tells us that the client has some kind of cognitive impairment and cannot plan very well. He has been smoking hashish for several years, which has affected his brain. She says further that she has been working on this plan for four months. What to do now? Should she go on with the plan? The frontline manager, who always gets the last word when it comes to decisions about residential treatment, says that it is for the best to continue with this plan while the client is still motivated.

In this scene, a decision to give the client residential care had already been made. But in the face of somewhat surprising events, the decision had to be reconsidered four months later. In caring for people with substance use problems, unanticipated things happen all the time. Relapses are part of the everyday work; clients do not show up at treatment sessions or they are suspended from housing facilities. Such incidents need not always affect the operative decision if the client is motivated to receive treatment, as we saw above. But after repeated incidents, it is often seen as necessary to alter the present decision, to try a different treatment, change housing or whatever intervention is in question. Such attention to unpredictabilities is an essential part of the logic of care (Mol 2008).

Yet another aspect of the ongoing decision-making can be seen when a decision has been made and a client has been remitted for treatment. In the realm of treatment, there are different rationales for interpreting clients’ needs and capacities to cope with treatment, which may cause a treatment professional to reconsider a decision made by an investigative social worker. In the following, at a meeting with the treatment unit of the agency where new cases are presented, the treatment professionals have problems accepting a decision that a recurrent client should once again be offered cognitive behavioral therapy (CBT) group treatment:

T (client) has received CBT earlier, but was then considered ‘difficult’. Now, he has been promised CBT again and the treatment professionals do not think it will work. But according to the investigative social worker, T has undergone some kind of change. D (treatment professional) does not know how they should respond, because if they at the treatment unit talk with T, he might get false hopes of starting the treatment once again. D needs to know what it is that has changed. They conclude that D first of all shall talk with the social worker and then make a judgment – something that D agrees to reluctantly after pressure from the treatment unit manager.

From the perspective of the social worker making the decision earlier on, this may seem like a perfectly natural decision; the client was willing to participate in the treatment program and outpatient treatment within the agency is relatively inexpensive. However, from the perspective of the treatment professionals who are supposed to carry out the treatment, things are a little bit different. For them it is crucial that the client ‘fits’ in with the group of other clients, since a great deal of group treatment rests on achieving a good team spirit within the group; otherwise it is pointless to include him or her. Therefore, it is necessary for the treatment professionals to make an independent judgment or decision whether clients are able to participate in their treatments.

By pointing to these aspects of decision-making, I am arguing that decision-making in practice cannot be described as a simple matter of making an individual choice, clear-cut in time. Rather, as the logics of care and organization suggest, decision-making more resembles open-ended care processes that are iterative and unpredictable. The clients’ unstable motivation and daily life are essential aspects, but the different organizational rationales within the agency for interpreting the clients’ needs are also important factors that contribute to the distributed nature of treatment decision-making (Rapley 2008). A treatment decision may look stable when studied in an investigation report, but this is always written in hindsight, when everything has been assembled. In real-time practice, however, decisions tend to have properties of emergent phenomena that evolve and are transformed over time. This may also be an explanation as to why critical appraisal, not only at this agency, has faced implementation difficulties (see Gabbay and le May 2004; Bellamy et al. 2008; Gray et al. 2012). Critical appraisal describes decision-making in hindsight, as a rationalization neatly packaged in five discrete steps followed by a single decision. The social worker, however, acts in real-time practice where many small decisions are made along the way and where unanticipated events happen and different organizational rationales affect the course of a treatment decision. Given this temporal distribution of decisions one might well ask, at what point shall a critical appraisal be performed?

Who makes treatment decisions?

We have already seen that treatment decisions are distributed over time, and we also touched upon the question of who actually makes treatment decisions. In line with the logic of choice, critical appraisal cherishes the autonomy of the social worker who shall weigh together the evidence, the client’s wishes and values, along with her own expertise when making decisions about treatment. But within a social services agency, there are different organizational roles that come into play in each treatment decision, for example, managers and the different kinds of social workers in the chain of care, who all have their own organizational rationales for making sense of difficult cases. The social workers advocate the clients’ interests and try to provide the best treatment in line with the specific demands of their organizational roles. The managers’ primary task, however, is to allocate the agency’s resources in the most efficient manner. As we saw previously, the managers are also the ones advocating that research evidence be used in the social workers’ treatment decisions. These differential organizational rationales sometimes collide, which is most apparent in cases where costly inpatient treatment is being considered.

In a so-called network meeting, a client and her professional contacts (a treatment assistant, a doctor and a social worker) sit down and plan for her future care: medication, controls, substance use treatment. The client, recently diagnosed with ADHD, has been using amphetamines for a long time, which has had a negative influence on her life, especially her physical health. Now, she is staying at a rurally located residential treatment center and has been drug-free for three whole months. The social worker asks her what kind of support she would need from the social services:

‘I’m thinking like this. I’ve heard from others and I’ve actually seen it with my own eyes that people can be successful here (at the center)’. She thinks it’s a matter of time, that she’ll need a longer stay at the center in order to make it. She needs a stable ‘platform’ so that she can be able to fix her driver’s license and get her own place to stay. ‘I could easily relapse if I were placed in a hostel or a shelter in the city. It would never work. I would like to stay here for a year, and then we’ll see’. ‘OK’ the social worker says, somewhat reserved, ‘because the current decision is until the last of October (in 20 days)’. A moment of silence arises. The client and the treatment assistant exchange a glance and sigh. They seem disappointed. The social worker tries to explain the situation. She says she has no mandate to decide about further stay at the center, but that she cannot see why the current decision should not be prolonged. She points out that she is not allowed to make a decision for as long as a year. ‘It’s three months at a time’, she says. Then adds, ‘At most three months’.

The social worker is caught between the client’s wishes and managerial demands at the agency. It is easy to understand the client’s wishes. After a hard life as an amphetamine user in the city, she has found respite at the center where she has been able to quit drugs and begun to turn her life around. At this point, she needs some time off from her old friends and her old habits in the city. Even though the social worker empathizes with the client’s wishes, she cannot promise anything since the decision about inpatient treatment must be made together with the unit manager, who also has to take financial considerations into account. Further, the client’s wish for a year at the center can simply not be approved since the housing guidelines within the agency says that these kinds of placements need to be kept short, never more than three months. Squeezed between the client and the agency, then, the social worker only has limited freedom within which to take into account the client’s wishes in her treatment decision. She can negotiate with the manager about the client’s stay at the center for only up to three months.

It is in these instances that the political and organizational constraints on the social workers’ autonomy (and the clients’ preferences) are most apparent. In other situations, when clients have less severe problems and truly wish for psychosocial treatment, the social workers have a much greater freedom to choose between the treatments that the agency has to offer: different psychosocial approaches, in a group or individually. In these cases, the social worker informs about the treatments available and how they may fit the client’s specific problems, and leaves the final treatment decision to the client. Since there are no economic interests at stake when choosing between these options, the clients are permitted some freedom of choice. And since many of the psychosocial approaches in the treatment program offered are recommended by the national guidelines, the decision is also likely to be ‘evidence-based’.

This shows that treatment decision-making in social work does not follow the logic of choice, but is highly dependent on political and organizational factors. The social worker does not and cannot act alone within a social services agency. This is far from a new insight (cf. Lipsky 1980; Evans and Harris 2004; Östberg 2010; Evans 2013), but it bears being pointed out again, given how normative decision-making models routinely disregard organizational aspects.

What is the role of research evidence? Interpreting evidence within an organizational logic

In the critical appraisal model, it is the individual social worker that is responsible for finding, appraising, and applying research evidence. This can be compared with the guideline model in which the practitioner can rely on clinical practice guidelines produced by experts. In an agency trying to work in line with a critical appraisal model, one would therefore expect that the social workers regularly search for evidence. But this is not the case. During my fieldwork at the agency, the NBHW clinical practice guidelines were the only source of research evidence used in treatment decisions.

During an introduction to EBP for a new co-worker, a senior social worker explained that they always try to use the guidelines as a source of evidence in their investigations. There was no mention of searching for primary studies or other sources of evidence that could be used to incorporate evidence into treatment decisions. The social workers do not even have access to databases where they can search and access research. In fact, not even the managers who initiated the agency’s work with EBP support this idea of extensive search for evidence: ‘I don’t think the social workers would want to, and I don’t even know if we would want the social workers to spend that much time’. Thus, extensive critical appraisal is seen as too time consuming. Instead, the agency relies heavily on the NBHW guidelines, resulting in a somewhat watered down version of critical appraisal or a hybrid between critical appraisal and the guideline model. The guidelines are used as much as possible, almost unquestioningly, to justify treatment decisions, even though they are not always easily applicable:

‘Where does it say that outpatient treatment is better than residential care?’ one of the social workers asks, quite aggravated, during the next item on the client conference agenda. ‘It doesn’t say so here’, she bursts out, waving the NBHW guidelines in front of the other participants in the room. She was recently told that in one of her investigations she had to include a justification that outpatient treatment is better than residential care. Someone in the group remarks that it should indeed say something about that in the guidelines, but that it is not very clear. The frontline manager says that it may also be their housing policy. ‘We could also refer to our own experiences’, she adds.

This scene, taking place at a client conference, articulates an underlying tension between the social workers and the managers of the agency concerning how the NBHW guidelines should be interpreted. The social worker is frustrated about having to put one of her clients in outpatient treatment when she would have preferred inpatient treatment. In addition, she was told by the agency manager to use the guidelines as a reference to support this decision in the written investigation report. But now, when she has looked through the guidelines, she cannot find anything to support this claim. In fact, the evidence regarding the effectiveness of inpatient and outpatient treatment is inconclusive and the guidelines do not address this question clearly.

The managers have been successful at proposing their perspective on the judgment between outpatient and inpatient treatment. Within their financial rationale, it is important to argue for cheaper outpatient treatment. As both external research evidence and the agency’s own follow-up data do not suggest any significant differences in outcome between the two, the managers have used this as argument for sparse use of inpatient treatment. This has to a large extent been accepted by the social workers at the agency, who find it difficult to argue against. But, as the social worker takes up for discussion, using the guidelines as evidence to legitimize a decision about outpatient treatment is questionable. Absence of clear evidence opens up for other organizational rationales in the judgment of suitable treatment for clients, and the managers’ dominant rationale circumscribes the space for professional judgment in line with the logic of care.

Inpatient treatment is sometimes motivated because it can offer a protective environment where the clients’ harmful drinking and drug use can be controlled. Such interventions are not always aimed at long-term changes but rather about improving the clients’ ‘here and now’ situation. They aim at caring, not curing. And within this logic, evidence of the (future) effectiveness of inpatient treatment is not relevant. But it is clearly the managers’ interpretation of evidence that prevails within the agency:

SW1 … In a normal case, of course you should try outpatient treatment. Then we’ll evaluate and see how it goes. Many times it is not problematic at all. But in some cases, when it’s becoming a matter of life and death –

SW2: – Then it’s problematic.

SW1: SW2 and I have a case now, that has been ongoing and where we’re really wondering what we’re doing. If this client dies, we’ll quit. I mean, for real …

Here, even though the social workers fear for the client’s life they feel forced to submit to the organizational demands. Moreover, even though the client has legal rights to receive social services, the Social Services Act (SFS 2001:453) is a framework law that does not give precedence to any kind of treatment, only stating that services ‘shall be of good quality’.

Taken together, the agency’s use of research evidence does not resemble the logic of choice in which facts of the matter are collected and thereafter acted upon. Rather, the research evidence is interpreted in line with the organizational logic, where financial considerations are (most) important. The agency’s reliance on the NBHW guidelines fills a strategic function here; it is an important symbol of evidence which lends it certain legitimacy, and it exempts social workers from the time-consuming (and therefore costly) activities of searching for and appraising primary studies. In line with previous studies of organizational decision-making, this shows that research evidence is not external to, but in fact is defined within the decision-making process (Sjögren 2006). As previous studies in healthcare also have shown, different groups of actors have different views as to what constitutes relevant evidence (see Fernler 2011, 2015; Sager 2011). Although the social workers in the agency are not content with the managers’ interpretation of evidence, they feel incapable of challenging it.

Conclusions

Looking at treatment decision-making as it occurred in real-time practices of a social services agency, I have shown how in several respects it markedly deviates from how decision-making is described in the critical appraisal model. I suggest that treatment decisions are better understood in line with organizational and care logics. Rather than seeing deviation from a perfect logic of choice as problematic, the organizational and care logics suggest that it is a common and inevitable aspect of working with clients with unstable motivation and life situations. But whereas Mol’s formulation of the logic of care centers on the relationship between practitioner and client, I suggest that an organizational logic must be added. The social worker is able to make treatment decisions because of the organizational context, but this implies a constraint that also must be considered.

Thus, making treatment decisions within a bureaucratic organization such as the social services requires attention being paid both to the client and to the organization. In this process, there are many heterogeneous things that must be coordinated or negotiated over an extended period of time in order to arrive at decisions about treatment: different aspects of the clients’ daily life, different organizational and professional rationales, research evidence, and availability of treatments. These things are not given in advance – as the logic of choice and critical appraisal suggests, but are defined in relation to each other as the decision process proceeds.

Comparing critical appraisal with ‘decisions-in-practice’ we can see that they differ in three crucial respects (see Table 1). The notion of ‘decisions-in-practice’ is in no way an exhaustive model of how treatment decisions are made in social work practice. However, it may be a first step towards understanding in a more empirical fashion how treatment decisions are actually arrived at in real-time practice.

Table 1. A comparison between treatment decisions-in-practice and treatment decisions according to critical appraisal.

	Critical appraisal	Decisions-in-practice
Decision process	Linear	Ongoing
Decision factor	Stable	Adjustable
Decision-maker	Individual	Organizational

Whereas critical appraisal sees the decision process as clearly defined in time, the logic of care suggests that they are more open-ended. This is mostly due to unpredictable events in the clients’ lives or that treatments did not turn out the way they were planned. Decisions-in-practice do not differ from critical appraisal concerning the amount or scope of factors that must be considered in a treatment decision. Rather, the difference lies in how the factors influencing the decision are viewed. Whereas critical appraisal tends to see them as separate and relatively stable over time, the organizational and care logics suggest that they can be interpreted differently and be adjusted to each other.

Lastly, whereas critical appraisal assumes an individual and autonomous decision-maker, the organizational and care logics highlight that treatment decisions are made within an organizational framework and to some extent by the organization. An organizational perspective is indeed introduced in Haynes’, Devereaux’ and Guyatt’s (2002) reformulation of critical appraisal that adds ‘clinical state and circumstances’. However, the broader implications of this factor have not been developed. My findings show that the social worker acts within and through an organization that shapes both how decisions are made and what kinds can be made; treatment decisions are dependent on different organizational rationales, and how treatment is organized shapes the freedom and constraints of choosing between different treatments. These additions imply a complexity that is not usually accounted for in the rationalist decision models – even when the notion of ‘circumstances’ is added, as in the case of Haynes, Devereaux, and Guyatt (2002).

Implications

Showing how decision-making in practice differs from the critical appraisal model, this paper suggests an alternative way of understanding the difficulties of implementing critical appraisal in social work practice. Whereas previous research has pointed to lacking skills and knowledge or negative attitudes towards EBP and critical appraisal (Manuel et al. 2009; Gray et al. 2012), I point to the very idea of critical appraisal as an important explanation for why it is not used to a larger extent. As a general inspiration for thinking about how decision-making can be improved, the critical appraisal model certainly fills some function. But as a practical stepwise guide to making decisions, it is uncertain whether it is a fruitful way forward. This conclusion is in line with a large body of research studies in other national, organizational and professional contexts (Lindblom 1959; March 1988; Gabbay and le May 2004; Sjögren 2006; Rapley 2008; Smith 2014a, 2014b). Highlighting the organizational and ongoing aspects of decision-making, the results are also in line with Plath’s (2012) study of critical appraisal in an Australian human service organization. But whereas Plath sees critical appraisal as a relevant model for human service organizations, this study suggests a different conclusion.

We should instead try to find other ways to improve and incorporate evidence in treatment decisions that are more attuned to the realities of decision-making practices. There are two concrete implications that follow from my general conclusions. First, whereas rational decision models may view the unpredictable open-ended decision processes described here as problematic, my findings show that this is often an inevitable aspect of making treatment decisions. Thus, rather than trying to organize ‘away’ such decision processes and making them rational in a universal rule-based sense, social services agencies should instead try to strengthen situationally rational ways of acting based on professional experience and judgment.¹ Second, my findings point to the shortcomings of an individualist conception of treatment decision-making and evidence use in social work. Even if a social worker should perform a state-of-the-art critical appraisal, it is far from sure that this decision can be backed up organizationally. We saw how the pressed economic situation of the agency forced some doubtful treatment decisions, but also that evidence-based decision alternatives can be created. Therefore, there is need to consider how social services agencies can make room for clients’ preferences, the social workers’ professional judgment, and a less biased interpretation of evidence (cf. Nutley, Walter, and Davies 2007).

Disclosure statement

No potential conflict of interest was reported by the author.

Funding

This work was funded by the Social Science Faculty at Stockholm University, and Riksbankens Jubileumsfond [grant number FSK-150896:1].

Notes

1. This is a central argument of Mol’s (2008) ‘logic of care’, and one that has been presented earlier by Waerness (1984) who argues that caring is a distinct form of rationality that needs to be strengthened in public administration.