Service user views and service user research in the Journal of Mental Health

Introduction

The Journal of Mental Health has an unsurpassed tradition of publishing papers which articulate the perspectives of mental health service users (Agar-Jacombi & Read, 2009; Bengtsson-Tops & Svensson, 2010; Rogers & Dunne, 2011; Ulivi et al., 2009) and papers written by mental health service user researchers themselves (Armes, 2009). I belong to this latter category and the purpose of this editorial is to introduce myself to the Journal's readership. I see my role as editor as continuing the tradition of giving voice to service users and service user researchers and promoting high-quality work written by them. In this editorial, I would like to give the readership a flavour of the work I have done in the past 10 years, both empirically and conceptually. This work is situated in the Service User Research Enterprise (SURE) at the Institute of Psychiatry, King's College London where I was first project co-ordinator, then senior lecturer and now reader in user-led research. The unit is collaborative with a balanced management structure – myself as a service user researcher and Professor Til Wykes as clinical academic and conventional university researcher. Sometimes this makes for tensions particularly when different forms of expertise conflict (Trivedi & Wykes, 2002).

User-centred systematic reviews

The first piece of work undertaken by SURE was a systematic review of service users' perceptions of electroconvulsive therapy therapy (ECT) (Rose et al., 2003, 2004, 2005). This was funded by the Department of Health and ran concurrently with a systematic review of the safety and efficacy of ECT (UK ECT Review Group, 2003). The two researchers who conducted the empirical work had experienced ECT themselves and this ‘insider knowledge’ is critical to all SURE's work. Although the project followed some of the standard features of systematic reviews, these were adapted and added to. In the quantitative meta-analysis, we included the ‘grey’ literature written by service user organisations or in collaboration with them. We also included ‘testimonies’, i.e., first hand accounts of the experience of receiving ECT. These two adaptations were designed to make our review more user-focused.

The results were, perhaps, not surprising. The recipient literature painted a very different picture of the experience of receiving ECT than the clinical literature. In fact, when we constructed a Forest plot of benefits/satisfaction with ECT, there was no overlap between the two literatures thus making a non sense of the process of averaging to find a ‘true’ estimate. The user literature was criticised by our Advisory Group as biased. This is a common criticism of user research and must be taken seriously. However, the two researchers who had been recipients themselves were able to use their experience to point to potential bias in the clinical work.

Service users who have received ECT often complain of long-term memory loss as the worst ‘side-effect’ and indeed we found this in the testimonies and the user literature. Moreover, it was evident in the clinical literature but here there was something strange. Although the ‘Results’ section of several papers showed the experience of memory loss amongst a significant proportion of recipients, the ‘Discussion’ concluded that this did not occur (e.g. Freeman et al., 1980). This discrepancy between the Results and the Discussion did not appear to us scientific – the very accusation often levelled at user-led research.

This work, however, was the first piece of user-led research to have an impact on national policy in the UK. NICE was developing new guidelines on ECT as we were doing our work and this was incorporated. There are now much more stringent rules for information and consent and potential recipients of ECT must be told of the risk of memory loss.

Although to date, the method for service-user centred systematic reviews has so far only been used in SURE on this one occasion, it is a general method and could be applied to other domains. Clearly, these would need to be ones where a substantial literature on service user reports and views already exists.

Trials and outcome measures

It has been pointed out for some time that the outcomes measures used in trials and other large-scale studies are considered important from the perspectives of clinicians but may not be from the point of view of service users (Faulkner & Thomas, 2002). For example, conventional researchers prize measures of symptom reduction but service users may be more concerned with side-effects or other aspects of their lives affected by medication. In a nominal group study, Crawford et al. (2011) found that service users prized outcome measures that attended to side effects and to wellbeing. They were less enthusiastic about measures of symptoms and found measures of social functioning to be irrelevant and normative.

SURE has developed a method of generating outcome measures entirely from the service user's perspective. With the current vogue for PROMs these are not just measures that are completed by service users but actually developed by them. Many PROMs are developed in an entirely conventional manner and do not take the perspective of service users into account when they are generated. Our method has been summarised by Rose et al. (2011) and examples can be found for user satisfaction with cognitive remediation therapy (Rose et al., 2008), continuity of care (Rose et al., 2009) and user responses to CBT for psychosis (Greenwood et al., 2010).

As is the case with patient-centred systematic reviews, critical to this method is the involvement of researchers who have themselves experienced the treatment or service to be assessed. The procedure starts with an intensive set of focus groups, the results of which are analysed qualitatively. The researchers then draft a measure and take this to expert panels for amendment and review. One expert panel is comprised of focus group members to ensure their views have been accurately captured. The second expert panel is independent thus enabling a fresh look at the measure.

We do not eschew psychometric testing. In the first place, the measures must be shown to be robust. However, psychometric testing such as concurrent or criterion validity also enables the user-generated measures to be compared to existing ones. We emphatically do not expect them to be exactly the same, in content or in structure, precisely because of how they were generated. Psychometric testing is thus used in a novel way to determine similarities and differences with measures generated in a conventional manner.

The method described here has now been extended both to professional groups such as nurses and to other medical conditions such as rheumatoid arthritis.

Is this science?

Service user research is often criticised, explicitly (Tyrer, 2002) or implicitly, for being biased, anecdotal and carried out by people who are over-involved. Indeed, when I was asked to write a theme piece (Rose, 2003) these were the issues the editor asked me to address.

Bias

User researchers do not shy away from saying that their work comes from a specific standpoint. But our argument is that all academic work comes from a specific standpoint. The difficulty with conventional research is that the topics chosen and methods used appear ‘obvious’ to researchers. For example, it appears ‘obvious’ that randomised controlled trials (RCTs) should be the gold standard and that these deliver results that are transparent. In the first place, the discussion above regarding outcome measures shows that RCTs are not unbiased when it comes to the choice of outcome measures. These are the choice of researchers who occupy a particular place in the institutions of academia and the clinic. Second, the tables of figures and ‘p’ values that are the outputs of RCTs are not transparent and obvious because otherwise there would be no need for a ‘Discussion’ section in papers where, typically, different interpretations of the same dataset are set against each other (Rose et al., 2006).

I have said that all research comes from a specific standpoint. What is the standpoint of mental health service user researchers? Much is made these days of the idea of ‘experts by experience’ and certainly I would argue that having experience of treatments and services confers a knowledge which conventional researchers do not possess. Beresford (2003) puts forward a more sophisticated version of this in his ‘theory of distance’. However, reliance on the concept of experience seems inadequate to me if it is claimed that all service users construct their experience in the same way which is patently not the case. Several years ago, Scott (1992) argued that there was no such thing as ‘raw’ or ‘authentic’ experience and that all experience is constructed. I think it no accident that many user researchers, especially in the early days, had their roots in the service user movement and were ‘primed’ to take a certain position in relation to the production of knowledge. This may not be so true today but it is no accident that those who embrace the service user researcher identity have other and past identities which are consistent with wishing to see change.

We can also learn from feminist standpoint epistemology where Harding (1991) has developed the concept of ‘strong objectivity’. This means that feminist scientists, excluded from knowledge for so long, have access to two discourses – that of conventional science and that of the science they have forged for themselves. Service user researchers in mental health, I would argue, find themselves in the same position and this ‘double identity’ gives access to a broader range of knowledge than that confined to mainstream science. Further, the difficulty we have is similar to the initial arguments made by standpoint epistemologists – that women are seen as on the wrong side of certain Enlightenment divides: nature/culture; reason/unreason; emotion/intellect. Seen to be embodying the first of these oppositions stands also as a reason to exclude service user researchers from science. The concept of ‘strong objectivity’ can be applied to service user research to rescue us from this Enlightenment fallacy without falling into the reductionism of the concept of ‘experts by experience’.

Are we over-involved?

This criticism is easier to deal with. Of course, service user researchers are committed to articulating the views and experiences of the participants with whom they work. But I have yet to come across a conventional researcher who likewise is not committed to the therapy or service which they investigate. But we must be careful. In either case, if the ‘hypothesis’ is not supported or the views of participants depart from what was expected, this must not be hidden or the data skewed to make it seem that what was expected transpired even though it did not. This applies to any research and this goes without saying. However, because the prevailing view in some quarters is that service user research is not ‘scientific’, we must be doubly cautious. I have argued that conventional researchers see certain methods and means of analysis as ‘obvious’. This we must avoid and continuously reflect on the research we do to ensure that our own preoccupations are not imposed on the participants we engage. This returns me to the concept of standpoint. Because we have access to more than one scientific discourse, this means that we can reflect on our own work and not make the mistake of thinking that what we do is ‘obvious’.

Anecdote

It may be argued that the charge of anecdote reduces to a critique of qualitative research. Many mainstream researchers are suspicious of qualitative research because it generally involves small numbers and therefore they believe that results are not generalisable. In addition, it may be believed that qualitative work is ‘easy’ compared to handling large numerical datasets. For these, specialist statistical expertise is often brought in, lending the work some esoteric value.

SURE does not confine itself to qualitative work but there is a preponderance of this in the work that we do. On the issue of small samples, the argument is that 40 or more people discussing an issue for four hours (2 × 2 focus groups, for example) gives a completely different picture to 400 people spending 10 min filling in a questionnaire. The latter gives little scope for participants' own views to develop as discussion in an interview or focus group proceeds or for their own in-depth experiences to be revealed. Nevertheless, I would argue that in qualitative research samples can be too small such that one view dominates. This is the difficulty I have with, for example interpretive phenomenological analysis (IPA). Qualitative research, for the most part, should purposively sample according to the domain under consideration to ensure a broad range of views.

Qualitative analysis is not easy! It is a time-consuming affair, driven by the data and requiring constant returns to the data and revisions of the means of analysis. It is an interpretive activity which engages the analyst in the identifying the meanings in the text. The activity is interpretive because themes do not, as is often said, ‘emerge from the data’ but are constructed through the means of analysis. This is also why the analyst must constantly reflect on what they are doing to check that the model they construct captures the meanings in the text without pretending that it simply re-describes them.

Recovery

No contemporary piece of writing on the service user experience in mental health can avoid the concept of recovery and indeed it has entered the pages of the Journal of Mental Health (Salgado et al., 2010). Recovery was originally a concept fashioned by service users such as Pat Deegan in the USA and others in Australasia. It may be argued that the key text in the UK is now Repper and Perkins (2004) and, for professionals, Slade (2009). But ‘recovery’ is not just in texts or even just in the internet, it is everywhere in mental health services today. What are we to make of this? The centrality of hope to the concept of recovery makes it very attractive as for many decades mental health services have been imbued with a sense of pessimism about their clients and this can become internalised. However, I believe we must be cautious and here I speak from experience. In some ‘recovery services’ the service user, it seems, cannot be vulnerable or can only be vulnerable for less than two weeks (the maximum time the service is available). Expectations can be unrealistically high. This dovetails into the new government emphasis on ‘clusters’ where interventions are to be time limited depending on diagnosis. It may be argued that this will force a divide between those who are ‘ready’ to recover and those who are not with the latter receiving much inferior services. And this does not even begin to touch on the thorny issue of the alignment of the recovery movement and welfare reforms in the UK.

Conclusion

The Journal of Mental Health will continue to publish pieces written by service users and I hope these will increase. Quality cannot be compromised but it may be differently defined as I have outlined above. Although service user research is accepted now much more than 15 years ago, it continues to be controversial. The Journal of Mental Health prizes debate and the inclusion of service user research should contribute to this.

Acknowledgements

The author is supported by the Specialist Biomedical Research Centre in Mental Health at the South London and Maudsley Trust and Institute of Psychiatry, King's College London.