The case against routine outcome monitoring

References

• Aafjes Van Doorn, K. & Meisel, J. (2022). Implementing routine outcome monitoring in a psychodynamic training clinic: It’s complicated. Counselling Psychology Quarterly, 36(3), 1–20. https://doi.org/10.1080/09515070.2022.2110451
   Web of Science ®Google Scholar
• Amble, I., Gude, T., Stubdal, S., Andersen, B. J., & Wampold, B. E. (2015). The effect of implementing the outcome questionnaire-45.2 feedback system in Norway: A multisite randomized clinical trial in a naturalistic setting. Psychotherapy Research, 25(6), 669–677. https://doi.org/10.1080/10503307.2014.928756
   PubMed Web of Science ®Google Scholar
• Armytage, Fels, Cockram, & McSherry. (2021). Royal Commission into Victoria’s Mental Health System. Final report: Summary and recommendations. Parl paper No. 202, session 2018–21 (document 1 of 6). pp. 3, 8. https://finalreport.rcvmhs.vic.gov.au/wp-content/uploads/2021/02/RCVMHS_FinalReport_ExecSummary_Accessible.pdf
   Google Scholar
• Australian Psychological Society. (2019). The future of psychology in Australia: A blueprint for better mental health outcomes for all Australians through Medicare – white paper. https://psychology.org.au/getmedia/a1c6fc1f-8356-471c-9247-36832da61299/aps-white-paper-the-future-of-psychology-in-australia-june-2019-final.pdf.
   Google Scholar
• Barkham, M., De Jong, K., Delgadillo, J., & Lutz, W. (2023). Routine outcome monitoring (ROM) and feedback: Research review and recommendations. Psychotherapy Research, 1–15. https://doi.org/10.1080/10503307.2023.2181114
   PubMed Web of Science ®Google Scholar
• Batty, M. J., Moldavsky, M., Foroushani, P. S., Pass, S., Marriott, M., Sayal, K., & Hollis, C. (2012). Implementing routine outcome measures in child and adolescent mental health services: From present to future practice. Child and Adolescent Mental Health, 18(2), 82–87. https://doi.org/10.1111/j.1475-3588.2012.00658.x
   PubMed Web of Science ®Google Scholar
• Beresford, P. & Branfield, F. (2006). Developing inclusive partnerships: User‐defined outcomes, networking and knowledge− a case study. Health & Social Care in the Community, 14(5), 436–444. https://doi.org/10.1111/j.1365-2524.2006.00654.x
   PubMed Web of Science ®Google Scholar
• Börjesson, S. & Boström, P. K. (2020). “I want to know what it is used for”: Clients’ perspectives on completing a routine outcome measure (ROM) while undergoing psychotherapy. Psychotherapy Research, 30(3), 337–347. https://doi.org/10.1080/10503307.2019.1630780
   PubMed Web of Science ®Google Scholar
• Boswell, J. F., Kraus, D. R., Miller, S. D., & Lambert, M. J. (2015). Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions. Psychotherapy Research, 25(1), 6–19. https://doi.org/10.1080/10503307.2013.817696
   PubMed Web of Science ®Google Scholar
• Burgess, P., Pirkis, J., & Coombs, T. (2015). Routine outcome measurement in Australia. International Review of Psychiatry, 27(4), 264–275. https://doi.org/10.3109/09540261.2014.977234
   PubMed Web of Science ®Google Scholar
• Callaly, T. & Hallebone, E. L. (2001). Introducing the routine use of outcomes measurement to mental health services. Australian Health Review, 24(1), 43–50. https://doi.org/10.1071/ah010043
   PubMedGoogle Scholar
• Carl, E., Witcraft, S. M., Kauffman, B. Y., Gillespie, E. M., Becker, E. S., Cuijpers, P., Van Ameringen, M., Smits, J. A. J., & Powers, M. B. (2020). Psychological and pharmacological treatments for generalized anxiety disorder (GAD): A meta-analysis of randomized controlled trials. Cognitive Behaviour Therapy, 49(1), 1–21. https://doi.org/10.1080/16506073.2018.1560358
   PubMed Web of Science ®Google Scholar
• Clark, D. M., Fairburn, C. G., & Wessely, S. (2008). Psychological treatment outcomes in routine NHS services: A commentary on Stiles et al. (2007). Psychological Medicine, 38(5), 629–634. https://doi.org/10.1017/s0033291707001869
   PubMed Web of Science ®Google Scholar
• Crawford, M. J., Robotham, D., Thana, L., Patterson, S., Weaver, T., Barber, R. Wykes T, & Rose, D. (2011). Selecting outcome measures in mental health: The views of service users. Journal of Mental Health, 20(4), 336–346. https://doi.org/10.3109/09638237.2011.577114
   PubMed Web of Science ®Google Scholar
• Dalal, F. (2018). CBT: The cognitive behavioural tsunami: Managerialism, politics and the corruptions of science. Routledge. https://doi.org/10.4324/9780429457814
   Google Scholar
• Devlin, N. J. & Appleby, J. (2010). Getting the most out of PROMS: Putting health outcomes at the heart of NHS decision-making. The King’s Fund. https://www.kingsfund.org.uk/sites/default/files/Getting-the-most-out-of-PROMs-Nancy-Devlin-John-Appleby-Kings-Fund-March-2010.pdf
   Google Scholar
• Dexter, K. R. K. (2017). Routine outcome monitoring and clinical supervision: Do therapists really care about their patients? University of Alaska Fairbanks.
   Google Scholar
• Duggan, M., Harris, B., Chislett, W. K., & Calder, R. (2020). Nowhere else to go: Why Australia’s health system results in people with mental illness getting ‘stuck’ in emergency departments. Mitchell Institute commissioned report. Victoria University. https://vuir.vu.edu.au/41956/1/nowhere-else-to-go-people-mental-illness-stuck-emergency-departments-report-mitchell-institute.pdf
   Google Scholar
• Duncan, E. A. & Murray, J. (2012). The barriers and facilitators to routine outcome measurement by allied health professionals in practice: A systematic review. BMC Health Services Research, 12(1), 1–9. https://doi.org/10.1186/1472-6963-12-96
   PubMedGoogle Scholar
• Duong, M. T., Lyon, A. R., Ludwig, K., Wasse, J. K., & McCauley, E. (2016). Student perceptions of the acceptability and utility of standardized and idiographic assessment in school mental health. International Journal of Mental Health Promotion, 18(1), 49–63. https://doi.org/10.1080/14623730.2015.1079429
   Web of Science ®Google Scholar
• Evans, C., & Carlyle, J.-A. 2021. Outcome measures and evaluation in counselling and psychotherapy. Sage.
   Google Scholar
• Fair, L. (2023, March 3). FTC says online counselling service BetterHelp pushed people into handing over health information – and broke its privacy promises. Federal Trade Commission. https://www.ftc.gov/business-guidance/blog/2023/03/ftc-says-online-counseling-service-betterhelp-pushed-people-handing-over-health-information-broke
   Google Scholar
• Fornells‐Ambrojo, M., Johns, L., Onwumere, J., Garety, P., Milosh, C., Iredale, C., Peters, E., Webster, A., & Jolley, S. (2017). Experiences of outcome monitoring in service users with psychosis: Findings from an improving access to psychological therapies for people with severe mental illness (IAPT - SMI) demonstration site. British Journal of Clinical Psychology, 56(3), 253–272. https://doi.org/10.1111/bjc.12136
   PubMed Web of Science ®Google Scholar
• Gilbody, S. M., House, A. O., & Sheldon, T. A. (2002). Outcomes research in mental health: Systematic review. The British Journal of Psychiatry, 181(1), 8–16. https://doi.org/10.1192/bjp.181.1.8
   PubMedGoogle Scholar
• Gimeno-Peon, A., Barrio-Nespereira, A., & Prado-Abril, J. (2018). Routine outcome monitoring and feedback in psychotherapy. Papeles del psicólogo, 39(3), 174–182. https://doi.org/10.23923/pap.psicol2018.2872
   Google Scholar
• Goldberg, S. B., Babins-Wagner, R., Rousmaniere, T., Berzins, S., Hoyt, W. T., Whipple, J. L., Miller, S. D., & Wampold, B. E. (2016). Creating a climate for therapist improvement: A case study of an agency focused on outcomes and deliberate practice. Psychotherapy Theory, Research, Practice, Training, 53(3), 367–375. https://doi.org/10.1037/pst0000060
   PubMed Web of Science ®Google Scholar
• Goldberg, S. B., Rousmaniere, T., Miller, S. D., Whipple, J., Nielsen, S. L., Hoyt, W. T., & Wampold, B. E. (2016). Do psychotherapists improve with time and experience? A longitudinal analysis of outcomes in a clinical setting. Journal of Counseling Psychology, 63(1), 1–11. https://doi.org/10.1037/cou0000131
   PubMed Web of Science ®Google Scholar
• Guthrie, D., McIntosh, M., Callaly, T., Trauer, T., & Coombs, T. (2008). Consumer attitudes towards the use of routine outcome measures in a public mental health service: A consumer‐driven study. International Journal of Mental Health Nursing, 17(2), 92–97. https://doi.org/10.1111/j.1447-0349.2008.00516.x
   PubMed Web of Science ®Google Scholar
• Happell, B. (2008). Meaningful information or a bureaucratic exercise? Exploring the value of routine outcome measurement in mental health. Issues in Mental Health Nursing, 29(10), 1098–1114. https://doi.org/10.1080/01612840802319852
   PubMedGoogle Scholar
• Hoenders, R. H., Bos, E. H., Bartels-Velthuis, A. A., Vollbehr, N. K., Van der Ploeg, K., De Jonge, P., & de Jong, J. T. (2014). Pitfalls in the assessment, analysis, and interpretation of routine outcome monitoring (ROM) data: Results from an outpatient clinic for integrative mental health. Administration and Policy in Mental Health and Mental Health Services Research, 41(5), 647–659. https://doi.org/10.1007/s10488-013-0511-7
   PubMed Web of Science ®Google Scholar
• Huhn, M., Tardy, M., Spineli, L. M., Kissling, W., Förstl, H., Pitschel-Walz, G., Leucht, C., Samara, M., Dold, M., Davis, J. M., & Leucht, S. (2014). Efficacy of pharmacotherapy and psychotherapy for adult psychiatric disorders: A systematic overview of meta-analyses. JAMA Psychiatry, 71(6), 706–715. https://doi.org/10.1001/jamapsychiatry.2014.112
   PubMed Web of Science ®Google Scholar
• Impact Economics and Policy. (2023). Reform critical: A fragmented health system at breaking point. Health Services Union. https://www.hsu.asn.au/assets/Uploads/Newsletter-attachments/HSU-Report-February-2022-v07.pdf
   Google Scholar
• Ionita, G. & Fitzpatrick, M. (2014). Bringing science to clinical practice: A Canadian survey of psychological practice and usage of progress monitoring measures. Canadian Psychology/Psychologie canadienne, 55(3), 187–196. https://doi.org/10.1037/a0037355
   Web of Science ®Google Scholar
• Iorfino, F., Carpenter, J. S., Cross, S. P., Crouse, J., Davenport, T. A., Hermens, D. F., Yee, H., Nichles, A., Zmicerevska, N., Guastella, A., Scott, E. M., & Hickie, I. B. (2022). Social and occupational outcomes for young people who attend early intervention mental health services: A longitudinal study. Medical Journal of Australia, 216(2), 87–93. https://doi.org/10.5694/mja2.51308
   PubMed Web of Science ®Google Scholar
• Kendrick, T., Dowrick, C., Lewis, G., Moore, M., Leydon, G. M., Geraghty, A. W. Griffiths G, Zhu S, Yao GL., May C, Gabbay M., & Stuart, B. (2023). Use of the patient health questionnaire as a patient-reported outcome measure for monitoring depression in primary care: The PROMDEP cluster randomised controlled trial. https://ssrn.com/abstract=4522402;http://dx.doi.org/10.2139/ssrn.4522402
   Google Scholar
• Kendrick, T., El‐Gohary, M., Stuart, B., Gilbody, S., Churchill, R., Aiken, L., Bhattacharya, A., Gimson, A., Brütt, A. L., de Jong, K., & Moore, M. (2016). Routine use of patient reported outcome measures (PROMs) for improving treatment of common mental health disorders in adults. Cochrane Database of Systematic Reviews, (7). https://doi.org/10.1002/14651858.cd011119
   Web of Science ®Google Scholar
• Kendrick, T. & Maund, E. (2020). Do PROMS improve outcomes in patients with depression in primary care? BMJ: British Medical Journal, 370. https://doi.org/10.1136/bmj.m3313
   Web of Science ®Google Scholar
• Kim, J. (2023). Data brokers and the sale of Americans’ mental health data. Cyber Policy Program. https://techpolicy.sanford.duke.edu/wp-content/uploads/sites/4/2023/02/Kim-2023-Data-Brokers-and-the-Sale-of-Americans-Mental-Health-Data.pdf
   Google Scholar
• Kisely, S. R., Bastiampillai, T., Allison, S., & Looi, J. C. (2023). More smoke and mirrors: Fifteen further reasons to doubt the effectiveness of headspace. Australasian Psychiatry, 31(3), 282–291. https://doi.org/10.1177/10398562231167683
   PubMed Web of Science ®Google Scholar
• Lambert, M. J. & Harmon, K. L. (2018). The merits of implementing routine outcome monitoring in clinical practice. Clinical Psychology Science & Practice, 25(4), e12268. https://doi.org/10.1111/cpsp.12268
   Web of Science ®Google Scholar
• Lambert, M. J. & Shimokawa, K. (2011). Collecting client feedback. In J. C. Norcross (Ed.), Psychotherapy relationships that work (2nd ed.). Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199737208.003.0010.
   Google Scholar
• Lambert, M. J., Whipple, J. L., & Kleinstäuber, M. (2018). Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring. Psychotherapy Theory, Research, Practice, Training, 55(4), 520–537. https://doi.org/10.1037/pst0000167
   PubMed Web of Science ®Google Scholar
• Leichsenring, F. & Rabung, S. (2011). Long-term psychodynamic psychotherapy in complex mental disorders: Update of a meta-analysis. The British Journal of Psychiatry, 199(1), 15–22. https://doi.org/10.1192/bjp.bp.110.082776
   PubMed Web of Science ®Google Scholar
• Lindgren, A., Werbart, A., & Philips, B. (2010). Long‐term outcome and post‐treatment effects of psychoanalytic psychotherapy with young adults. Psychology & Psychotherapy: Theory, Research & Practice, 83(1), 27–43. https://doi.org/10.1348/147608309x464422
   PubMed Web of Science ®Google Scholar
• Medicare Benefits Schedule Review Taskforce. (2020, December). An MBS for the 21st century. Recommendations, learning and ideas for the future. Final report to the Minister for Health. Australian Government Department of Health and Aged Care. https://www.health.gov.au/sites/default/files/documents/2020/12/medicare-benefits-schedule-review-taskforce-final-report-an-mbs-for-the-21st-century-recommendations-learnings-and-ideas-for-the-future.pdf
   Google Scholar
• Merry, S., Stasiak, K., Parkin, A., Seymour, F., Lambie, I., Crengle, S., & Pasene-Mizziebo E. (2001). Child and youth outcome measures: Examining current use and acceptability of measures in mental health services and recommending future directions. Health Research Council of New Zealand.
   Google Scholar
• Miller, S. D., Hubble, M. A., Chow, D., & Seidel, J. (2015). Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy Theory, Research, Practice, Training, 52(4), 449–457. https://doi.org/10.1037/pst0000031
   PubMed Web of Science ®Google Scholar
• Moran, P., Kelesidi, K., Guglani, S., Davidson, S., & Ford, T. (2012). What do parents and carers think about routine outcome measures and their use? A focus group study of CAMHS attenders. Clinical Child Psychology and Psychiatry, 17(1), 65–79. https://doi.org/10.1177/1359104510391859
   PubMed Web of Science ®Google Scholar
• National Collaborating Centre for Mental Health. (2023). The improving access to psychological therapies manual. National Health Service.
   Google Scholar
• Nehrig, N., Prout, T. A., & Aafjes‐van Doorn, K. (2019). Whose anxiety are we treating, anyway? Journal of Clinical Psychology, 75(10), 1866–1878. https://doi.org/10.1002/jclp.22824
   PubMed Web of Science ®Google Scholar
• Ogden, T. H. (2015). Intuiting the truth of what’s happening: On Bion’s “notes on memory and desire”. The Psychoanalytic Quarterly, 84(2), 285–306. https://doi.org/10.1002/psaq.12000
   PubMed Web of Science ®Google Scholar
• Perry, T., Barkham, M., & Evans, C. (2013). The CORE‐OM and CORE‐OM (SV) in secure settings: A template analysis of the experiences of male patients and their staff. The Journal of Forensic Practice, 15(1), 32–43. https://doi.org/10.1108/14636641311299068
   Google Scholar
• Pirkis, J., Currier, D., Harris, M., Mihalopoulos, C., Arya, V., Banfield, M., Bassilios B., Buchanan B., Butterworth P., Brophy L., & Burgess P. Williamson, M. (2022). Evaluation of better access. Main report. The University of Melbourne. https://www.health.gov.au/sites/default/files/2022-12/main-report-evaluation-of-the-better-access-initiative.pdf
   Google Scholar
• Productivity Commission. (2020). Mental health. Report no.95. Australian Government. www.pc.gov.au/inquiries/completed/mental-health/report.
   Google Scholar
• Shedler, J. (2010). The efficacy of psychodynamic psychotherapy. American Psychologist, 65(2), 98–109. https://doi.org/10.1037/a0018378
   PubMed Web of Science ®Google Scholar
• Shedler, J., Mayman, M., & Manis, M. (1993). The illusion of mental health. American Psychologist, 48(11), 1117–1131. https://doi.org/10.1037/0003-066X.48.11.1117
   PubMed Web of Science ®Google Scholar
• Solstad, S. M., Castonguay, L. G., & Moltu, C. (2019). Patients’ experiences with routine outcome monitoring and clinical feedback systems: A systematic review and synthesis of qualitative empirical literature. Psychotherapy Research, 29(2), 157–170. https://doi.org/10.1080/10503307.2017.1326645
   PubMed Web of Science ®Google Scholar
• Solstad, S. M., Kleiven, G. S., Castonguay, L. G., & Moltu, C. (2021). Clinical dilemmas of routine outcome monitoring and clinical feedback: A qualitative study of patient experiences. Psychotherapy Research, 31(2), 200–210. https://doi.org/10.1080/10503307.2020.1788741
   PubMed Web of Science ®Google Scholar
• Stasiak, K., Parkin, A., Seymour, F., Lambie, I., Crengle, S., Pasene-Mizziebo, E., & Merry, S. (2013). Measuring outcome in child and adolescent mental health services: Consumers’ views of measures. Clinical Child Psychology and Psychiatry, 18(4), 519–535. https://doi.org/10.1177/1359104512460860
   PubMed Web of Science ®Google Scholar
• Sullivan, G. M. & Feinn, R. (2012). Using effect size—or why the P value is not enough. Journal of Graduate Medical Education, 4(3), 279–282. https://doi.org/10.4300/jgme-d-12-00156.1
   PubMedGoogle Scholar
• Trauer, T., Gill, L., Pedwell, G., & Slattery, P. (2006). Routine outcome measurement in public mental health—what do clinicians think? Australian Health Review, 30(2), 144–147. https://doi.org/10.1071/ah060144
   PubMedGoogle Scholar
• Treichler, E. B. & Spaulding, W. D. (2018). Provider-associated measurement error in routine outcome monitoring in community mental health. Administration and Policy in Mental Health and Mental Health Services Research, 45(5), 780–789. https://doi.org/10.1007/s10488-018-0861-2
   PubMed Web of Science ®Google Scholar
• van der Wees, P. (2013). Patient-reported outcomes in the Netherlands: Routine use in clinical practice and performance measurement [slides]. http://www.kingsfund.org.uk/sites/files/kf/media/phil-van-der-wees-proms-netherlands-dec13.pdf
   Google Scholar
• Wampold, B. E. (2015). Routine outcome monitoring: Coming of age—with the usual developmental challenges. Psychotherapy Theory, Research, Practice, Training, 52(4), 458–462. https://doi.org/10.1037/pst0000037
   PubMed Web of Science ®Google Scholar
• Wolpert, M. (2014). Uses and abuses of patient reported outcome measures (PROMs): Potential iatrogenic impact of PROMs implementation and how it can be mitigated. Administration and Policy in Mental Health and Mental Health Services Research, 41(2), 141–145. https://doi.org/10.1007/s10488-013-0509-1
   PubMed Web of Science ®Google Scholar
• Wolpert, M., Curtis-Tyler, K., & Edbrooke-Childs, J. (2016). A qualitative exploration of patient and clinician views on patient reported outcome measures in child mental health and diabetes services. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 309–315. https://doi.org/10.1007/s10488-014-0586-9
   PubMed Web of Science ®Google Scholar
• Wolpert, M., Ford, T., Trustam, E., Law, D., Deighton, J., Flannery, H., & Fugard, R. J. (2012). Patient-reported outcomes in child and adolescent mental health services (CAMHS): Use of idiographic and standardized measures. Journal of Mental Health, 21(2), 165–173. https://doi.org/10.3109/09638237.2012.664304
   PubMed Web of Science ®Google Scholar
• Wolpert, M., Fugard, A. J., Deighton, J., & Görzig, A. (2012). Routine outcomes monitoring as part of children and young people’s improving access to psychological therapies (CYP IAPT)–improving care or unhelpful burden?. Child and Adolescent Mental Health, 17(3), 129–130. https://doi.org/10.1111/j.1475-3588.2012.00676.x
   PubMed Web of Science ®Google Scholar
• Wood, H. (2010). Integrating research with NHS clinical practice: Unwelcome intrusion or constructive triangulation? Psychoanalytic Psychotherapy, 24(3), 252–261. https://doi.org/10.1080/02668734.2010.501571
   Google Scholar