Development and validation of an 8-item version of the Real Relationship Inventory–Client form

References

• Alessi, E. J., Dillon, F. R., & Van Der Horn, R. (2019). The therapeutic relationship mediates the association between affirmative practice and psychological well-being among lesbian, gay, bisexual, and queer clients. Psychotherapy, 56(2), 229–240. https://doi.org/10.1037/pst0000210
   PubMed Web of Science ®Google Scholar
• Baker, F. B., & Kim, S.-H. (2017). The basics of item response theory using R. Springer International Publishing. https://doi.org/10.1007/978-3-319-54205-8
   Google Scholar
• Barkham, M., De Jong, K., Delgadillo, J., & Lutz, W. (2023). Routine outcome monitoring (ROM) and feedback: Research review and recommendations. Psychotherapy Research, 33(7), 841–855. https://doi.org/10.1080/10503307.2023.2181114
   PubMed Web of Science ®Google Scholar
• Bhatia, A., & Gelso, C. J. (2018). Therapists’ perspective on the therapeutic relationship: Examining a tripartite model. Counselling Psychology Quarterly, 31(3), 271–293. https://doi.org/10.1080/09515070.2017.1302409
   Web of Science ®Google Scholar
• Bland, J. M., & Altman, D. G. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307–310. https://doi.org/10.1016/S0140-6736(86)90837-8
   Web of Science ®Google Scholar
• Cena, L., Rota, M., Calza, S., Massardi, B., Trainini, A., & Stefana, A. (2021). Estimating the impact of the COVID-19 pandemic on maternal and perinatal health care services in Italy: Results of a self-administered survey. Frontiers in Public Health, 9. https://www.frontiersin.org/articles/10.3389fpubh.2021.701638
   PubMed Web of Science ®Google Scholar
• Chalmers, R. P. (2012). Mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6). https://doi.org/10.18637/jss.v048.i06
   Web of Science ®Google Scholar
• Chandler, J., & Shapiro, D. (2016). Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology, 12(1), 53–81. https://doi.org/10.1146/annurev-clinpsy-021815-093623
   PubMed Web of Science ®Google Scholar
• Christensen, A. P., Garrido, L. E., Guerra-Peña, K., & Golino, H. (2023). Comparing community detection algorithms in psychometric networks: A Monte Carlo simulation. Behavior Research Methods, https://doi.org/10.3758/s13428-023-02106-4
   PubMed Web of Science ®Google Scholar
• Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155
   PubMed Web of Science ®Google Scholar
• Comrey, A. L., & Lee, H. B. (2013). A first course in factor analysis. Psychology Press. https://doi.org/10.4324/9781315827506
   Google Scholar
• de Ayala, R. J. (2022). The theory and practice of item response theory (2nd ed.). Guilford Publications.
   Google Scholar
• Debelak, R., & Tran, U. S. (2016). Comparing the effects of different smoothing algorithms on the assessment of dimensionality of ordered categorical items with parallel analysis. PLoS One, 11(2), e0148143. https://doi.org/10.1371/journal.pone.0148143
   PubMed Web of Science ®Google Scholar
• DeVellis, R. F., & Thorpe, C. T. (2022). Scale development: Theory and applications (5th ed.). Sage.
   Google Scholar
• Eubanks, C. F., Muran, J. C., & Safran, J. D. (2018). Alliance rupture repair: A meta-analysis. Psychotherapy, 55(4), 508–519. https://doi.org/10.1037/pst0000185
   PubMed Web of Science ®Google Scholar
• Eugster, S. L., & Wampold, B. E. (1996). Systematic effects of participant role on evaluation of the psychotherapy session. Journal of Consulting and Clinical Psychology, 64(5), 1020–1028. https://doi.org/10.1037/0022-006X.64.5.1020
   PubMed Web of Science ®Google Scholar
• Faro, J. M., Nagawa, C. S., Orvek, E. A., Smith, B. M., Blok, A. C., Houston, T. K., Kamberi, A., Allison, J. J., Person, S. D., & Sadasivam, R. S. (2021). Comparing recruitment strategies for a digital smoking cessation intervention: Technology-assisted peer recruitment, social media, ResearchMatch, and smokefree.gov. Contemporary Clinical Trials, 103, 106314. https://doi.org/10.1016/j.cct.2021.106314
   PubMed Web of Science ®Google Scholar
• Feuerstahler, L. M., Waller, N., & MacDonald, A. (2020). Improving measurement precision in experimental psychopathology using item response theory. Educational and Psychological Measurement, 80(4), 695–725. https://doi.org/10.1177/0013164419892049
   PubMed Web of Science ®Google Scholar
• Flückiger, C., Del Re, A. C., Wampold, B. E., & Horvath, A. O. (2018). The alliance in adult psychotherapy: A meta-analytic synthesis. Psychotherapy, 55(4), 316–340. https://doi.org/10.1037/pst0000172
   PubMed Web of Science ®Google Scholar
• Gelso, C. J. (2009). The real relationship in a postmodern world: Theoretical and empirical explorations. Psychotherapy Research, 19(3), 253–264. https://doi.org/10.1080/10503300802389242
   PubMed Web of Science ®Google Scholar
• Gelso, C. J. (2011). The real relationship in psychotherapy. American Psychological Association.
   Google Scholar
• Gelso, C. J. (2014). A tripartite model of the therapeutic relationship: Theory, research, and practice. Psychotherapy Research, 24(2), 117–131. https://doi.org/10.1080/10503307.2013.845920
   PubMed Web of Science ®Google Scholar
• Gelso, C. J., & Hayes, J. A. (1998). The psychotherapy relationship: Theory, research, and practice. John Wiley & Sons
   Google Scholar
• Gelso, C. J., Kelley, F. A., Fuertes, J. N., Marmarosh, C., Holmes, S. E., Costa, C., & Hancock, G. R. (2005). Measuring the real relationship in psychotherapy: Initial validation of the therapist form. Journal of Counseling Psychology, 52(4), 640–649. https://doi.org/10.1037/0022-0167.52.4.640
   Web of Science ®Google Scholar
• Gelso, C. J., Kivlighan, D. M., Busa-Knepp, J., Spiegel, E. B., Ain, S., Hummel, A. M., Ma, Y. E., & Markin, R. D. (2012). The unfolding of the real relationship and the outcome of brief psychotherapy. Journal of Counseling Psychology, 59(4), 495–506. https://doi.org/10.1037/a0029838
   PubMed Web of Science ®Google Scholar
• Gelso, C. J., Kivlighan, D. M., & Markin, R. D. (2018). The real relationship and its role in psychotherapy outcome: A meta-analysis. Psychotherapy, 55(4), 434–444. https://doi.org/10.1037/pst0000183
   PubMed Web of Science ®Google Scholar
• Gelso, C. J., Kivlighan, D. M., & Markin, R. D. (2019). The real relationship. In C. J. Gelso, D. M. Kivlighan, & R. D. Markin (Eds.), Psychotherapy relationships that work (pp. 351–378). Oxford University Press. https://doi.org/10.1093/med-psych/9780190843953.003.0010
   Google Scholar
• Gelso, C. J., & Kline, K. V. (2019). The sister concepts of the working alliance and the real relationship: On their development, rupture, and repair. Research in Psychotherapy: Psychopathology, Process and Outcome, 22(2). https://doi.org/10.4081/ripppo.2019.373
   Google Scholar
• Gelso, C. J., & Silberberg, A. (2016). Strengthening the real relationship: What is a psychotherapist to do? Practice Innovations, 1(3), 154–163. https://doi.org/10.1037/pri0000024
   Google Scholar
• Hair, J. F., Babin, B. J., Anderson, R. E., & Black, W. C. (2022). Multivariate data analysis. Cengage Learning.
   Google Scholar
• Harris, P. A., Scott, K. W., Lebo, L., Hassan, N., Lightner, C., & Pulley, J. (2012). Researchmatch: A national registry to recruit volunteers for clinical research. Academic Medicine, 87(1), 66–73. https://doi.org/10.1097/ACM.0b013e31823ab7d2
   PubMed Web of Science ®Google Scholar
• Hatcher, R. L., & Gillaspy, J. A. (2006). Development and validation of a revised short version of the working alliance inventory. Psychotherapy Research, 16(1), 12–25. https://doi.org/10.1080/10503300500352500
   Web of Science ®Google Scholar
• He, W. (2023). Positive and negative affect facilitate creativity motivation: Findings on the effects of habitual mood and experimentally induced emotion. Frontiers in Psychology, 14, 1014612. https://doi.org/10.3389/fpsyg.2023.1014612
   PubMed Web of Science ®Google Scholar
• Hill, C. E., Gelso, C. J., Chui, H., Spangler, P. T., Hummel, A., Huang, T., Jackson, J., Jones, R. A., Palma, B., Bhatia, A., Gupta, S., Ain, S. C., Klingaman, B., Lim, R. H., Liu, J., Hui, K., Jezzi, M. M., & Miles, J. R. (2014). To be or not to be immediate with clients: The use and perceived effects of immediacy in psychodynamic/interpersonal psychotherapy. Psychotherapy Research, 24(3), 299–315. https://doi.org/10.1080/10503307.2013.812262
   PubMed Web of Science ®Google Scholar
• Hill, C. E., & Kellems, I. S. (2002). Development and use of the helping skills measure to assess client perceptions of the effects of training and of helping skills in sessions. Journal of Counseling Psychology, 49(2), 264–272. https://doi.org/10.1037/0022-0167.49.2.264
   Web of Science ®Google Scholar
• Hoyle, R. H. (Eds.). (2023). Handbook of structural equation modeling (2nd ed.). The Guilford Press.
   Google Scholar
• Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
   Web of Science ®Google Scholar
• Irwing, P., Booth, T., & Hughes, D. J. (Eds.). (2018). The wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development (1st ed.). Wiley. https://doi.org/10.1002/9781118489772
   Google Scholar
• Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19. https://doi.org/10.1037/0022-006X.59.1.12
   PubMed Web of Science ®Google Scholar
• Kaiser, J., Hanschmidt, F., & Kersting, A. (2021). The association between therapeutic alliance and outcome in internet-based psychological interventions: A meta-analysis. Computers in Human Behavior, 114, 106512. https://doi.org/10.1016/j.chb.2020.106512
   Web of Science ®Google Scholar
• Kelley, F. A., Gelso, C. J., Fuertes, J. N., Marmarosh, C., & Lanier, S. H. (2010). The Real Relationship Inventory: Development and psychometric investigation of the client form. Psychotherapy: Theory, Research, Practice, Training, 47(4), 540–553. https://doi.org/10.1037/a0022082
   PubMed Web of Science ®Google Scholar
• Kertz, S., Bigda-Peyton, J., & Bjorgvinsson, T. (2012). Validity of the generalized anxiety disorder-7 scale in an acute psychiatric sample: Validity of the generalized anxiety disorder-7. Clinical Psychology & Psychotherapy, 456. https://doi.org/10.1002/cpp.1802
   Google Scholar
• Kline, K. V., Hill, C. E., Lu, Y., & Gelso, C. J. (2023). Transference and client attachment to therapist in psychodynamic psychotherapy. Psychotherapy, https://doi.org/10.1037/pst0000492
   PubMedGoogle Scholar
• Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x
   PubMed Web of Science ®Google Scholar
• Lent, R. W., Hoffman, M. A., Hill, C. E., Treistman, D., Mount, M., & Singley, D. (2006). Client-specific counselor self-efficacy in novice counselors: Relation to perceptions of session quality. Journal of Counseling Psychology, 53(4), 453–463. https://doi.org/10.1037/0022-0167.53.4.453
   Web of Science ®Google Scholar
• Löwe, B., Gräfe, K., Zipfel, S., Witte, S., Loerch, B., & Herzog, W. (2004). Diagnosing ICD-10 depressive episodes: Superior criterion validity of the patient health questionnaire. Psychotherapy and Psychosomatics, 73(6), 386–390. https://doi.org/10.1159/000080393
   PubMed Web of Science ®Google Scholar
• Martinez-Aran, A., & Vieta, E. (2022). Precision psychotherapy. European Neuropsychopharmacology, 55, 20–21. https://doi.org/10.1016/j.euroneuro.2021.10.771
   PubMed Web of Science ®Google Scholar
• Munder, T., Wilmers, F., Leonhart, R., Linster, H. W., & Barth, J. (2009). Working Alliance Inventory-Short Revised (WAI-SR): Psychometric properties in outpatients and inpatients. Clinical Psychology & Psychotherapy, 17(3), 231–239. https://doi.org/10.1002/cpp.658
   Web of Science ®Google Scholar
• Muran, J. C., Safran, J. D., Gorman, B. S., Samstag, L. W., Eubanks-Carter, C., & Winston, A. (2009). The relationship of early alliance ruptures and their resolution to process and outcome in three time-limited psychotherapies for personality disorders. Psychotherapy: Theory, Research, Practice, Training, 46(2), 233–248. https://doi.org/10.1037/a0016085
   PubMed Web of Science ®Google Scholar
• Nickodem, K., & Halpin, P. (2022). Kfa: K-fold cross validation for factor analysis. Https://CRAN.R-project.org/package = kfa.
   Google Scholar
• Norcross, J. C., & Lambert, M. J. (2019). Psychotherapy relationships that work (3rd ed.). Volume 1: Evidence-based therapist contributions. Oxford University Press.
   Google Scholar
• Pierce, B. S., Perrin, P. B., & McDonald, S. D. (2020). Demographic, organizational, and clinical practice predictors of U.S. psychologists’ use of telepsychology. Professional Psychology: Research and Practice, 51(2), 184–193. https://doi.org/10.1037/pro0000267
   Web of Science ®Google Scholar
• Pierce, B. S., Perrin, P. B., Tyler, C. M., McKee, G. B., & Watson, J. D. (2021). The COVID-19 telepsychology revolution: A national study of pandemic-based changes in U.S. mental health care delivery. American Psychologist, 76(1), 14–25. https://doi.org/10.1037/amp0000722
   PubMed Web of Science ®Google Scholar
• Probst, G. H., Berger, T., & Flückiger, C. (2019). The alliance-outcome relation in internet-based interventions for psychological disorders: A correlational meta-analysis. Verhaltenstherapie, 32(Suppl. 1), 135–146. https://doi.org/10.1159/000503432
   Google Scholar
• Qureshi, A., & Collazos, F. (2011). The intercultural and interracial therapeutic relationship: Challenges and recommendations. International Review of Psychiatry, 23(1), 10–19. https://doi.org/10.3109/09540261.2010.544643
   PubMed Web of Science ®Google Scholar
• Reinard, J. C. (2006). Communication research statistics. SAGE.
   Google Scholar
• Revelle, W., & Condon, D. M. (2019). Reliability from α to ω: A tutorial. Psychological Assessment, 31(12), 1395–1411. https://doi.org/10.1037/pas0000754
   PubMed Web of Science ®Google Scholar
• Rosseel, Y. (2012). Lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48(2). https://doi.org/10.18637/jss.v048.i02
   Web of Science ®Google Scholar
• Samstag, L. W., Batchelder, S. T., Muran, J. C., Safran, J. D., & Winston, A. (1998). Early identification of treatment failures in short-term psychotherapy. An assessment of therapeutic alliance and interpersonal behavior. The Journal of Psychotherapy Practice and Research, 7(2), 126–143.
   PubMedGoogle Scholar
• Smith, G. T., McCarthy, D. M., & Anderson, K. G. (2000). On the sins of short-form development. Psychological Assessment, 12(1), 102–111. https://doi.org/10.1037/1040-3590.12.1.102
   PubMed Web of Science ®Google Scholar
• Soto, C. J., & John, O. P. (2017a). Short and extra-short forms of the Big Five Inventory–2: The BFI-2-S and BFI-2-XS. Journal of Research in Personality, 68, 69–81. https://doi.org/10.1016/j.jrp.2017.02.004
   Web of Science ®Google Scholar
• Soto, C. J., & John, O. P. (2017b). The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Personality and Social Psychology, 113(1), 117–143. https://doi.org/10.1037/pspp0000096
   PubMed Web of Science ®Google Scholar
• Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006). A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097. https://doi.org/10.1001/archinte.166.10.1092
   PubMed Web of Science ®Google Scholar
• Stefana, A., Fusar-Poli, P., Vieta, E., & Youngstrom, E. A. (2023). Therapeutic relationship elements and therapy session outcomes: Protocol for a longitudinal study of the patient’s perspective. Open Research Europe, 3, 133. https://doi.org/10.12688/openreseurope.16466.1
   Google Scholar
• Stefana, A., Fusar-Poli, P., Vieta, E., & Youngstrom, E. (2024). Assessing the patients' affective perception of their psychotherapist: Validation of the in-Session Patient Affective Reactions Questionnaire (SPARQ). Frontiers in Psychiatry.
   Google Scholar
• Stefana, A., Langfus, J. A., Vieta, E., Fusar-Poli, P., & Youngstrom, E. A. (2023). Development and initial validation of the in-session patient affective reactions questionnaire (SPARQ) and the rift In-session questionnaire (RISQ). Journal of Clinical Medicine, 12(15), 5156. https://doi.org/10.3390/jcm12155156
   PubMed Web of Science ®Google Scholar
• Stefana, A., Vieta, E., Fusar-Poli, P., & Youngstrom, E. (2024). Enhancing psychotherapy outcomes by encouraging patients to regularly self-monitor, reflect on, and share their affective responses toward their therapist: Protocol for a randomized controlled trial. JMIR Research Protocols, https://doi.org/10.2196/55369
   Google Scholar
• Stefana, A., Youngstrom, E. A., Hopwood, C. J., & Dakanalis, A. (2020). The COVID-19 pandemic brings a second wave of social isolation and disrupted services. European Archives of Psychiatry and Clinical Neuroscience, 270(6), 785–786. https://doi.org/10.1007/s00406-020-01137-8
   PubMed Web of Science ®Google Scholar
• Steiner, M., & Grieder, S. (2020). EFAtools: An R package with fast and flexible implementations of exploratory factor analysis tools. Journal of Open Source Software, 5(53), 2521. https://doi.org/10.21105/joss.02521
   Google Scholar
• Streiner, D. L., Norman, G. R., & Cairney, J. (2015). Health Measurement Scales: A practical guide to their development and use (5th ed.). Oxford University Press.
   Google Scholar
• Tasca, G. A., Angus, L., Bonli, R., Drapeau, M., Fitzpatrick, M., Hunsley, J., & Knoll, M. (2019). Outcome and progress monitoring in psychotherapy: Report of a Canadian psychological association task force. Canadian Psychology / Psychologie Canadienne, 60(3), 165–177. https://doi.org/10.1037/cap0000181
   Web of Science ®Google Scholar
• Thompson, E. R. (2007). Development and validation of an internationally reliable short-form of the positive and negative affect schedule (PANAS). Journal of Cross-Cultural Psychology, 38(2), 227–242. https://doi.org/10.1177/0022022106297301
   Web of Science ®Google Scholar
• Vaz, A. M., Ferreira, L. I., Gelso, C., & Janeiro, L. (2023). The sister concepts of working alliance and real relationship: A meta-analysis. Counselling Psychology Quarterly, 1–22. https://doi.org/10.1080/09515070.2023.2205103
   Web of Science ®Google Scholar
• Verheul, R., Andrea, H., Berghout, C. C., Dolan, C., Busschbach, J. J. V., Van Der Kroft, P. J. A., Bateman, A. W., & Fonagy, P. (2008). Severity Indices of Personality Problems (SIPP-118): Development, factor structure, reliability, and validity. Psychological Assessment, 20(1), 23–34. https://doi.org/10.1037/1040-3590.20.1.23
   PubMed Web of Science ®Google Scholar
• Wampold, B. E., & Flückiger, C. (2023). The alliance in mental health care: Conceptualization, evidence and clinical applications. World Psychiatry, 22(1), 25–41. https://doi.org/10.1002/wps.21035
   PubMed Web of Science ®Google Scholar
• Wampold, B. E., & Imel, Z. E. (2015). The great psychotherapy debate: The evidence for what makes psychotherapy work (2nd ed.). Routledge.
   Google Scholar
• Weekers, L. C., Hutsebaut, J., & Kamphuis, J. H. (2019). The Level of Personality Functioning Scale-Brief Form 2.0: Update of a brief instrument for assessing level of personality functioning: The Level of Personality Functioning Scale - Brief Form 2.0. Personality and Mental Health, 13(1), 3–14. https://doi.org/10.1002/pmh.1434
   PubMed Web of Science ®Google Scholar
• Youngstrom, E. A., Perez Algorta, G., Youngstrom, J. K., Frazier, T. W., & Findling, R. L. (2021). Evaluating and validating GBI mania and depression short forms for self-report of mood symptoms. Journal of Clinical Child & Adolescent Psychology, 50(5), 579–595. https://doi.org/10.1080/15374416.2020.1756301
   PubMed Web of Science ®Google Scholar
• Youngstrom, E. A., Salcedo, S., Frazier, T. W., & Perez Algorta, G. (2019). Is the finding Too good to Be true? Moving from “more Is better” to thinking in terms of simple predictions and credibility. Journal of Clinical Child & Adolescent Psychology, 48(6), 811–824. https://doi.org/10.1080/15374416.2019.1669158
   PubMed Web of Science ®Google Scholar
• Youngstrom, E. A., Van Meter, A., Frazier, T. W., Youngstrom, J. K., & Findling, R. L. (2020). Developing and validating short forms of the parent general behavior inventory mania and depression scales for rating youth mood symptoms. Journal of Clinical Child & Adolescent Psychology, 49(2), 162–177. https://doi.org/10.1080/15374416.2018.1491006
   PubMed Web of Science ®Google Scholar
• Zimmerman, M., Ruggero, C. J., Chelminski, I., Young, D., Posternak, M. A., Friedman, M., Boerescu, D., & Attiullah, N. (2006). Developing brief scales for Use in clinical practice: The reliability and validity of single-item self-report measures of depression symptom severity, psychosocial impairment Due to depression, and quality of life. The Journal of Clinical Psychiatry, 67(10), 1536–1541. https://doi.org/10.4088/JCP.v67n1007
   PubMed Web of Science ®Google Scholar