Systematic review of prospective adult mental health outcomes following affirmative interventions for gender dysphoria

Abstract

Introduction

Many transgender and gender diverse (TGD) people experience high levels of mental health difficulties. Different gender-affirmative interventions exist which aim to alleviate gender dysphoria, with the hope that this in turn will improve the mental health and quality of life of TGD people.

Objective

This systematic review aims to examine the effectiveness of affirmative dysphoria-focused interventions for adults in improving mental health outcomes.

Methods

Inclusion criteria were published, prospective design studies with pre and post-test intervention mental health outcomes, following a gender affirmative intervention. Single-person case studies and measures of sexual distress and body image were excluded. Database searches were conducted with additional hand-searching (database inception to August 2023). Study quality was double-rated using a validated tool.

Results

Twenty-nine studies (2789 total participants) were identified that quantitatively assessed mental health outcomes (including depression, anxiety and self-esteem) following hormone, surgical and/or psychotherapeutic interventions. No eligible qualitative research or studies of voice and communication therapy or hair removal/electrolysis were found. The majority of studies of hormone treatments found that mental health, and in particular depression outcomes, significantly improved post-intervention and at follow-up. Improvements following surgical interventions were reported at follow-up time points beyond 6 months after the end of intervention. The majority of surgical intervention studies at post-intervention to 6-month follow-up showed no significant anxiety change, with mixed results for other mental health conditions. The majority of psychological interventions reported improvements in various mental health outcomes. The majority of included studies were assessed as being ‘medium’ or ‘weak’ quality.

Conclusion

This review indicates that interventions for gender dysphoria, especially hormone and psychological treatments, may have mental health benefits for TGD people. Further research is required to examine the relationship between mental health post-intervention and minority stressors, visual congruity and intersectional factors.

Keywords:

• Transgender
• prospective
• gender dysphoria
• mental health
• systematic review

Introduction

Transgender and gender diverse (TGD) individuals are at higher risk of developing mental health difficulties compared to cisgender people (Auer et al., 2013; Davey et al., 2014; Heylens et al., 2014). At the point of assessment within gender clinics, TGD people have significantly higher levels of mental ill health than the cisgender population, with the most common presentations being depression and anxiety (Dhejne et al., 2016; Millet et al., 2017; Newfield et al., 2006). Bachmann and Gooch (2017) found that 46% of transgender people had thought about ending their own life—a significantly larger proportion compared to cisgender lesbian, gay and bisexual participants (31%). The survey reported that two-thirds of transgender participants had experienced depression in the past year, with 12% of participants having made an attempt to end their life (Bachmann & Gooch, 2017). These results are supported by a range of studies indicating high levels of mental health difficulties for TGD people (Jones et al., 2019; Newfield et al., 2006; Nobili, Glazebrook, Arcelus, et al., 2018). It is critical that TGD people access effective and useful mental health support. This systematic review aims to understand the effectiveness of gender dysphoria interventions on the mental health of TGD people. The review will build on existing research on TGD mental health and explore different types of existing gender-affirmative interventions.

Predictors of mental health difficulties

There are four theorized contributors to distress and mental health difficulties in TGD people: 1) gender dysphoria, 2) minority stressors, 3) reduced social support and 4) reduced access to healthcare services (Carmel & Erickson-Schroth, 2016; Riggs et al., 2015). Gender dysphoria describes the sense of feeling unhappy, uncomfortable and unaligned with one’s sex assigned at birth. This sensation is persistent for many TGD people and is often present from early childhood. The term ‘gender dysphoria’ is used by the American Psychological Association (DSM-V; APA, 2013) and has also been known as transsexualism (ICD-10) or gender incongruence (ICD-11). The term is conceptualized as an independent diagnostic identity that is not necessarily associated with comorbid psychiatric conditions. The dysphoria may cause clinical distress and impairment in functioning (Sohn & Bosinski, 2007).

Minority stress theory describes how minority stressors (e.g. bullying, discrimination, internalized anti-trans attitudes and microaggressions) are strong predictors of suicidality and mental health difficulties for TGD people (Tebbe & Moradi, 2016). TGD people experience high rates of discrimination, harassment, abuse and violence based on their gender identity (Bachmann et al., 2017; McNeil et al., 2012). TGD people experience inequality or discrimination regarding housing, education, crime, employment, economic status, citizenship, health and social care (Mitchell & Howarth, 2009). Compared to cisgender people, TGD people are less likely to have perceived family and social support, with this directly predicting higher levels of psychological distress (Bariola et al., 2015; Başar et al., 2016; Davey et al., 2014; Tebbe & Moradi, 2016). TGD people experience significant barriers to accessing mental health services, with common reported factors including care services not being knowledgeable about TGD health and issues, financial barriers and discrimination (Safer et al., 2016). This lack of access to mental health services in turn impacts on mental health treatment and recovery (McCann & Sharek, 2016).

Gender-affirming interventions for gender dysphoria

A number of interventions exist that aim to directly reduce gender dysphoria experienced by individuals. These interventions introduce physical, psychological, social and behavioral changes to reduce the individual’s experienced discrepancy between their sex assigned at birth (or biological sex) and their gender identity (Byne et al., 2012). These include medical interventions that alter the biological and/or physical appearance of an individual, psychological/psychosocial interventions (e.g. therapeutic peer-led group spaces) and adaptive methods to address physical appearance and internal congruity (e.g. breast binding, genital tucking, body sculpting). The theory behind these interventions is that alleviating the gender dysphoria-related distress may improve an individual’s discomfort, self-identity, self-esteem and interaction with minority stressors which may in turn lead to mental health benefits (Wylie et al., 2014). For the purpose of this review, the focus will be solely on gender-affirmative interventions, meaning that the purpose of the intervention is to alleviate one’s gender dysphoria without attempting to change or convert an individual’s self-perceived gender identity to a cisgender identity.

Medical interventions are characterized as gender-affirming surgical procedures and hormonal procedures and are increasingly available for adults experiencing gender incongruence (Yarbrough, 2018). Examples of gender-affirming surgical interventions for transgender women include vaginoplasties, augmentation mammoplasty and facial feminization surgery. Examples of gender-affirming surgical interventions for transgender men include chest masculinization surgery, phalloplasty and scrotoplasty, amongst others (Yarbrough, 2018). Common hormonal treatments for TGD people include hormonal replacement therapy (HRT) and administration of estrogens, testosterone, hormone blockers and androgens (Costa & Colizzi, 2016). These interventions often follow rigorous assessment and can be offered alongside psychotherapeutic and social support (Wylie et al., 2016).

There is a growing body of evidence to suggest that medical interventions may improve the quality of life for TGD people. Several reviews of cross-sectional research (comparing TDF with non-TGD participants) have found that hormonal treatment is closely linked to an improvement in anxiety (Costa & Colizzi, 2016), depression, and quality of life (Nguyen et al., 2018; Rowniak et al., 2019). Reviews of surgical interventions have found improvements in quality of life for transgender men (Defreyne et al., 2017; Passos et al., 2020) and women (Zagami et al., 2019). One review found that the majority of studies reported significant improvements in psychological symptoms/quality of life following co-occurring hormonal and surgical intervention (Murad et al., 2010), with concerns over poor overall study quality and inconsistencies in outcomes between studies. There has been a rapid increase of research in recent years into the effectiveness of TGD-related interventions (Tomita et al., 2019). Despite this, there has not been an up-to-date review that captures prospective pre- and post-intervention design outcomes. Furthermore, there has not been a review of surgical interventions focusing specifically on mental health outcomes.

For some TGD people, medical interventions may not be available or necessary in reducing feelings of gender dysphoria. Many other interventions are utilized prior to, alongside or alternatively to medical procedures (Nieder et al., 2019). Gender-affirming psychological therapies and peer-support initiatives have been developed to explore the dysphoria in an affirming and supportive environment (Austin & Craig, 2015). Other, more adaptive interventions are used such as breast binding, genital tucking, exercises and workouts, body-sculpting, and voice and communication therapy. Despite the many different interventions that exist, little research has been conducted into the impact of such methods on mental health. There are no known reviews of research on non-medical interventions designed to alleviate gender dysphoria.

Whilst previous reviews have examined cross-sectional and longitudinal outcomes for medical interventions, there has not been a review of existing research of all types of intervention, including medical, psychotherapeutic, social and adaptive. There has been little study of the differences between these types of interventions and their respective impact on the mental health outcomes of participants. This may be useful in expanding our understanding of the respective and relative benefits of each of the interventions.

The present study

This systematic review aims to analyze and synthesize studies of gender affirmative interventions designed to affirm gender identity and alleviate gender dysphoria in adults. The study aims to build on previous systematic literature reviews by conducting an analysis of existing pre- and post-test prospective longitudinal research to better understand the temporality and effectiveness of interventions on mental health outcomes. It will examine:

1. The effectiveness of gender-affirming interventions for adults experiencing gender dysphoria in improving mental health outcomes for participants

2. Whether specific factors such as type of intervention and length of time post-intervention impact on different mental health outcomes

3. The quality and areas of existing research to identify new areas of study

The review will examine only studies that include a pre- and post-intervention outcome measure of subjective mental health. We also intend to examine the presence of qualitative research that captures changes in mental health pre- and post-intervention. Different types of intervention were included in order to compare and contrast outcomes between interventions and identify gaps in the research literature.

This review will provide important information about whether gender-affirmative interventions that aim to reduce gender dysphoria are effective in improving the mental health of TGD adults. Furthermore this review will identify ways to optimize the focus, design and reporting of future research.

Method

Protocol and registration

This systematic review method follows the guidelines from the AMSTAR checklist for systematic review quality (Shea et al., 2017). The search terms and study inclusion criteria were developed under consultation with a research librarian and experts-by-experience from a NHS Gender Identity Clinic. Two expert-by-experience groups were used in this consultation with transgender men and transgender women to ensure that a range of interventions and search terms were considered. A research protocol for the review was devised on the basis of the existing literature and PROSPERO records prior to starting the review data search. The protocol and search strategy were registered on PROSPERO; registration number CRD42021241800.

Search strategy

An electronic search was conducted in May 2022 and then updated in August 2023 using the following electronic databases: Medline (PubMed), PsycInfo, Web of Science, Scopus and Embase. Additional hand-searching was conducted in GenderWatch (ProQuest) and LGBT Life with Full Text (EBSCO). The search terms used were formed of three main terms, using the keywords (transgender OR transsexual OR "gender dysphori*" OR "gender incongruen*") AND (interven* OR program* OR treatment OR therapy OR medicat* OR surg* OR group*) AND ("mental health" OR depress* OR anxi* OR trauma OR PTSD OR mood OR "quality of life" OR emotion*). These search terms were based on multiple scoping searches. The aim of these terms was to generate a large database of gender dysphoria interventions measuring mental health outcomes to ensure a comprehensive and inclusive list of possible articles. Secondary searches were also conducted via hand-searching, using the reference lists of included articles and exploring grey literature online. Search terms were used to search databases that contained articles with these terms in the abstract or title. All eligible studies based on the abstract and title were added to an EndNote database and transferred into Microsoft Excel. These were de-duplicated automatically.

Inclusion criteria

Eligible studies in the present review had to fulfill the following inclusion criteria: (1) published within a peer reviewed journal or as a doctoral dissertation; (2) published in the English language, due to a lack of resources for accurate translation; (3) the intervention must be gender-affirmative; (4) the intervention must have a prospective pre- and post-intervention measure of outcomes; (5) the study participants are adults over the age of 18. Studies that had an age range of participants of 16+ were included if the mean age was above 21; (6) the main function of the intervention was to help with alleviating gender dysphoria-related distress or mental health difficulties; (7) a self-report mental health outcome is reported. Quantitative, qualitative and mixed-method research were included in the search. Any length of follow-up for longitudinal studies was included.

Exclusion criteria

Studies were excluded: (1) where the focus of the intervention was on physical health, HIV/AIDS or substance misuse prevention were not included; (2) there was no self-report outcome; (3) there was recorded outcome data for at least one DSM-5 internalizing disorder (depression, anxiety and related disorders, psychotic disorders, PTSD and OCD) (American Psychiatric Association, 2013) or measures of psychopathology, emotional functioning, distress and self-esteem that took the form of either clinical diagnoses or relevant symptoms from a validated symptom rating scale; (4) studies that included a measure of sexual distress, body image, body dysmorphia or quality of life without a mental health related outcome (e.g. income, stigma and social support); (5) single-person case studies were not included due to difficulties in making generalizable conclusions.

Screening procedure

A pilot screening was first conducted to ensure inter-coder reliability. The first stage of selection involved the lead researcher independently screening the papers’ abstracts and titles based on the inclusion criteria. A research assistant was a second rater during this screening stage for 600 studies. Outcomes between researchers were compared and Cohen’s kappa (κ) statistics were calculated for categorical screening codes and found to be strong (κ = .81; % of agreement: 98%; McHugh, 2012). In the second stage the same researchers screened the full text of 25% of the remaining papers. Reliability between raters was checked and found to be excellent (κ = .89%; % of agreement: 98%). Discrepancies between researchers were resolved through discussion with the second and third authors.

The screening procedure and number of articles screened and accepted at each stage can be seen in Figure 1.

Figure 1. A PRISMA Flow diagram detailing the number of papers included within each section of screening (Page et al., 2021).

Data extraction and review synthesis

The data were extracted from the studies by the lead researcher. A pilot data extraction was conducted with 10 studies with a research assistant and excellent inter-coder reliability was found (κ = 95%; McHugh, 2012). Discrepancies were resolved via discussion with the coauthors. A data extraction form recorded details about study characteristics, intervention design, the results and measures of outcomes. The standardized form for data extraction recorded (a) study design; (b) study location; (c) recorded demographic information including participant ethnicity, socio-economic status, sexuality and relationship status; (d) number of transgender male, transgender female and non-binary participants; (e) mean age of participants and age range; (f) type of intervention (e.g. surgical, hormonal, psychological, adaptive) and description of intervention procedure; (g) follow-up time points; (h) internalizing disorder/symptom measures used; (i) summary of relevant outcomes; (j) background and biases of authors, where possible to record.

Further data was requested from several authors of papers and this was included, if provided. The study characteristics and outcomes were extracted and synthesized as a review.

Risk of bias

All studies were systematically evaluated for risk of bias using the Effective Public Health Practice Project (EPHPP) Quality Assessment Tool for Quantitative Studies (EPHPP, 1998; Silverman et al., 2012). This standardized assessment tool allows classification of studies based on quality of design and measures (1) selection bias, (2) study design, (3) confounders, (4) blinding, (5) data collection methods, (6) withdrawals and dropouts, (7) intervention integrity and (8) analyses. Total scores were generated, with 1 indicating the highest quality level, and 3 being the lowest. A score of 3 on two or more of the criteria would result in a ‘weak’ rating. The assessment was conducted by the lead author and repeated by a research assistant for reliability. Discrepancies were resolved in discussion with the coauthors.

Results

This section synthesizes the existing literature of pre- and post- gender-dysphoria focused interventions with mental health outcomes. Given the significant differences between different intervention modalities, the findings are presented by intervention type, separately considering studies of hormone treatments, surgical interventions and psychotherapy, social and adaptive interventions. Other study-specific factors are considered when examining intervention type. The reporting quality of included studies is recorded in order to understand possible biases within the existing research literature.

Study characteristics

Twenty-nine studies were included in the present review, including a total of 2789 participants at pretest. The sample size in each study varied from eight to 873 participants. All but seven (24%) of the studies included medical interventions (hormonal or surgical). The most common study location was Italy (n = 7, 24%) followed by UK (n = 6, 21%), the USA (n = 4, 14%), Germany (n = 2, 7%), Sweden (n = 2, 7%), Belgium (n = 2, 7%) and the Netherlands (n = 2, 7%). One multisite study was conducted in Netherlands, Belgium, Norway and Italy (Matthys et al., 2021). Single studies were also conducted in Brazil, Iran, Japan, Switzerland, Thailand and Türkiye. Further details about the factors relating to intervention content and study delivery can be found in Table 1.

Table 1. Summary of included studies.

Author(s) Country	N and study’s own description of sample	Mean age at pretest	Category (1 = HT, 2 = surgery, 3 = therapy)	Study description of intervention	Time-point from pretest (months)	Outcome measure	Results at post-test/follow-up	Quality rating (1 = strong, 2 = moderate, 3 = weak)
Aldridge et al. (2020) UK	178 95 AMAB 83 AFAB	23	1	Gender-affirming hormone treatment	18	HADS-D HADS-A	Depression: Sig. lower after intervention Anxiety: Lower scores but not significantly different after intervention.	2
Briggs et al. (2018) USA	8 MtF, FtM, Non-binary, Gender-fluid	“Adult”	3	Psychotherapy (10 weekly 90-minute group sessions based on Somatic Experiencing Informed Psychotherapy)	2.5	PHQ-9 GAD-7 WHOQoL	Depression: No change Anxiety: No change Psychological wellbeing: Sig. lower after intervention	3
Budge et al. (2021) USA	17 Non-binary, trans-feminine and trans-masculine	Two groups: 27.5 and 31.1	3	Transgender Affirmative Psychotherapy (12 weekly sessions) and psychoeducation session (Building Awareness of Minority Related Stressors)	3, 9	OQ-45	Distress: Significant reduction at post-test but no difference between baseline and follow-up	2
Cardoso da Silva et al. (2016) Brazil	48 MtF	31.2	2, 3	Surgery (penile inversion vaginoplasty, urethroplasty). Participants also attended “supportive group therapy sessions” weekly or fortnightly	At least 12 months	WHOQoL	Mental health/negative feelings: Sig. improvement after intervention Self-esteem: No change after intervention	2
Chaovanalikit et al., 2022 Thailand	37 MtF	26.2	2	Vaginoplasty	6	RSESPHQ-9	Self-esteem: Significant decrease after intervention Depression: Significant decrease after intervention	1
Colizzi et al. (2013) Italy	70 45 MtF 25 FtM	MtF: 29.3 FtM: 26.8	1	Hormone treatment (MtF: transdermal estradiol gel, FtM: intramuscular testosterone injections)	12	PSS	Stress: Sig. lower after intervention	2
Colizzi et al. (2014) Italy	107 78 MtF 29 FtM	29.2 MtF: 29.3 FtM: 29.1	1	Hormone treatment (MtF: transdermal estradiol gel, FtM: intramuscular testosterone injections)	12	SDS SAS SCL-90-R	Depression: Sig. lower after intervention Anxiety: Sig. lower after intervention Obsessive-Compulsive Disorder: Sig. lower after intervention Psychoticism: Sig. lower after intervention	2
Colizzi et al. (2015) Italy	118 82 MtF 36 FtM	30.2 MtF: 30.4 FtM: 29.8	1, 2	Hormone treatment (MtF: transdermal estradiol gel, FtM: intramuscular testosterone injections) 22 participants also received sex reassignment surgery (mammoplasty, vaginoplasty, phalloplasty)	Post-HT: 12 Post GRS: 24	DDIS DES	Dissociation: Sig. lower for both intervention groups after intervention	1
Costantino et al. (2013) Italy	50 FtM	29.8	1, 2	Hormone treatment for 12 months and then surgery (Testosterone under various preparation and routes, hysteroannexiectomy and reductive mammoplasty)	Post-HT: 12 Post-GRS: 18	Measure designed for study	Depression: No change	3
Cotton and Wood (2019) UK	8 7 AMAB 1 AFAB	45	3	Psychotherapy (6 two-hour fortnightly group sessions using a CBT approach)	3	WEMWBS	Mental wellbeing: Statistical analysis not conducted due to small sample size. Authors describe change as "meaningful increase" in wellbeing	2
Drydakis (2016) UK	40 23 MtF 17 FtM	MtF: 35.8 FtM: 34.1	2	Sex reassignment surgery (vaginoplasty or phalloplasty)	24	CES-D	Depression: Sig. lower after intervention for both MtF and FtM groups	3
Fisher et al. (2016) Italy	54 28 MtF 26 FtM	MtF: 32.5 FtM: 26.3	1	Hormone treatment (MtF: oral or transdermal estradiol treatment, FtM: intramuscular testosterone injections)	3, 6, 12, 24	BDI-II SCL-90-R	Depression: Sig. lower after intervention at all time-points for both groups Overall distress: Sig. lower at all time points for MtF group. Sig. lower for FtM group only at time points 3 and 4	2
Gümüşsoy et al. (2022) Türkiye	63 FtM	27.3	2	Hysterectomy and mastectomy	6	WHOQoL-BREF	Psychological health domain: Significant improvement in outcome after surgery	2
Heylens et al. (2014) Belgium	57 46 MtF 11 FtM	“Adult”	1, 2	Hormone treatment and surgery (genital surgery, breast augmentation/removal, facial hair removal, female feminization surgery). 11 participants did not receive surgery	3–6 after HT, 1–12 after GRS Mean length at follow-up = 39	SCL-90-R	Depression: Sig. lower after hormone treatment. No further change following surgery Anxiety: Sig. lower after hormone treatment. No further change following surgery Psychoticism: Sig. lower after hormone treatment. No further change following surgery	2
Isung et al. (2017) Sweden	10 MtF	44	2	Surgery (facial feminization surgery)	6	HADS-D HADS-A	Depression: No change Anxiety: No change	2
Knutson et al. (2021) USA	14 5 AMAB 9 AFAB	31.6	3	Psychotherapy (text-message based treatment based on cognitive-behavioral therapy (CBT))	1.5, 4.5	PROMIS	Depression: Sig. lower after intervention. No change at follow-up compared to pretest Anxiety: Sig. lower at post-intervention and follow-up	2
Lindqvist et al. (2017) Sweden	146 MtF	36	2	Surgery (gender reassignment surgery)	12, 36, 60	SF-36	Mental health: No change	3
Matthys et al. (2021) Netherlands, Belgium, Norway, Italy	451 MtF 422 FtM	MtF:27.0 FtM:22.0	1	Hormone treatment (range of interventions)	3, 6, 9, 12, 18, 24, 36	PANAS	Negative affect: No change in both groups in median score at 36-month follow-up	3
Megeri and Khoosal (2007) UK	40 MtF	41.8	2	Surgery (gender realignment surgery)	Post-surgery	HADS BDI STAI GHQ	Depression: No change Anxiety: No change	3
Mazzoli et al. (2022) Italy	62 24 transwomen 38 transmen	Average: 27	1	Hormone treatment (oral, transdermal and/or injection) and ‘psychological support’ every 3 months	12	LSAS SCL-90-R	Social anxiety: Significant decrease Distress/psychopathology: Significant decrease post-intervention	2
Meier (2014) USA	79 FtM	26.6	1	Hormone treatment (administration of testosterone)	3–4, 10–12	DASS RSES SF-36	Depression: Sig. lower at post-intervention and follow-up Anxiety: Sig. lower at post-intervention and follow-up Self-esteem: No change Stress: Sig. lower at post-intervention and follow-up	2
Metzger and Boettger (2019) Switzerland, Germany	26 FtM	27.2	1	Hormone treatment (subcutaneous, intramuscular or transdermal administration of testosterone)	3, 6	BDI-II NEO-PI-R	Depression: Sig. lower after intervention for both measures at both time-points Neuroticism: Sig. lower after intervention	1
Naeimi et al. (2019) Iran	42 FtM	34.2	2	Surgery (gender reassignment surgery, penile and scrotal reconstruction)	6	SF-36	Mental health: Sig. improvement after intervention	2
Nobili et al. (2020) UK	118 59 AFAB 59 AMAB	27.95 AFAB: 22.5 AMAB: 33.4	1	Hormone treatment. Participants may have had surgery in that time but numbers are not reported.	12	HADS-A	Anxiety: No change	3
Oda and Kinoshita (2017) Japan	14 FtM	23.9	1, 3	Hormone treatment (cross-sex hormone treatment) and psychotherapy	12	MMPI-2	Depression: Sig. lower after intervention Anxiety subscale, also labeled as “Hysteria” in paper: Sig. lower after intervention Schizophrenia sub-scale: Sig. lower after intervention	3
Papadopulos et al. (2017) Germany	39 MtF	38.6	2, 3	Surgery (orchiectomy, penectomy and vaginoplasty). Undescribed psychotherapy ran from before the pretest to beyond the post-test	12	PHQ-4 RSES	Depression: Sig. improvement after intervention Self-esteem: Sig. improvement after intervention	2
Udeze et al. (2008) UK	40 MtF	47.3	2	Gender reassignment surgery	6	SCL-90-R	Obsessive-compulsive disorder: No change Depression: No change Anxiety: No change Psychoticism: No change	2
van de Grift et al. (2016) Netherlands	33 Transmen	26.1	2	Surgery (mastectomy)	6	RSES	Self-esteem: No change	1
van Dijk et al. (2019) Netherlands, Belgium, Italy	398 193 Trans men 205 Trans women	Trans men: 23 Trans women: 29	1	Hormone treatment (Trans men: intramuscular or transdermal testosterone administration, Trans women: oral or transdermal estradiol treatment)	3, 6, 12	Single question on nervous-ness and anxiety	Anxiety: No change	3

Sample: AFAB: Assigned Female at Birth; AMAB: Assigned Male at Birth; MtF: Male to Female transgender person; FtM: Female to Male transgender person. Intervention: CBT: Cognitive-Behavioral Therapy; HT: Hormone Treatment; GRS: Gender Reassignment/Realignment Surgery. Outcome measure: BDI-II Beck’s Depression Inventory; CES-D: Center for Epidemiological Studies Depression; DASS: Depression Anxiety Stress Scales; DDIS: Dissociative Disorders Interview Schedule; DES: Dissociative Experiences Scale; GAD-7: Generalized Anxiety Disorder Assessment; GHQ: General Health Questionnaire; HADS: Hospital Anxiety and Depression Scale; LSAS: Liebowitz Social Anxiety Scale; MMPI-2: Minnesota Multiphasic Personality Inventory; NEO-PI-R: Revised NEO Personality Inventory; OQ-45: Outcome Questionnaire 45; PANAS: Positive and Negative Affect Scale; PHQ-9: Patient Health Questionnaire; PROMIS: Patient-Reported Outcomes Measurement Information System; PSS: Perceived Stress Scale; RSES: Rosenberg Self-Esteem Scale; SAS: Zung Self-Rating Anxiety Scale; SCL-90-R: Symptom Checklist-90-Revised; SDS: Zung Self-Rating Depression Scale; SF-36: 36-Item Short Form Survey; STAI: The State-Trait Anxiety Inventory; WEMWBS: The Warwick-Edinburgh Mental Wellbeing Scale; WHOQoL: World Health Organization Quality of Life; Scale. Results: Sig: Statistically significant.

All of the studies used a pre- and post-test design and collected self-reported outcome data. Nine of the studies (31%) also recorded cross-sectional or experimental outcome data, comparing post-test outcomes with previously collected population data (Lindqvist et al., 2017; Papadopulos et al., 2017), cisgender participants (Keo-Meier et al., 2015; Mazzoli et al., 2022; Meier, 2014; Metzger & Boettger, 2019; van de Grift et al., 2016) or participants who had experienced gender dysphoria who had not received the intervention (Fisher et al., 2016; Nobili, Glazebrook, Bouman, et al., 2018). There were no qualitative studies that captured the mental health experiences of participants pre and post-intervention.

The mean age of participants across the studies ranged from 22.0 (Matthys et al., 2021) to 47.3 (Udeze et al., 2008). Seven studies (24%) described interventions for people who were assigned female at birth (AFAB), seven (24%) for people who were assigned male at birth (AMAB) and 15 (52%) for both. Three studies (10%) reported the inclusion of non-binary participants. No study reported the inclusion of intersex participants.

Seventeen studies (59%) measured depression outcomes, 13 studies (46%) measured anxiety outcomes, five studies (17%) measured self-esteem outcomes and 11 studies (38%) measured generic mental health, mental wellbeing, distress and negative affect. Studies also included measures of dissociation, psychoticism, obsessive-compulsive disorder symptoms and stress.

Thirteen studies (45%) measured outcomes at additional follow-up time-points beyond the post-test. Post-test and follow-up time points ranged from immediately post-surgery (Megeri & Khoosal, 2007) to 60 months (Lindqvist et al., 2017).

Hormone treatment

Hormone treatment describes the use of prescribed medication in order to change hormone levels to be consistent with a gender identity. Fourteen studies (48%) examined a hormone intervention, recording outcomes from 2204 total participants (range per study: 14-873). Almost all of these studies reported outcomes after participants had received hormonal treatment for 12 months, with two studies (7%) reported final post-test outcomes at six months (Heylens et al., 2014; Metzger & Boettger, 2019) and one at 18 months (Aldridge et al., 2020). Two studies measured follow-up time points of beyond 12 months. One of these found no change at any follow-up time points (18, 24 and 36 months: Matthys et al., 2021), and the other reported a significant reduction in outcome measures (depression, distress) in all follow-up time points compared to pretest (Fisher et al., 2016).

The average age of participants in each study ranged from 23 to 33.4 which was notably lower than the range of other intervention types. There was no identified difference in outcomes between studies that had a younger mean age compared to older mean age.

The most commonly reported hormonal treatment were oral or transdermal estradiol gel and intramuscular testosterone injections. There were no studies that only included AMAB participants. Four studies included solely AFAB participants (Costantino et al., 2013; Meier, 2014; Metzger & Boettger, 2019; Oda & Kinoshita, 2017) where intramuscular testosterone was the main treatment. Three of these four studies (excluding Costantino et al., 2013) found significant decreases in outcome measures at post-test. The rest of the studies (n = 10) included both AMAB and AFAB participants. In these studies, no differences in outcomes were reported between AMAB and AFAB participants, apart from at one time-point measure in one study, with there being no difference at further follow-up time points (Fisher et al., 2016). There were no observed differences in outcomes for participants of other participant demographics (e.g. ethnicity, socio-economic status).

Seven studies of hormonal intervention measured depression outcomes, with six of these reporting a significant reduction in depression scores at pretest, and one study reporting no change following intervention (Costantino et al., 2013). These changes were maintained in studies that reported multiple follow-up time points. Heylens et al. (2014) reported that both depression and anxiety improved following hormonal intervention but no further change in symptoms was found after additional surgical intervention.

Four studies (14%) found that anxiety significantly reduced and three (10%) showed no significant change after hormonal intervention. All ten studies (34%) with other mental health outcome measures (stress, OCD, psychoticism, dissociative symptoms and distress) reported significant improvements following hormone intervention, apart from one study reported no change in negative affect between pretest and 36-month follow-up (Matthys et al., 2021).

Two studies were found to have ‘strong’ reporting quality, seven were ‘moderate’ and five were ‘weak’, with this being consistent with the distribution of quality ratings across the review (see later Quality Assessment section). There were no notable differences in outcomes correlating with the quality assessment.

Surgical interventions

Surgical interventions describe the medical alteration of a person’s physical appearance or functioning of existing genitalia to reduce dysphoria and lead to changes more closely aligned with a person’s gender identity. Fourteen studies (48%) involved a surgical intervention, recording outcomes from 763 total participants (range per study: 10-146). The average age of participants within each study ranged from 26.1 to 47.3. None of the three studies with an average of participants over the age of 40 reported statistically significant improvements in any outcomes. There were no observed differences between or within studies with different participant demographics (e.g. ethnicity, socio-economic status).

Eight studies reported outcomes at 6-month follow-up after surgery, with five studies reporting outcomes at 12+ months. Whilst the majority of studies (six of the eight) at 6-month follow-up found no change in any mental health outcomes, significant improvements were reported in four of the five studies with follow-up time-points of 12 months or more.

Three studies included both AFAB and AMAB participants (Colizzi et al., 2013; Drydakis, 2016; Heylens et al., 2014) with the majority of studies including solely AFAB (n = 4) or AMAB participants (n = 7), which determined the type of surgical intervention received. These included reductive mammoplasty/breast augmentation (n = 5), vaginoplasty (n = 5), phalloplasty (n = 3), facial feminization surgery (n = 2), facial hair removal (n = 1), hysterectomy (n = 1) and penectomy/orchiectomy (n = 1). Three studies did not report details of the surgery (e.g. “gender-affirming surgery”), with mixed effectiveness outcomes reported in these studies. All studies that included a vaginoplasty or phalloplasty found a significant improvement in mental health outcomes, including depression and dissociation. There were mixed findings for studies that reported mammoplasty. There was no observed change following facial feminization surgery or facial hair removal in outcomes compared to pretest (Isung et al., 2017) or compared to post-test after a hormone intervention (Heylens et al., 2014).

When comparing different outcome symptoms at post-test across the studies, three studies reported a significant improvement in depression outcomes following surgical intervention, with five studies reported that there was no change in depression outcomes. All four studies that measured anxiety outcomes following surgery reported no significant change in outcomes compared to pretest scores. There were similar mixed findings for other outcome measures of overall mental health, self-esteem and dissociative/psychoticism symptoms.

Three studies were found to have ‘strong’ reporting quality, seven were ‘moderate’ and four were ‘weak’. There were no notable differences in outcomes correlating with the quality assessment.

Psychotherapy, social and adaptive interventions

The majority of studies were of hormone or surgical approaches. Seven studies (24%) described interventions that used a psychotherapeutic approach, recording outcomes from 140 total participants (range per study: 8–48). Three of these studies included a psychotherapeutic approach alongside either a hormonal or surgical intervention. The average age of participants within each study ranged from 27.5 to 45. There were no notable differences in outcome depending on age. The post-test measure occurred between 2.5 to 12 months after the pretest. Two studies recorded a follow-up outcome (no change in outcomes at 9 months: Budge et al., 2021; significantly reduced anxiety but not depression at 4.5 months: Knutson et al., 2021). Four studies included both AFAB and AMAB participants, one included solely AFAB participants and two included solely AMAB participants, with no notable difference in outcomes between these studies.

One study described Somatic Experiencing Informed Psychotherapy, a ten-session 90-minute group therapy which focuses on the autonomic nervous system and emotional regulation (Briggs et al., 2018). This intervention placed emphasis on the “felt sense”, identifying biopsychosocial elements of a gender diverse identity which contributes to gender dysphoria and providing ways of helping individuals build emotional regulation skills when distressed.

Two studies integrated Cognitive-Behavioral Therapy (CBT) principles. One of these included via six two-hour group therapy sessions (Cotton & Wood, 2019) adapting an existing transgender affirmative cognitive behavioral therapy (Austin & Craig, 2015) which includes cognitive modification to negative thoughts and core beliefs (e.g. “I will never be a ‘real’ woman”) and behavioral activation to encourage and help participants to feel more comfortable living authentically within the community. The intervention also included modules on problem solving and management of stressful social situations related to TGD experiences. Another study also adapted the transgender affirmative cognitive behavioral therapy (Austin & Craig, 2015) approach to create a six-week 24-day manualised text-message intervention alongside psychoeducation and skill-based exercises which considered the role of gender dysphoria in distress and text-message based intervention (Knutson et al., 2021).

One study described 12 sessions of Transgender Affirmative Psychotherapy (TA) or the same intervention with an additional psychoeducation session on Building Awareness of Minority-Related Stressors (BAMS) (Budge et al., 2021). There was no additional detail to the content of the intervention used in this study, as the therapists were instructed to follow their own choice of theoretical framework (one therapist provided psychodynamic psychotherapy and the other provided person-centred psychotherapy).

The remaining three studies described how psychotherapy took place alongside another intervention. One of these was a group intervention (Cardoso da Silva et al., 2016) and another had an average period of psychotherapy of 28.7 months (Papadopulos et al., 2017). However, beyond this, there was no additional information about the type, content or form of psychotherapy.

All of these studies reported an improvement in mental health at post-test, psychological wellbeing, depression and/or anxiety symptoms, apart from one intervention which found no change in depression or anxiety symptoms (Briggs et al., 2018).

No studies were found to have ‘strong’ reporting quality, with five studies having ‘moderate’ and two with ‘weak’ quality. There was no notable difference in findings between studies with different quality ratings.

Social and adaptive interventions

There were no studies found of other types of gender-dysphoria focused interventions for TGD people with gender dysphoria. This included interventions such as voice and communication therapy, hair removal/electrolysis, binding, genital tucking, exercise and workouts, body-sculpting, internet support groups, peer support and self-medicating of hormone medication (Gendered Intelligence, 2021; UCSF Transgender Care, 2021).

Quality assessment

The quality assessment classified studies into ‘strong’, ‘moderate’ and ‘weak’ based on how information was transparently presented across six domains (selection bias, study design, confounders, blinding, data collection method, withdrawals). It also considers intervention integrity and analytic methods. A summary of the quality assessment of each study can be found in Appendix A. Four studies (14%) were rated as having a ‘strong’ reporting quality, 15 (52%) with a ‘moderate’ quality and 10 (34%) with a ‘weak’ quality. The main reasons for poor quality assessment ratings were ignoring of confounders, selection bias and participant withdrawals. Sixteen studies (54%) did not discuss or examine the presence of any confounders such as age, demographics or pretest scores on outcomes. Eight studies (28%) were assessed to have a significant level of selection bias, with the most common reason being that less than 60% of people who registered their interest or were approached for the study ended up completing the pretest measure (EPHPP, 1998; Silverman et al., 2012). Nine studies (31%) reported high drop-out rates, with over 40% of participants not completing the post-test measures. Several of these papers did not report participant drop-out rates and only described participants that completed both pre- and post- measures. There were no observed differences between studies with varying assessed quality and outcomes when considering different types of intervention and participant demographics.

Discussion

This systematic review synthesized 29 studies in which mental health outcomes were measured before and after an intervention designed to alleviate gender dysphoria in adults. The majority of identified studies were of medical interventions, specifically hormone treatment and surgical procedures that altered physical characteristics of individuals. For hormone treatment, the majority of studies suggested that intervention was beneficial in improving mental health outcomes, especially for depression symptoms. The majority of studies of surgical interventions reported no change in outcomes following intervention at post-test, with greater improvements at more than 6-month follow-up time point measurements. It is important to note that in many of the studies, hormone therapy for participants had preceded the surgical intervention, which may have led to a ceiling effect for mental health improvements. The overall research quality of the included studies was categorized as ‘medium’ to ‘weak’. This is consistent with reports from other reviews which suggest a high risk of bias in study designs, small sample sizes and limited consideration of confounding factors (Baker et al., 2021; Dhejne et al., 2016). Due to the high heterogeneity of research methods across studies, it is difficult to make conclusive statements about the relative effectiveness of the different types of intervention. There is a need for more high-quality intervention studies within this area of research.

This review highlighted the lack of qualitative research that captured the ‘before and after’ experience of participants undergoing hormone or surgical intervention. Qualitative research enables multi-faceted health-care interventions to be developed in a way that considers the complexities of people, procedures and context (Thirsk & Clark, 2017). An understanding of the lived experience of participants will ensure that future interventions have a thorough theoretical underpinning and are optimally adapted to ensure that individuals are supported and given appropriate post-intervention care (Duggleby et al., 2020).

Interventions aiming to alleviate the dysphoria felt by many TGD people may not be sufficient to reduce mental health difficulties experienced by this population. Whilst gender dysphoria is a key predictor of mental health outcomes for TGD people, there are still many other significant factors that may predict poor mental health that may or may not be addressed by such interventions. There is a strong evidence base emphasizing that, especially for TGD people, mental health and resilience is predicted by the presence of peer-support, feeling connected to others (often referred to as TGD community connectedness; TCC) and the availability of safe spaces (Matsuno & Israel, 2018; Pflum et al., 2015; Puckett et al., 2019; Zeeman et al., 2017). For instance, one study of transgender youth found that the use of a chosen name by members of an individual’s family and wider system is associated with reduced depressive symptoms and suicidal ideation (Russell et al., 2018). Additional factors linked to mental health outcomes include having positive role models, as well as having an online internet supportive community (Matsuno & Israel, 2018; Pilecki, 2015). One study found that psychosocial wellbeing was moderated by social support for individuals following surgery (Schultz, 2002). It may be the case that many of the interventions described in this review integrated elements of peer-support, affirmative care and inclusion into a supportive community, all of which may have contributed to overall mental health outcomes. Future intervention studies should aim to examine these factors at post-intervention and follow-up (e.g. by using The Gender Minority Stress and Resilience measure; Testa et al., 2015) and explore whether this is more closely correlated with mental health outcomes.

Hormone treatment

This review indicates that hormonal treatments are effective in improving low mood and other mental health difficulties for TGD people, whilst there were mixed findings for specifically anxiety measure outcomes. These findings are consistent with similar reviews of hormone treatment finding improvements in depression (Nguyen et al., 2018; Rowniak et al., 2019). These outcomes may be explained by interventions contributing to a reduction in gender dysphoria, which in turn may decrease participants’ global distress and improve their own sense of self-affirmation and self-efficacy. Hormone treatments often impact on an individual’s ability to visually and aurally ‘pass/blend’ as their identified gender, which may result in reduced instances of discrimination and associated perceived minority stress, all of which may impact on mental health outcomes (Cohn et al., 2018; To et al., 2020). Other possible benefits to such interventions are that they often enable access to trans-related healthcare and the integration of healthcare services within often “closely-knit trans communities” (Fomotar, 2016, p. 1; Puckett et al., 2018). Undergoing hormone interventions to reduce gender dysphoria may enable individuals “to live full and authentic lives” (Fomotar, 2016, p. 1). The average and ranged ages of participants in hormone studies were notably lower than studies of surgical or psychotherapeutic interventions. Future research should explore whether mental health outcomes are mediated by other quality of life factors including minority stress, discrimination and self-perceived authenticity, and whether age plays a significant role in explaining differences in outcomes.

Surgical interventions

Whilst previous reviews have found improvements in more holistic quality of life outcomes following surgery such as sexual satisfaction, psychosocial wellbeing and physical health (Defreyne et al., 2017; Passos et al., 2020), there has been little systematic examination of studies measuring mental health specifically following surgery. This review presented mixed findings on the effectiveness of surgical interventions at improving the mental health of TGD people in the short term (e.g. up to 6 months after surgery), with no improvements found for anxiety and fewer than half of the studies reporting depression improvements. This finding is consistent with mixed anecdotal blogs and experiences shared online of TGD people experiencing both satisfaction and/or depression post-surgery, with testimonies often raising themes of whether the surgery met expectations, whether they felt more validated and less dysphoric post-surgery and whether or not they feel ‘complete’ (Allen, 2023; Haimson, 2020). It is important to note that to the best of our knowledge there is not a peer-reviewed qualitative study exploring this post-surgery experience.

There are many feasible explanations for these findings. It may be the case, especially when relative to long-term and gradual change that occurs during hormonal treatment, that surgical interventions can be perceived as a fast and sudden change which may require a significant amount of time for individuals to adjust to. Future research should capture the long-term mental health effects of surgery. Surgical interventions generally occur much later for most TGD people’s treatment pathways (as reflected in the average age of participants in the studies), and therefore a ceiling effect may exist, where gains may have already been made from other interventions (e.g. hormone treatment) much earlier in transition. Another reason may be linked to esthetic change. There is evidence suggesting that visual conformity with affirmed gender (VCAG) or “passing” is inversely related to depression and anxiety outcomes for transgender people (To et al., 2020). One possible reason for the discrepancy in findings between hormone treatment and surgical interventions may be that, when compared to genital surgery, long-term hormone use may be more closely linked with changes in overall visual appearance and subsequent VCAG. Facial feminization surgery is associated with improved VCAG and satisfaction (Morrison et al., 2016) and yet the two included studies of this intervention reported no significant change in outcomes, suggesting that there may be other factors beyond VCAG that explain the lack of mental health changes following surgery. No studies examined the impact of voice and communication interventions on participant mental health. Previous studies have found that having a voice that is congruent with participants’ own gender identity is associated with greater psychological wellbeing and quality of life (Hancock et al., 2011; Watt et al., 2018). Future studies should examine whether changes in voice and perceived VCAG post-intervention mediates mental health outcomes post-surgery.

Psychotherapy, social and adaptive interventions

This review also examined other types of non-medical interventions. All studies but one that included psychotherapeutic elements found an improvement in mental health outcomes. Two studies utilized CBT elements, drawing on recently developed trans-affirmative interventions (TA-CBT) which aim to incorporate cognitive-behavioral CBT theory alongside addressing relevant sources of stress for TGD people (Austin & Craig, 2015). Four papers included very limited information about the type or content of the psychotherapeutic approach and occurred alongside other interventions, which makes it difficult to determine the relative effectiveness of this approach or identify hypothetical active components in treatment.

Previous qualitative research has emphasized the value of psychotherapeutic approaches in exploring the dysphoria and mental health difficulties faced by many TGD people (Mizock & Lauren, 2016; Stephens, 2019). These studies have shown the importance of affirmation from the therapist within sessions and a therapist awareness of minority stress (Stephens, 2019). Common barriers identified to successful psychological interventions include the therapist “placing the burden of education on the client”, pathologizing gender and the therapist avoiding discussions about gender-related issues (Mizock, 2017; Mizock & Lauren, 2016; Stephens, 2019).

Social and adaptive interventions

There were no found appropriate studies examining social and adaptive interventions. Many studies incorporating social interventions did not meet the review’s inclusion criteria and yet may provide insight into how effective interventions may be designed. A significant number of studies exist in which the primary aim is to support and educate TGD people about living with HIV/AIDS. Many of these incorporate gender dysphoria interventions, peer-led mental health/substance-misuse education and community support alongside HIV/AIDS education (e.g. Hunter-Jones et al., 2021; Nemoto et al., 2005). Similar peer-led and community projects that focus on ways to alleviate gender dysphoria should be evaluated using comparable research methods. This review has highlighted the need for research into the efficacy of widely used adaptive interventions such as voice and communication therapy, hair removal/electrolysis, binding, genital tucking, exercise and workouts, amongst others.

Strengths and limitations

This review was conducted with a high level of systematic rigor. The search for studies was exhaustive, with study selection, data extraction and quality assessment all completed with high inter-rater reliability. The method followed AMSTAR guidelines to ensure high quality reporting of findings (Shea et al., 2017).

It is important to recognize the low research quality of several included studies and the possible impact this may have on the validity of any conclusions made. The limited reporting of study characteristics and confounding factors (e.g. length of intervention, relationship between outcome and participant age) made it difficult to clearly establish the association between study factors and outcomes. There is potential for authorship bias throughout much of the research literature, as several studies were authored by clinical staff in which interventions are routinely offered. All future intervention studies should conform to pre-established guidelines and checklists to improve the quality of the research and reporting style of studies (Shea et al., 2017).

There was a wide range in the measures used for different mental health difficulties, with many outcomes being sub-scales of larger measures of quality of life. This may have impacted on the reliability of certain measures and the validity that different measures were examining the same mental health domain.

Despite the prevalence of mental health difficulties in TGD people, many do not experience mental health difficulties. These participants may be less likely to experience any change in their mental health by post-intervention. For many of the studies, the pre-intervention mental health of participants compared to normative outcomes is not considered when measuring change. Subsequently this is not explored in this review but may be important when interpreting the effectiveness of interventions.

This review does not fully account for intersectional factors such as class, economic circumstances, ethnicity and age (amongst others) when exploring the research findings (Cho, Crenshaw, & McCall, 2013), as well as research into intersex gender dysphoria experiences and subsequent support. Intersectional factors may have a unique impact on an individual’s experience of healthcare and mental health beyond what is reported within the included studies (Bower-Brown & Zadeh, 2020; White & Fontenot, 2019) and should be included in study reporting.

Conclusion

Many TGD people experience significant mental health difficulties directly related to their gender dysphoria (Sohn & Bosinski, 2007). This review tentatively indicates that hormone and psychological interventions are effective at improving the mental health of TGD people, especially for depression outcomes. There is less conclusive evidence to suggest that surgical interventions improve mental health for TGD people, although further investigation of longer-term change is required. Future research should focus on examining other types of methods used to alleviate gender dysphoria and investigate the relationship between post-intervention mental health and gender minority stress factors.

Acknowledgements

Thank you to the service-users at the Northamptonshire Gender Identity Clinic for your feedback, ideas and thoughts around the research topic and search criteria.

Disclosure statement

The authors declare that they have no conflict of interest. It is important to acknowledge our own positioning and potential bias when synthesizing the research findings. LS is a cisgender white male who has a gender-affirmative positioning regarding TGD care, rights and support. Whilst care was made to select and synthesize research papers in an objective and rigorous scientific method, it is feasible that the author’s biases may have impacted on the analysis and presentation of research findings.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.

Appendix A.

Quality assessment of articles using EPHPP tool (EPHPP, 1998; Silverman et al., 2012)

Table

Author(s) Country	Selection bias	Study design	Confounders	Blinding	Data collection methods	Withdrawals and drop-outs	Intervention integrity	Analyses	Quality rating (1 = strong, 2 = moderate, 3 = weak)
Aldridge et al. (2020) UK	2	2	1	2	1	3	1	1	2
Briggs et al. (2018) USA	3	2	3	2	1	2	1	2	3
Budge et al. (2021) USA	3	2	1	2	1	1	1	2	2
Cardoso da Silva et al. (2016) Brazil	2	2	1	2	2	1	3	2	2
Chaovanalikit et al., 2022 Thailand	1	2	2	2	1	1	2	1	1
Colizzi et al. (2013) Italy	1	2	2	2	1	3	1	1	2
Colizzi et al. (2014) Italy	2	2	3	2	1	1	1	1	2
Colizzi et al. (2015) Italy	2	2	2	2	1	1	3	1	1
Costantino et al. (2013) Italy	3	2	3	2	3	1	1	3	3
Cotton and Wood (2019) UK	2	2	3	2	1	2	1	1	2
Drydakis (2016) UK	3	2	3	2	1	2	2	1	3
Fisher et al. (2016) Italy	2	2	3	2	1	1	2	2	2
Gümüşsoy et al. (2022) Türkiye	1	2	3	2	2	1	3	2	2
Heylens et al. (2014) Belgium	1	2	3	2	1	2	2	2	2
Isung et al. (2017) Sweden	2	2	3	2	1	1	2	1	2
Knutson et al. (2021) USA	2	2	3	2	1	2	2	1	2
Lindqvist et al. (2017) Sweden	2	2	3	2	3	3	2	3	3
Matthys et al. (2021) Netherlands, Belgium, Norway, Italy	3	2	1	2	1	3	2	1	3
Megeri and Khoosal (2007) UK	2	2	3	1	2	3	3	1	3
Mazzoli et al. (2022) Italy	2	2	2	2	1	3	2	2	2
Meier (2014) USA	3	2	2	2	2	2	3	3	2
Metzger and Boettger (2019) Switzerland, Germany	1	2	2	2	1	1	2	2	1
Naeimi et al. (2019) Iran	2	2	1	2	1	3	2	1	2
Nobili et al. (2020) UK	2	2	3	2	2	3	2	1	3
Oda and Kinoshita (2017) Japan	3	2	3	2	3	3	3	3	3
Papadopulos et al. (2017) Germany	1	2	3	2	1	1	2	1	2
Udeze et al. (2008) UK	2	2	2	2	2	3	3	2	2
van de Grift et al. (2016) Netherlands	2	2	1	2	1	2	2	2	1
van Dijk et al. (2019) Netherlands, Belgium, Italy	3	2	2	2	3	2	1	2	3

Shaded area is not included to calculate total quality assessment rating.