9,156
Views
8
CrossRef citations to date
0
Altmetric
Research Article

Sex- and gender-sensitive public health research: an analysis of research proposals in a research institute in the Netherlands

, , & ORCID Icon
Pages 109-119 | Received 10 Jan 2020, Accepted 03 Oct 2020, Published online: 18 Oct 2020

ABSTRACT

Taking sex and gender into account in public health research is essential to optimize methodological procedures, bridge the gender gap in public health knowledge, and advance gender equality. The aim of this study was to evaluate the current status of sex and gender considerations in public health research proposals in a Dutch research institute. We screened a random sample of 38 proposals submitted for review to the institute’s science committee between 2011 and 2016. Using the Canadian Institutes of Health Research’ Gender and Health Institute criteria for gender-sensitive research and qualitative content analysis, we assessed if, and how sex and gender were considered throughout the proposals (background, research aim, design, data collection, and analysis). Our results show that in general, both sex and gender were poorly considered. Gender was insufficiently taken into account throughout most proposals. When sex was mentioned in a proposal, its consideration was often inconsistent and fragmented. Finally, we identified common methodological pitfalls. We recommend that public health curricula and funding bodies increase their focus on implementing sex and gender in public health research, for instance through quality criteria, training programs for researchers and reviewers, and capacity building initiatives.

Introduction

Integrating sex and gender into health research is necessary to achieve gender equality in the social impact of science (Buitendijk and Maes Citation2015). However, researchers often fail to consider potential differences between men, women and other genders in their design, data collection methods, and analysis (EC Citation2015), which produces and maintains gender bias in research outcomes. In public health, variation in health outcomes between social groups can often be explained by biological and sociocultural aspects of being assigned male, female, or intersex at birth. It is therefore essential that public health researchers possess the awareness, knowledge, and skills to adequately consider sex and gender throughout the research process. We assessed the extent to which sex and gender considerations were integrated in study proposals submitted to the science committee of a public health research institute in the Netherlands.

Sex, gender, and public health research

Sex and gender are important concepts for public health, because they are associated with epidemiology, symptoms, and prognosis of diseases such as HIV/AIDS, cardiovascular disease, and psychiatric disorders (Maas et al. Citation2011; Pal and Hurria Citation2010; Shariat et al. Citation2010; ZonMW Citation2015). Sex refers to a biological classification of sexually-reproducing organisms, generally female, male, or intersex, according to functions that derive from chromosomes, hormones, or reproductive organs (Schiebinger et al., Citation2017). Gender refers to cultural attitudes and behaviors that shape feminine and masculine behaviors, products, technologies, environments, and knowledge. Important public health issues such as lifestyle and health behaviors are influenced by gender norms, beliefs, and roles (Schiebinger et al, Citation2017). Gender might interact with sex to increase or reduce individual and group risk, for instance in the case of HPV infection (Branković, Verdonk, and Klinge Citation2013). Incorporating sex and gender in public health research is therefore especially important: not accounting for sex and gender might cause specific risk factors or vulnerable groups to be overlooked, and introduce gender bias.

Examples of gender bias in (bio)medical research are ubiquitous. In animal studies, the significant majority of animals is male, or sex is not reported at all (Beery and Zucker Citation2011; Shah, McCormack, and Bradbury Citation2014). Women have been significantly underrepresented as participants in clinical trials (Beery and Zucker Citation2011; Curno et al. Citation2016; Melloni et al. Citation2010; Zucker and Beery Citation2010). Underrepresentation of women in pharmacological studies has resulted in more adverse drug reactions (Rademaker Citation2001; Raz and Miller Citation2012), and because coronary heart disease (CHD) has been studied predominantly in men, misdiagnosis and CHD mortality is higher in women (Bairy, Citation2014; ZonMW Citation2015). The adverse consequences of gender bias call for a thorough integration of sex and gender in all health-related research. For instance, the European Commission (EU) Horizon2020 research and innovation program prioritized the expansion of gender mainstreaming (GM) into research (EC Citation2016; Rees Citation2002). GM is understood as the systematic integration of gender equality into all systems and structures, policies, programs, processes and projects, into “ways of seeing and doing” (Mergaert and Minto Citation2015; Rees Citation2002). In EU-funded projects, gender should be mainstreamed to reduce gender imbalance in participation and decision-making at a health policy level, and integrated in research by systematically collecting and analyzing information about gender and gender inequality to enhance the scientific quality and societal relevance of new knowledge (EC Citation2016). Over the last decades, GM-perspectives have guided the development of implementation tools and approaches for the scientific research setting, for example the Toolkit ‘Gender in EU-funded Research’ (Consultants YWM Citation2011). Beyond Europe, other countries have also taken measures to incorporate sex and gender in their research policies and funding criteria, for instance Canada (CIHR Citation2015).

Unfortunately, the public health research community has been slow to mainstream gender, and despite recent advancements, it has yet to become standard procedure (e.g. Heidari et al. Citation2016; Johnson and Beaudet Citation2012). This might be due to a lack of awareness about sex and gender as a good health research practice. Often sex and gender are ignored or used interchangeably, gender and sex–gender interactions remain largely unaddressed (e.g. Nadeau and Lippel Citation2014; Regitz‐Zagrosek Citation2012; Springer, Hankivsky, and Bates Citation2012), and when data are stratified by sex it is often insufficient to explain variability regarding sex and gender (Mauvais-Jarvis et al. Citation2020). As a result, gender bias in public health research maintains, which results in lower societal relevance and inaccurate conclusions about generalizability (e.g. Heidari et al. Citation2016; Johnson et al. Citation2015). Moreover, gender inequalities in health care are reproduced or even created, disadvantaging millions of people, particularly women.

Aim of this study

We aimed to investigate the current status of sex and gender considerations in public health research by assessing if and how sex and gender appeared in public health research protocols. Analyzing research protocols is useful, as it provides insight in researchers’ sex and gender-related decisions early on in the research process, which might manifest in study results. Several studies investigated sex and gender sensitivity in the fields of biomedical, clinical, and public health research (e.g. Oertelt-Prigione et al. Citation2010). However, in-depth studies are scarce (Phillips and Hamberg Citation2016) and robust data are lacking.

We asked the following research question: What is the current status as regards integration of sex and gender in research proposals submitted for review by a Dutch public health research institute? Ultimately, this study aims to (a) increase researchers’ awareness, knowledge, and skills regarding sex and gender-sensitive research methods, (b) encourage granting agencies to incorporate sex and gender into their application criteria, and thus (c) support the process of gender mainstreaming into public health research.

Materials and methods

Research context

The study was carried out at a large public health research institute in The Netherlands, affiliated with a large teaching hospital. At the time of this study, the institute consisted of four vertically integrated scientific programs: Lifestyle, Overweight and Diabetes, Mental Health, Quality of Care, and Musculoskeletal Health. Individual programs are connected to intramural (e.g. obstetrics and gynecology) and extramural departments (e.g. public and occupational health) within the hospital. In 2016, the institute was externally evaluated according to the Standard Evaluation Protocol 2015–2021, with positive results (EMGO Citation2017). As such, the institute represented the highest level of quality in the Netherlands, and its practices likely reflected practices in other Dutch public health research institutes, and beyond.

We used proposals submitted to the institute’s science committee. The committee reviewed all protocols before they were submitted for funding, and had not incorporated sex and/or gender in their review criteria. According to the review procedure, two senior researchers were invited by the committee to review the proposals for relevance, feasibility, and quality, after which the committee decided whether or not the study can be incorporated in a research program. Proposals that received a negative review could be revised and resubmitted. Currently, reorganizations have taken place in the research school as a result of a merger with another teaching hospital and medical faculty, and procedures have changed. These changes are beyond the scope of this study, but to date, sex and gender have not been implemented in procedures and protocols.

Sampling and data collection

This project was approved by the director of the research institute. Our sample consisted of 45 research proposals, randomly selected from 350 research proposals submitted for review to the science committee between 2011 and 2016. The proposals were randomized by choosing every seventh proposal from the list of submitted proposals by date of submission. The only eligibility criterion was the involvement of human participants (e.g. patients, health-care staff, or experts on the proposals’ topic). After selection, the proposals’ authors were asked to provide written consent to include their proposal in a scientific article. Seven proposals were excluded from analysis (one because the authors did not consent, six because of nonresponse). The final sample consisted of 38 proposals (18 from ‘Quality of Care,’ 10 from ‘Mental Health,’ six from ‘Lifestyle, Overweight and Diabetes’ and four from ‘Musculoskeletal Health’). Sixteen proposals used quantitative methods, six used qualitative methods, and 16 used mixed methods. The study designs and research programs of the research proposals are presented in . If present, revised protocols were included in the sample. Three proposals were subsets of large EU projects and were included as one single research proposal.

Table 1. Anonymized characteristics of the research proposals

Evaluation: screening and content analysis

First, we used “The Integrating Sex & Gender Checklist – Partnership Development Grants for the Healthy & Productive Work Initiative,” developed by the Gender and Health Institute of the world-leading Canadian Institutes of Health Research (CIHR Citation2015), as a thematic framework to screen the proposals for sex and gender. In framework analysis, researchers use existing theories and research to classify their data. The method is suitable for generating policy and practice-oriented findings (Green and Thorogood Citation2014). We edited the checklist to fit the Dutch context. Edits were validated by two experts from the Gender and Health Alliance, a national collaboration of stakeholders who aim to increase gender-sensitivity of the Dutch healthcare system. The checklist is included in Appendix 1. Rephrasings or additions are indicated.

Per checklist item, proposals received one of three ratings. A proposal was rated “adequate” if all sub criteria were correctly addressed, or proper justifications were given why they were not addressed. A proposal was rated “some consideration” if not all sub criteria were addressed, or if they were incompletely addressed. A proposal was rated “no consideration” if none of the sub criteria were addressed. The first author rated all research proposals. The last author also evaluated ten randomly selected proposals. Differences were discussed until intersubjective agreement was achieved. Finally, reviewers’ comments were screened for any mentions of sex and gender. Full screening documents can be retrieved from the authors at request.

Results

General findings

Although levels of sex and gender integration varied across the proposals, none showed consistent full consideration of sex and gender. When sex was included the variable was operationalized categorically and dichotomously (male/female), without considering other relevant aspects of biological sex, such as body composition or hormone levels. Gender aspects such as gender beliefs, norms, or identity were not explicitly included. For instance, an intended study on health care for lesbian, gay, and bisexual elderly people did not mention sex or gender as factors to be accounted for in the research process. Furthermore, different proposal sections contained varying and inconsistent levels of sex and/or gender consideration. For example, some authors discussed sex differences in the literature review, but subsequently did not include sex in their analysis plan. Conversely, other authors include sex in their analysis but did not provide background for this choice. Below, we will discuss the level of consideration per proposal section.

Sex and gender per section of the research protocol

Vision, rationale, and added value

None of the proposals included sex or gender in their rationale and objective. In three cases, authors expressed the aim to achieve health benefits for a specific vulnerable group. However, mechanisms behind gender-specific vulnerabilities were not addressed. In other research proposals, authors did not specify if collection of sociodemographic data included sex, gender, or both. For example, in a proposal about quality of life of people with Korsakoff syndrome, authors mentioned the aim to measure ‘patient characteristics,’ but did not specify which characteristics.

Literature review

Often, proposals did not mention sex and gender in their background section. Sometimes, authors described sex-based differences in prevalence or other outcomes, but frequently, reflection on the mechanisms behind these differences, or commentary on the gaps in knowledge regarding these mechanisms, was lacking. Sometimes, potentially gendered phenomena such as social support and occupation were mentioned but not discussed from a sex or gender perspective, as well as factors known to often intersect with sex and/or gender (e.g. socioeconomic status, age, and ethnicity). The sample included two proposals for a single-sex study. One (on solitary confinement in prisons) focussed on men only, but did not explain this choice. A proposal about a lifestyle intervention effectiveness study measuring C-peptide, cortisol, and glucose in various intervention groups included only males to “avoid disturbances in outcomes by hormonal influences of the menstrual cycle in females.” No evidence for “disturbances in outcomes” was given, nor were options for accounting for cycle-dependent hormone levels explored.

Research question

Objectives or research questions rarely involved sex or gender. Some proposals included a sub question about the relationship between sex and other unspecified demographic factors (described as “demographic factors” or “patient characteristics”) and an outcome variable. A proposal about a lifestyle intervention in schoolchildren left out gender but did include sex and SES in its objective as a factor in intervention success; however, the authors did not proceed to propose investigating interaction effects. In a proposal on retirement, authors aimed to examine “socio-cultural context” as a predictor for retirement age. Whether this included gender aspects, remained unclear.

Study design & methods

In most proposals, sex and/or gender were not mentioned in study design and methodology. Most research proposals anticipated convenience sampling, which is more likely to result in a sampling bias regarding diversity. Some researchers expressed a desire for maximum variation in participant characteristics. For instance, a study about involuntary treatment in mental health care stated that the study aimed for diversity among participants and professionals involved regarding sex, age, and experience with involuntary treatment. No further explanation or sampling procedure was given, and assessing the feasibility to achieve diversity was therefore difficult. Regarding materials used, e.g. questionnaires, none of the research proposals specified whether materials were validated for both sexes and/or gender-diverse people.

Analysis & reporting

Sex and gender were not included in the anticipated statistical analysis in most research proposals. For qualitative studies, it was difficult to determine if sex and/or gender were going to be included in the analysis and reporting, and mostly this remained unclear. In some quantitative studies, sex was incorporated as a potential confounder and not as a predictor. Thus, researchers primarily intended to correct for sex rather than explore it at as a predictive or moderating variable, and did not plan to carry out stratified analyses. In addition, researchers did not mention an aim to explore gender aspects, or interactions between gender and other sociodemographic variables. Included power calculations aimed at exploring differences between the intervention and control group regarding the outcome variable, but never accounted for sex differences. Power calculations were done for primary outcome measures only, and sex was never a primary variable.

Knowledge translation strategies & ethics

Knowledge translation strategies were not always included in the research proposals. If they were, reflection on implementing study findings into a sex and gender-diverse society was lacking, such as considering using different channels to reach men and women when relevant. Diversity in general was lacking in knowledge dissemination plans. Currently, the Medical Ethics Committee, to our knowledge, does not use criteria regarding sex and gender or diversity in general in its assessment and advice. This was reflected in the research proposals. Ethical issues never referred to sex, gender, and/or diversity.

Reviewers’ feedback

Sex and gender were rarely mentioned by reviewers of the science committee. In one proposal on medicine adherence, a reviewer of the initial proposal comments on the fact that relevant personal and socio-cultural factors reflected on in the literature review were not included in the questionnaire, which was then improved in the revised version of the proposal. This is one example of how reviewers can influence sex and gender integration. Most reviewers positively rated the submitted research proposals, implying that reviewers did not consider sex and/or gender in the proposal as a quality criterion. In some cases, reviewers’ feedback contained advice to account for sex or other demographic factors as confounding variables. This way, the predictive and analytic value of sex/gender was lost.

Discussion

We screened a random sample of public health research proposals for sex and gender considerations, using CIHR’s criteria for sex and gender in health research (CIHR Citation2015). We found that sex and gender variables were operationalized as binaries and the concepts were used interchangeably (Hammarström and Annandale Citation2012; Hammarström et al. Citation2014; Springer, Hankivsky, and Bates Citation2012). Overall, sex and gender were absent from or poorly integrated in the research proposals, and when they were, integration was inconsistent.

In line with the findings of Phillips and Hamberg (Citation2016), we found that sex and gender ‘disappeared’ from the proposals in several ways. In many cases, the concepts appeared in the background section, for instance as scientific evidence for sex and gender differences, but did not return as a consideration for design or analysis. In other cases, sex was used as a confounder, which eliminates its predictive value. Other, more subtle mechanisms of sex and gender disappearance include the use of ‘patients’ or ‘participants’ instead of ‘men’ and ‘women,’ or conducting single-sex studies without adequate justification, including visible considerations regarding the consequences of this choice for generalizability and societal relevance of study outcomes.

Such ‘disappearing acts’ can be understood as a process of neutralization, one in which sex and gender are obscured instead of highlighted as an opportunity for innovation. The ‘neutralization’ of sex and gender in our sample can be explained in several ways. First, researchers may be unaware of the importance of sex and gender considerations, and lack research skills to incorporate them. Second, proposals are written in line with criteria of funding bodies and research institutions. Finally, researchers may actively resist incorporating sex and gender. The dominant ideology in health research and other fields of biomedical science is objectivity, and it is assumed that the product of this ideology, scientific knowledge, is inherently value-free and unbiased. Moreover, scientific inquiry is traditionally heavily invested in valuing detachment, research ‘untainted’ by societal factors (Ali and Sichel Citation2018; Harding Citation1987). Considering the impact of sex and gender, however, requires researchers to explicitly account for difference and social factors. Paradoxically then, the conscious or subconscious strive for value-free and objective knowledge might replicate sex and gender bias when it ignores aspects of diversity such as sex, gender, ethnicity and class, further reproducing existing health inequalities.

Strengths and limitations

The study has several strengths. We used the best available criteria (CIHR Citation2015) to screen randomly selected research proposals representative for public health research practice in the Netherlands, and examined all sections of the proposals, including science committee reviewers’ comments. A limitation is that the CIHR criteria might have been less applicable to qualitative studies, especially the ‘Analysis and reporting’ items. Furthermore, using criteria or ‘checkboxes’ to screen proposals limits an in-depth critical analysis, and hinders insight in the feasibility of sex and gender considerations, as well as in the extent to which sex and gender are cohesively integrated. Finally, the social position of the authors of this paper may have influenced how data were collected and analyzed. All authors are white, Dutch, female-identified and female-bodied people, they are sex and gender researchers and/or health-care professionals. Gender – and, possibly, other aspects of diversity as well – might influence how gender-sensitive methods in research are applied and results are interpreted (e.g. Nielsen et al. Citation2017). Although we studied gender bias, other types of bias might have been present in the proposals, e.g. class and racial bias, which requires an intersectionality-based analysis (Verdonk et al. CitationForthcoming)

Implications for research and practice

We propose several recommendations to improve sex and gender-responsiveness of public health research. First, implementation of sex and gender criteria in application forms and review guidelines at local public health research institutes and medical ethics committees could contribute to more sex and gender-sensitive research practices (see also Johnson et al. Citation2014). However, criteria alone are not sufficient. Gender mainstreaming should take place at all levels of research policy-making (Schofield Citation2012).

Second, researchers should become aware of the relevance of sex and gender, for instance through educating researchers, reviewers, and committee members on accounting for sex, gender, and other aspects of diversity in their work. Such education could consist of singular tools, such as the online CIHR modules (Institute of Gender and Health, part of CIHR Citation2016), or more in-depth local training programs. The CIHR advices to appoint a ‘Sex and Gender Champion’ in research teams (CIHR Citation2015). In addition, further implementation of sex, gender, and diversity in health profession curricula in The Netherlands can add to increased awareness, knowledge, and skills among future doctors and health researchers (e.g. Verdonk, Muntinga, and Croiset Citation2016).

Third, considering sex and gender and their intersections with, e.g. age, class, cultural diversity, and sexuality (intersectionality) is important to predict public health outcomes and determine individual needs (Hankivsky Citation2012; Bauer, Citation2014; Schiebinger et al, Citation2017). Such an intersectionality-based sex and gender analysis might help expose vulnerable groups and design tailored interventions.

Moreover, funding bodies have a responsibility to promote gender mainstreaming through grant programs. In 2015, the Netherlands Organization for Health and Care Research launched the four-year ‘Gender and Health’ research program, which focussed on reducing health inequalities between men and women (ZonMW Citation2015). Such programs encourage sex and gender research in and outside the Netherlands. Journals should aim to publish gender-sensitive research. An overview of journals with explicit sex and gender policies is available in the Gendered Innovations project (Schiebinger et al, Citation2017). Finally, to build capacity for sex and gender-sensitive public health research, gender mainstreaming efforts should be take place at multiple levels (local, national, and international) and locations (research institutes, universities, funding agencies, and journals) at the same time.

Future research might compare results of sex and gender mainstreaming across health research fields, especially because fields such as cardiology seem ahead. Such research can specify points of improvement for each field, identify good practices and encourage the exchange of such practices between fields. Furthermore, positive results of gender mainstreaming efforts could clarify how sex and gender methods offer innovation, such as the examples provided in the Gendered Innovations project (Schiebinger et al, Citation2017).

Conclusions and recommendations

Overall, this study provides insight in the lack of sex and gender considerations in public health research proposals in The Netherlands, and results are in line with other studies. Given that the institute is a highly respected research institute with good evaluations, we think it is fair to assume that our findings generalize to other public health research institutes as well (External Evaluation Committee Report, 2016). To improve societal and scientific relevance of public health findings, (a) awareness must be raised across the research community on how sex and gender can lead to innovation, (b) researchers must be trained on integrating sex and gender in their research, and (c) gender and diversity must be mainstreamed into institutions’ research policies, in policies of funding agencies, and journal guidelines. Mainstreaming sex, gender, and other aspects of diversity into public health research is essential for a beneficial impact of science for all women, men, and gender-diverse people equally.

Data availability statement

An anonymized version of the findings per section of all research proposals is available at request.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The study was not externally funded.

References

  • Ali, A., and C. E. Sichel. 2018. Humanizing the scientific method. In The crisis of connection. Roots, consequences and solutions, ed. A. Ali, C. Gilligan, and P. Noguera, 211–27. New York: New York University Press.
  • Bairey Merz, C. N. 2014. Sex, death, and the diagnosis gap. Circulation 130 (9):740–42.
  • Bauer, G. R. 2014. Incorporating intersectionality theory into population health research methodology: Challenges and the potential to advance health equity. Social Science & Medicine 110:10–17.
  • Beery, A. K., and I. Zucker. 2011. Sex bias in neuroscience and biomedical research. Neuroscience & Biobehavioral Reviews 35 (3):565–72.
  • Branković, I., P. Verdonk, and I. Klinge. 2013. Applying a gender lens on human papillomavirus infection: Cervical cancer screening, HPV DNA testing, and HPV vaccination. International Journal for Equity in Health 12:14. doi:10.1186/1475-9276-12-14.
  • Buitendijk, S., and K. Maes (2015). Gendered research and innovation: Integrating sex and gender analysis into the research process. League of European Research Universities. Accessed March 08, 2017 http://www.leru.org/files/publications/LERU_AP18_Gendered_research_and_innovation_final.pdf
  • CIHR (2015). Integrating sex & gender checklist – partnership development grants for the healthy & productive work initiative. Ottowa: Canadian Institute for Health Research. Accessed February 28, 2017: http://www.cihr-irsc.gc.ca/e/49336.html
  • Consultants YWM. 2011. Toolkit Gender in EU-funded Research. Brussels: European Commission, Directorate-General for Research and Innovation.
  • Curno, M. J., S. Rossi, I. Hodges-Mameletzis, R. Johnston, M. A. Price, and S. Heidari. 2016. A systematic review of the inclusion (or exclusion) of women in HIV research: From clinical studies of antiretrovirals and vaccines to cure strategies. JAIDS Journal of Acquired Immune Deficiency Syndromes 71 (2):181–88.
  • EC (2015). Horizon 2020 monitoring report 2015 (B-1094). Brussels: European Commission.
  • EC (2016). Promoting gender equality in research and innovation – horizon 2020. Accessed March 08, 2017: https://ec.europa.eu/programs/horizon2020/en/h2020-section/promoting-gender-equality-research-and-innovation
  • EMGO (2017). Extensive assessment of a research proposal (n.d.). Amsterdam: VU University Medical Centre, EMGO Institute for Health and Care Research. Accessed March 14, 2017 www.emgo.nl/files/1860
  • Green, J., and N. Thorogood. 2014. Part III: Managing and analysing data. Qualitative methods for health research. London: Sage.
  • Hammarström, A., and E. Annandale. 2012. A conceptual muddle: An empirical analysis of the use of ‘sex’ and ‘gender’ in ‘gender-specific medicine’ journals. PLoS One 7 (4):e34193.
  • Hammarström, A., K. Johansson, E. Annandale, C. Ahlgren, L. Aléx, M. Christianson, … P. E. Gustafsson. 2014. Central gender theoretical concepts in health research: The state of the art. Journal of Epidemiology and Community Health 68 (2):185–90.
  • Hankivsky, O. 2012. Women’s health, men’s health, and gender and health: Implications of intersectionality. Social Science & Medicine 74 (11):1712–20.
  • Harding, S. G., Ed. 1987. Feminism and methodology: Social science issues. Bloomington: Indiana University Press.
  • Heidari, S., T. F. Babor, P. De Castro, S. Tort, and M. Curno. 2016. Sex and gender equity in research: Rationale for the SAGER guidelines and recommended use. Research Integrity and Peer Review 1 (1):2.
  • Institute of Gender and Health, part of the Canadian Institutes of Health Research (2016). Online training modules: integrating sex & gender in health research. Accessed March 8, 2017http://www.cihr-irsc.gc.ca/e/49347.html
  • Johnson, J., Z. Sharman, B. Vissandjee, and D. E. Stewart. 2014. Does a change in health research funding policy related to the integration of sex and gender have an impact? PLoS One 9 (6):e99900.
  • Johnson, J. L., and A. Beaudet. 2012. Sex and gender reporting in health research: Why Canada should be a leader. Canadian Journal of Public Health 104 (1):80–81.
  • Johnson, P. A., T. Fitzgerald, A. Salganicoff, S. F. Wood, J. M. Goldstein, and Y. L. Colson (2015). Sex-specific medical research: why women’s health can’t wait. A Report of the Mary Horrigan Connors Center for Women’s Health & Gender Biology at Brigham and Women’s Hospital. Retrieved from: http://www.brighamandwomens.org/Departments_and_Services/womenshealth/ConnorsCenter/Policy/ConnorsReportFINAL.pdf.
  • Maas, A. H., Y. T. van der Schouw, V. Regitz-Zagrosek, E. Swahn, Y. E. Appelman, G. Pasterkamp, … K. Eizema. 2011. Red alert for women’s heart: The urgent need for more research and knowledge on cardiovascular disease in women. European Heart Journal 32 (11):1362–68.
  • Mauvais-Jarvis, F., N. B. Merz, P. J. Barnes, R. D. Brinton, J. J. Carrero, D. L. DeMeo, … A. Lonardo. 2020. Sex and gender: Modifiers of health, disease, and medicine. The Lancet 396 (10250):565–82.
  • Melloni, C., J. S. Berger, T. Y. Wang, F. Gunes, A. Stebbins, K. S. Pieper, … L. K. Newby. 2010. Representation of women in randomized clinical trials of cardiovascular disease prevention. Circulation: Cardiovascular Quality and Outcomes 3 (2):135–42.
  • Mergaert, L., and R. Minto. 2015. Ex Ante and Ex post evaluations: Two sides of the same coin?: The case of gender mainstreaming in EU research policy. European Journal Risk Regulation 6 (1):47–56.
  • Nadeau, G., and K. Lippel. 2014. From individual coping strategies to illness codification: The reflection of gender in social science research on Multiple Chemical Sensitivities (MCS). International Journal for Equity in Health 13 (1):78.
  • Nielsen, M. W., J. P. Andersen, L. Schiebinger, and J. W. Schneider. 2017. One and a half million medical papers reveal a link between author gender and attention to gender and sex analysis. Nature Human Behaviour 1 (11):791.
  • Oertelt-Prigione, S., R. Parol, S. Krohn, R. Preißner, and V. Regitz-Zagrosek. 2010. Analysis of sex and gender-specific research reveals a common increase in publications and marked differences between disciplines. BMC Medicine 8 (1):70.
  • Pal, S. K., and A. Hurria. 2010. Impact of age, sex, and comorbidity on cancer therapy and disease progression. Journal of Clinical Oncology 28 (26):4086–93.
  • Phillips, S. P., and K. Hamberg. 2016. Doubly blind: A systematic review of gender in randomised controlled trials. Global Health Action 9:29597.
  • Rademaker, M. 2001. Do women have more adverse drug reactions? American Journal of Clinical Dermatology 2 (6):349–51.
  • Raz, L., and V. M. Miller (2012). Handbook of experimental pharmacology. In Considerations of sex and gender differences in preclinical and clinical trials. Accessed May 2017 https://link.springer.com/chapter/10.1007%2F978-3-642-30726-3_7.
  • Rees, T. 2002. Gender mainstreaming: Misappropriated and misunderstood. Stockholm: Department of Sociology, University of Sweden.
  • Regitz‐Zagrosek, V. 2012. Sex and gender differences in health: Science & society series on sex and science. EMBO Reports 13 (7):596–603.
  • Schiebinger, L., I. Klinge, H. Y. Paik, I. Sánchez de Madariaga, M. Schraudner, and M. Stefanick. ed. 2011-2017. Gendered innovations in science, health & medicine, engineering, and environment. genderedinnovations.stanford.edu
  • Schofield, T. 2012. Gender, health, research, and public policy. In Gender, sex & health research, ed. J. L. Oliffe and L. Greaves, 203–2015. Thousand oaks, California: SAGE Publications, Inc.
  • Shah, K., C. E. McCormack, and N. A. Bradbury. 2014. Do you know the sex of your cells? American Journal of Physiological Cell Physiology 306:C3–C18.
  • Shariat, S. F., J. P. Sfakianos, M. J. Droller, P. I. Karakiewicz, S. Meryn, and B. H. Bochner. 2010. The effect of age and gender on bladder cancer: A critical review of the literature. BJU International 105 (3):300–08.
  • Springer, K. W., O. Hankivsky, and L. M. Bates. 2012. Gender and health: Relational, intersectional, and biosocial approaches. Social Science & Medicine 74:1661–66.
  • Verdonk, P., M. Muntinga, and G. Croiset. 2016. Gender en diversiteit in het geneeskundeonderwijs. Tijdschrift Voor Genderstudies 19 (2):225–39.
  • Verdonk, P., M. Ridder, A. Kent, and M. Muntinga Building sex and gender research capacity in public health. Forthcoming.
  • ZonMW. 2015. Gender and health knowledge agenda. Den Haag: ZonMW, Alliantie Gender en Gezondheid.
  • Zucker, I., and A. K. Beery. 2010. Males still dominate animal studies. Nature 465 (7299):690–690.

Appendix 1

Checklist Integrating Sex and Gender in Public Health Research

1. Vision, rationale and added value

  • Clarity of the vision regarding the analysis and expected results/outcomes as they relate to sex and/or gender

  • Evidence of explicit consideration being given to achieving equitable health impacts across diverse patient/population sub-groups

2. Literature review

  • Literature review pointing to the extent to which past research has taken sex or gender into account (added)

  • Clear articulation of any known sex and/or gender differences in the epidemiology, risk factors, conditions, diseases or outcomes (rephrased)

  • Attention is given to possible diversity within groups under study (males/females/age groups; added)

  • Literature review describing known mechanisms explaining sex or gender differences, or lack thereof, in research area under study (added)

  • Key social determinants of gender, such as ethnicity, income, occupation, and social roles should be considered (rephrased)

3. Research question

  • Clear articulation of the type of research question being considered with respect to sex and/or gender, including:

    • Identifying sex and/or gender differences;

    • Explaining sex and/or gender differences;

    • Establishing sex and/or gender similarities in the mechanism under study;

    • Studying sex/gender as a confounder or interaction variable while testing the main study hypothesis (rephrased)

  • When it is stated that sex and/or gender are not relevant for the research question, valid arguments and/or evidence is provided (added)

4. Study Design and Methods

  • Inclusion and exclusion criteria that consider sex/gender and diverse populations of men and women (rephrased)

  • Appropriate design to capture between-group and within-group differences with respect to sex and/or gender with other relevant factors. Possibility to collect data disaggregated by gender (rephrased)

  • Scientifically sound justification for proposing a single-sex study, if applicable

  • Description of the recruitment strategies to acquire the required sample size of men and women

  • Choice of outcome measures or validation tests that are not gender biased and that are sensitive and responsive to sex/gender issues

  • Description of the data collection tools/use of administrative datasets with respect to capturing sex-related and gender-related variables of interest

5. Analysis & Reporting

  • Description of the data analysis plan (sex-disaggregated or stratified analyses, pathway modeling, use of sex and gender variables as confounders or in interaction terms, if applicable)

  • Sample size calculations to show adequate power for testing hypotheses with respect to sex/gender differences

  • Inclusion of a statement that negative findings with respect to sex/gender will be reported

  • Conclusions are linked to specific characteristics of the research population (added)

6. Knowledge translation strategies

  • Description of how the knowledge translation strategies intend to maximize uptake by men, women, boys, girls or gender-diverse people (e.g. will the content, messages, or products vary by sex or gender?)

8. Ethics (added)

  • Does the study design account for the relevant ethical issues that might have particular significance with respect to sex and/or gender?