Coding Responses to an Open-ended Gender Measure in a New Zealand National Sample

ABSTRACT

In light of the methodological and ethical issues associated with using a male/female tick box to collect gender data, researchers are increasingly questioning how to measure gender inclusively in survey research. Open-ended measures afford the greatest flexibility, though whether they are practical for large-scale surveys has yet to be tested. Here, we systematically assess the feasibility of open-ended gender measures drawing on a New Zealand national probability sample (New Zealand Attitudes and Values Study, N = 15,758). We asked participants “What is your gender?” as an open-ended measure of gender, and developed a simple, cost-effective coding scheme for coding qualitative gender data. Results indicate that very few participants (n = 15) self-identified as transgender, or outside of the male/female gender binary. Moreover, we find no evidence that implementation of the open-ended measure contributes to non-response rates or panel attrition. Taken together, these results demonstrate that large-scale surveys can feasibly implement inclusive measures of gender as an alternative to binary categorical measures. Because the single-measure approach likely underestimates the number of transgender participants, however, researchers interested in identifying all participants whose gender differs from their assigned sex should utilize two-step methods, which assess gender as well as assigned sex at birth.

Acknowledgements

We are indebted to our independent coder, Kealagh Robinson, for her helpful comments and time spent testing the usability of our coding scheme. We would also like to thank Jennifer Shields for her feedback on an earlier version of this manuscript. Finally, sincere thanks to Jasmine Khorasanee, Logan Hamley, Pearl Hindley, Vidushi Challapali, and Nikolas Rusten, who manually checked over 30,000 scanned surveys to ensure the accuracy of our missing data coding.

Notes

¹ It is worth noting that, although sex and gender researchers typically use the terms male/female to refer to sex and man/woman to refer to gender, this distinction is rarely made outside of the field of sex and gender; laypeople commonly use “male” and “female” to describe their gender, and researchers routinely use male/female as response options to questions about gender (Westbrook & Saperstein, 2015). Indeed, we have previously used a male/female tick box to measure gender in our own research. Although this conflation of sex and gender is incorrect, we follow the use of male/female to refer to gender at some points in this manuscript for pragmatic purposes: (1) to describe our own past research practices, and (2) to reflect common use of the terms male/female among our participants and the vast majority of researchers.

² Following the Health Resources and Services Administration’s 2016 announcement that all federally funded health centers must submit gender identity to their annual Uniform Data System report (Health Resources and Services Administration, 2016), use of the two-step method in healthcare systems is becoming more common in the United States (see, for example, Tordoff, Morgan, Dombrowski, Golden, & Barbee, 2019). Despite this, Grasso, McDowell, Goldhammer, and Keuroghlian (2018) noted that “most healthcare organizations have yet to implement systematic data collection due to concerns about making staff and patients uncomfortable, the inability of electronic health record (EHR) platforms to accommodate SO/GI information, and inadequate dissemination of best practices” (p. 66).

³ A notable exception is Jaroszewski et al.’s (2018) paper on coding qualitative gender data collected from two online social communities: Tumblr, a social media blogging site, and Fantasy Football, an online game. Given the unique characteristics of these samples, however, Jaroszewski et al identified a high proportion of aggressive, absurd, or humorous responses. As such, their coding scheme arguably cannot inform the process of coding gender data collected in the general population.

⁴ Totals do not add to 100% as participants could identify with more than one ethnic group.

⁵ Raw responses shown in quotation marks have not been altered to correct participants’ typographical errors.

⁶ We expect that other general population surveys would also result in a relatively small number of unique responses; however, research among LGBT or gender diverse populations is likely to result in higher numbers of unique responses to open-ended gender measures. As such, additional coding time should be set aside when using open-ended gender measures with these groups.

⁷ “The embodiment of both feminine and masculine spirits within one person” (Balsam, Huang, Fieland, Simoni, & Walters, 2004, p. 288).

⁸ As well as searching academic databases using the keywords “malware” and “gender,”, we searched popular online platforms such as tumblr.com and gender.wikia.com for tags of these terms, as new identity terms often appear on such websites prior to their use in academic literature.

⁹ This assumes the coder has some familiarity with gender identity terms. Researchers unfamiliar with gender diversity or gender identities outside of the male/female binary may need to consult a glossary of gender related terms. We recommend Gender Minorities Aotearoa’s (2017) guide, or consulting gender.wikia.com.

Additional information

Funding

The New Zealand Attitudes and Values Study is supported by a grant from the Templeton Religion Trust (TRT0196). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the article.