How Have Other Journals Compared to “The Top Seven” Journals in Family Social Science with Respect to LGBT-Related Research and Reviews? A Comment on “A Content Analysis of LGBT Research in Top Family Journals 2000–2015”? An Editorial Analysis

ABSTRACT

Van Eeden-Moorefield, Few-Demo, Benson, Bible, and Lummer (2018) recently reviewed the content of seven family social science journals with respect to theory and methods used in 153 articles featuring LGBT research or reviews of LGBT research. They concluded that less than three percent of 6,832 articles published in those journals between 2000 and 2015 were LGBT-related and that most had not used explicit theory and had used purposive cross-sectional samples consisting mostly of White and middle-class participants. During the tenure of the present editor, between 2010 and 2017, Marriage & Family Review featured 14 research reports, reviews, or commentary on LGBT-related issues out of 250 articles, not counting brief editorials or biographies between 2010 and 2017, an odds ratio of over 2.0 compared to the other “top-tier” journals. Likewise, the Journal of Family Theory & Review published at least 15 articles, mainly reviews or commentaries, on LGBT-related issues between 2009 and 2017. Articles in Marriage & Family Review and Journal of Family Theory & Review were more likely to be reviews of the literature, or feature explicit theory, to be qualitative in methodology, or to involve interviews or surveys, while other characteristics were similar to those in the other seven journals. Thus, even if some mainstream family journals did not publish often on LGBT-related issues, other important family science journals have done so. Furthermore, there are numerous scholarly journals that welcome LGBT-related research and reviews almost exclusively.

KEYWORDS:

• content analysis
• families
• LGBT
• methods
• theory

Introduction

Van Eeden-Moorefield, Few-Demo, Benson, Bible, and Lummer (2018) recently reported a detailed content analysis of LGBT research in seven top-tier family journals as published between 2000 and 2015. In their introduction, they highlighted the importance of theory, methodological rigor, and inclusivity in advancing the field of family social science, among others. With their assessment, we completely agree that theory, methodological rigor, and inclusivity are indeed important for scientific progress, in any field. They concluded that there were limitations in all three areas in the way that LGBT-related research had been published in the past fifteen years in the seven family journals that were part of their content analysis. Their assessment of theoretical limitations in LGBT-related research aligned with the assessment by Farr, Tasker, and Goldberg (2016), even though van Eeden-Moorefield et al. (2018) did not cite Farr et al. (2016) in their article.

Research questions

As editor-in-chief (senior author) for Marriage & Family Review, which was not included in their seven journals, I questioned how a content analysis of Marriage & Family Review would compare to the content analysis that van Eeden-Moorefield et al. (2018) had done. I also thought it useful to examine a similar journal, the Journal of Family Theory & Review, that I presumed had also published a number of reports on LGBT-related issues. Review of van Eeden-Moorefield et al.’s (2018) article suggested that comparisons could be made because of the clarity of their coding schemes and the completeness with which they had reported their results. Specific questions might include: Are more LGBT-related articles being published in Marriage & Family Review (MFR) or the Journal of Family Theory & Review (JFTR) than in other family journals? Have more LGBT-related articles been published in MFR since 2010 when the current editor assumed that role than in years prior (2000−2009)? Are the methodologies used in MFR or JFTR articles different than those being used in other family journals?

Methodology

Data from van Eeden-Moorefield et al. (2018) were extracted from their report with respect to the number of LGBT-related articles published per year, the explicit versus implicit use of theory, which theories had been used, the various types of methods used, the sample characteristics, and whether key variables such as outness, discrimination, or HIV/AIDS status had been assessed. Articles from MFR between the years 2000 and 2017 were searched for LGBT-related content and each article with such content was classified as either a review/commentary on a review or a research article. The same approach was used for JFTR articles between 2009 and 2017. Research articles were further classified using the same criteria used by van Eeden-Moorefield et al. (2018). An SPSS database was created to represent the number of LGBT-related articles published per year in both the seven journals assessed by van Eeden-Moorefield et al. (2018) and in MFR and JFTR, the former split between the time periods 2000−2009 and 2010–2017. It should be noted that many scholarly journals, including MFR and JFTR were not listed in the Reuters/Thomson journal rankings used by van Eeden-Moorefield et al. (2018) possibly because some publishers do not want to pay the fees required by Reuters/Thomson for listing in their rankings. Thus, selections of journals from their listings are not necessarily the “top” journals in the field, but rather the “top” journals in that particular listing.

Results

Relative percentages of LGBT-related articles

Between 2000 and 2009, MFR featured publication of 306 articles, excluding biographies and brief editorials, with one article published that contained information on same-sex marriage in Canada (Schlesinger, 2007). Between 2010 and 2017, MFR featured several reviews of LGBT-related literature or commentaries on such reviews, with reviews focusing on family interventions (Rosik, 2014; Whitton & Buzzella, 2012), research literature (Allen, 2015; Cameron & Cameron, 2012; Cameron, Cameron, & Proctor, 2017; Kulik, 2011; Schumm, 2010, 2012a, b), and commentary on research literature (Schumm, Crawford, Childs, Ateeq, Koochel, & Alshalan, 2017) as well as five empirical articles (Bong, 2011; Edwards, 2011; Negy, Pearte, & Lacefield, 2013; Sutphin, 2010), of which one concerned family life education program evaluation (Skogrand, Mendez, & Higginbotham, 2013) out of 250 published articles. The difference in rates between the early and later periods of MFR was statistically significant, with an odds ratio = 18.1 (95% CI, 2.36 to 138.6, p = .005), despite the small number of LGBT-related articles published in either time frame.

Between 2010 and 2015, van Eeden-Moorefield et al. (2018) reported 86 articles published on LGBT-related issues out of 3,206 articles, a rate (2.68%) that was significantly lower than the rate in MFR between 2010 and 2017 (14/250, 5.6%), with an odds ratio of 2.15 (95% CI, 1.21 to 3.84, p = .01). We also observed that JFTR published at least 15 articles (of 208) between 2009 and 2017 that dealt with LGBT-family research issues, a rate of 7.2%, higher than MFR or the top seven family journals; those articles included 14 reviews or commentaries (Allen, 2016; Berkowitz, 2009; Cao, Mills-Koonce, Wood, & Fine, 2016; Chrisler, 2017; Cohen, 2011; Doucet & Lee, 2014; Few-Demo, Humble, Curran, & Lloyd, 2016; Fuller, 2017; Glennon, 2012; Goldberg, 2013; Lauer & Yodanis, 2010; McGuire, Catalpa, Lacey, & Kuvalanka, 2016; McGuire, Kuvalanka, Catalpa, & Toomey, 2016; Widiss, 2016; as well as one empirical paper (Connidis, 2012), not counting editorials for special issues (e.g., Oswald, 2016). In addition, JFTR also published four articles that tangentially mentioned LGBT issues (Few-Demo, 2014; Galvin & Braithwaite, 2014; Sharp & Weaver, 2015; Sprey, 2009) and in the first issue of 2018 published two more articles that were clearly on LGBT issues (Allen & Mendez, 2018; Holman, 2018).

Research characteristics

Van Eeden-Moorefield et al. (2018) reported that 44.4% of their articles used an explicit theory while another 9.8% used an implicit theory, which compares to 80% and 20% (Bong), respectively, for the five MFR empirical reports and 100% for the one empirical JFTR report (Connidis, 2012). By comparison, Farr, Tasker, and Goldberg (2016) in their review of 30 highly cited papers on LGBT parenting found that only a third explicitly mentioned specific theories. Among the MFR articles, some of the theories used overlapped (multicultural feminism, minority stress, risk and resilience) while others were unique to the five articles (social support, social exchange, social demographic transition theory). All of the JTFR articles used explicit theories.

While van Eeden-Moorefield et al. (2018) found fewer than 15% of their articles referred to the critical variables of outness, stigma/discrimination, or HIV/AIDS, each of the five papers in Marriage & Family Review referred to at least one of those concepts (outness, Bong, Skogrand et al., Sutphin; stigma/discrimination, Bong, Edwards; HIV/AIDS, Edwards). All of the JFTR articles referred to those concepts directly or indirectly. Van Eeden-Moorefield et al. (2018) found 42% and 34% of their articles used quantitative or qualitative methodologies, respectively, which compared to 40% (Negy, Sutphin) and 60% (Bong, Edwards, Skogrand et al.) for the five MFR articles and 100% qualitative for the one JFTR report (Connidis, 2012). While van Eeden-Moorefield et al. (2018) found 88% of their articles to be cross-sectional, 100% of the five papers in MFR used that design. While nearly 20% of the articles considered by van Eeden-Moorefield et al. (2018) were reviews of the LGBT literature, nearly 67% of the articles in MFR were reviews or commentaries on reviews, a result that was statistically significant, odds ratio = 8.20 (95% CI, 2.61 to 25.78, p < .001) as were 93.3% of the articles in JFTR. Interviews and surveys represented 71% of the articles in van Eeden-Moorefield et al. (2018) compared to 100% of the six articles in MFR and JFTR. In person and online survey methods were 41.5% and 17.1% of the empirical articles considered by van Eeden-Moorefield et al. (2018), respectively, compared to 60% (Bong, Edwards, Skogrand et al.) and 40% (Negy, Sutphin) for MFR and 100% in JFTR. Van Eeden-Moorefield et al. (2018) found that 30% of their articles were comparative, while only 20% (Negy) were for MFR; Skogrand et al. had comparative data available but did not use it in their article. The one JFTR article did not use comparative data as both individuals studied were gay men.

Van-Eeden-Moorefield et al. (2018) found that nearly 93% of their articles used non-random sampling methods, as did all six of the MFR and JFTR articles; in terms of reporting of SES diversity, the former found that 46% of the articles did not report SES, compared to 60% of the MFR and none for the only JFTR report. Van Eeden-Moorefield et al. (2018) found that 18% of their articles reported nothing on the racial or ethnic diversity of participants; similar patterns were found for both sets of MFR papers (40%, Edwards, Skogrand et al. did not report race) and JFTR papers (none reported, Connidis, 2012). However, racial diversity was more evident in the MFR articles, with two using mixed race (Negy, 31% non-white; Sutphin, 17% non-white) and one entirely ethnic/racial (Bong) groups, compared to only 8.2% (3.3% mixed, 4.9% entirely ethnic/racial) for articles in the seven journals. Twenty percent of the articles in MFR and the only article in JFTR (Connidis, 2012) were based on international samples.

Discussion

Limitations

One limitation of this comparison is that the senior author, as editor-in-chief of MFR deliberately attempted to increase the number of LGBT-related reviews and reports since 2010 and it would seem that the editorial policies of JFTR were inclusive of LGBT issues. Editors of the other journals may not have attempted to be as inclusive of LGBTQ issues. Furthermore, another limitation to some may be that the MFR editor tried to provide a diversity of opinion on LGBT issues as part of a larger belief in the value of scholarly discussion and debate (Knapp, 2009). Some editors might not allow publication of critiques of themselves (i.e., the editors) in contrast to our editorial policy, whereby we can “take it” (Cameron & Cameron, 2012; Cameron et al., 2017) and “dish it out” in return (Schumm et al., 2017). Our sample size, of course, is small compared to the totals for the seven other journals, making many of the comparisons of questionable value in terms of statistical testing. We did include some reviews with commentaries back and forth, which might not have been included under the criteria used by van Eeden-Moorefield et al. (2018); however, even if half of our articles were disqualified, our rates for publishing LGBT-related research would be no worse than the rates reported for journals in general by van Eeden-Moorefield et al. JTFR may have surpassed MFR in its inclusiveness and some of the other quality characteristics.

Conclusions

There is a risk in accepting costly indices of journals as representing the “best” journals when they might represent the better financed journals (i.e., those sponsored by a well-funded professional organization) rather than the best in actual scholarship. Despite its limited budget and having who many might consider a relatively conservative editor, MFR appears to have held its own, compared to the seven journals featured in van Eeden-Moorefield et al. (2018) with a higher percentage of LGBT-related articles and a higher percentage of reviews and/or commentaries on reviews. A newer family journal, JFTR, as well as the older MFR, also exceeded the top seven family journals in terms of publishing research on LGBT-family issues.

In terms of methodologies, both sets of journal(s) relied upon cross-sectional methods, interviews and surveys either in person or online, and non-random sampling procedures. Limitations in terms of reporting or having diverse racial or socioeconomic backgrounds of the participants were found in both groups. It appeared that a higher percentage of key concepts (outness, discrimination, HIV/AIDS) were discussed in the MFR and JFTR groups of articles. In one sense, the results suggest that editorial policies can make a difference in the extent to which journals include a diversity of LGBT-related research. Future content analyses should probably not limit themselves to restricted lists of social science journals but try to cast a wider net to find greater diversity among the journals themselves. However, at the same time, adding together the citations only increased the overall percentage of LGBT-related articles from 2.24% to 2.42% (184/7611), suggesting that the omission of two journals did not make much difference in the overall percentage of articles on LGBT issues.

In defense of editors

One might interpret the results of van Eeden-Moorefield et al. (2018) to mean that editors of some of the top family journals were discriminating against submitted papers that dealt with LGBT issues. While that is a possibility, there are other possible reasons. First, as van Eeden-Moorefield et al.’s analyses indicate, much of the research on LGBT issues involves non-random, cross-sectional surveys of limited sample size, relatively weak methodologies that may have led to many LGBT-topical manuscripts receiving lower reviewer ratings. Second, Farr, Tasker, and Goldberg (2017) recently found that even among some of the most cited research articles on LGBT issues in the past few years, the use of family theory was often lacking and implicit at best in most of the articles, another weakness that might have influenced reviewers for the top journals. When confronted with research involving large random, even longitudinal, research studies in other topic areas with highly developed theory and research with less well developed theory and lower quality research, many reviewers (and editors) might well decide in favor of the former type of paper rather than the latter.

Second, we might advise scholars that the low rates of LGBT-related research being published in the “top seven” family science journals may, in part, reflect the availability of dozens of scholarly journals whose feature a primary, if not sole, focus and priority were on LGBT-related research, issues, and policy (e.g., Canadian Online Journal of Queer Studies in Education, Columbia Journal of Gender & Law, Duke Journal of Gender Law & Policy, Gay & Lesbian Review Worldwide, IAMURE International Journal of Lesbian, Gay, Bisexual, and Transgender Studies, InterAlia: A Journal of Queer Studies, International Journal of Transgenderism, Journal of Bisexuality, Journal of Homosexuality, Journal of GLBT Family Studies, Journal of LGBT Youth, Journal of Gay and Lesbian Mental Health, Journal of Gay & Lesbian Social Services, Journal of Lesbian Studies, Journal of LGBT Health Research, Journal of LGBT Issues in Counseling, and the International Journal of Sexual Health, LGBT Health, LBGTQ Policy Journal, Les Online: Digital Journal on Lesbian Issues, QED: A Journal in GLBTQ Worldmaking, Sextures, Sexualities, SGP: Sexuality Gender, and Policy Journal; Tulane Journal of Law & Sexuality, Transgender Health, Transgender Studies Quarterly, Psychology of Sexual Orientation and Gender Diversity).

If scholars involved in LGBT-related research perceive a greater chance of rejection of their submissions in certain family journals (whether that perception is correct or not) or perhaps longer times between submitting a paper and having it published, they have a multitude of alternatives for submission of their research, alternatives that may well feature higher acceptance rates and a chance for greater article impact, not to mention quicker times to publication (readers may search journals more often when those journals have a main focus on the type of content that is of primary interest to the reader). If a reader goes to, for example, the Journal of GLBT Family Studies, because of an interest in such topics, they may find more than one article of interest and cite many of them in their own research. If they searched other journals, they might find an article here or there, but might well only cite that one article. The net effect is that an article published in the Journal of GLBT Family Studies may gather more citations in the literature than a similar article published in a family journal with a more general focus. Assessing such differences might be one avenue for further research by family scholars interested in digging deeper into the issues raised by van Eeden-Moorefield et al. (2018).

Third, the past does not lock in the future. Even if top journals, or any journals for that matter, did not publish a high rate of LGBT manuscripts, that does not mean that the journals cannot change. For example, Family Relations recently published a special issue on “Intersectional variations in the experiences of queer families” (van Eeden-Moorefield, 2018), which included 13 additional articles on LGBT issues, which would have increased the total number of LGBT-related articles from 2000–2015 from 153 to 167, an increase of 9.2% from just one issue (of several per year) from one year of one of the seven journals. We are informed, for example, that the Journal of Family Issues is having a special issue on LGBTQ families later in 2018. Therefore, there is a considerable possibility that the dearth of published LGBT research cited by van Eeden-Moorefield et al. (2018) may become a relic of the past, of historical note but by no means an indication of any long-term trends. Hopefully, by pointing out the situation, the situation described so well by van Eeden-Moorefield et al. (2018) will change for the better as quickly as possible.

Fourth, there appears to be no dearth of articles being published on LGBT-related issues. A quick web search (Proquest) for the terms LGBT, GLBT, lesbian, gay, or same-sex published since the year 1999 yielded 54,815 results. Some of those might be duplicates, but nevertheless, it is clear that thousands of articles have been published within the last twenty years on LGBT-related matters. Our own research has found at least 72 published reviews of the literature on same-sex parenting since 2000 (Schumm & Crawford, 2018a) and at least 57 articles or other reports that have investigated the issue of whether LGBT parents are more likely or not to have nonheterosexual children (Schumm & Crawford, 2018b). When we tried to look in the web for reviews on work-family conflict, we only found 15 reviews of the literature since 2000 on that important topic, somewhat less than the 72 we had found with respect to same-sex parenting (which would be only one subset of research on LGBT-related issues). Perhaps the top seven family journals have been “late to the party” because numerous articles on LGBT-related issues have been published elsewhere, in other scholarly journals, in the past two decades, including both Marriage & Family Review and the Journal of Family Theory & Review.