A Content Analysis of Self-Reported Financial Relationships in Biomedical Research

Abstract

Introduction

Financial conflicts of interest (fCOI) present well documented risks to the integrity of biomedical research. However, few studies differentiate among fCOI types in their analyses, and those that do tend to use preexisting taxonomies for fCOI identification. Research on fCOI would benefit from an empirically-derived taxonomy of self-reported fCOI and data on fCOI type and payor prevalence.

Methods

We conducted a content analysis of 6,165 individual self-reported relationships from COI statements distributed across 378 articles indexed with PubMed. Two coders used an iterative coding process to identify and classify individual fCOI types and payors. Inter-rater reliability was κ = 0.935 for fCOI type and κ = 0.884 for payor identification.

Results

Our analysis identified 21 fCOI types, 9 of which occurred at prevalences greater than 1%. These included research funding (24.8%), speaking fees (20.8%), consulting fees (18.8%), advisory relationships (11%), industry employment (7.6%), unspecified fees (4.8%), travel fees (3.2%), stock holdings (3.1%), and patent ownership (1%). Reported fCOI were held with 1,077 unique payors, 22 of which were present in more than 1% of financial relationships. The ten most common payors included Pfizer (4%), Novartis (3.9%), MSD (3.8%), Bristol Myers Squibb (3.2%), AstraZeneca (3.1%), GSK (3%), Boehringer Ingelheim (2.9%), Roche (2.8%), Eli LIlly (2.5%), and AbbVie (2.4%).

Conclusions

These results provide novel multi-domain prevalence data on self-reported fCOI and payors in biomedical research. As such, they have the potential to catalyze future research that can assess the differential effects of various types of fCOI. Specifically, the data suggest that comparative analyses of the effects of different fCOI types are needed and that special attention should be paid to the diversity of payor types for research relationships.

Keywords:

• Conflict of interest
• ethics
• research
• drug industry
• health policy

Introduction

It is well established that financial conflicts of interest (fCOI) present a substantial risk to the integrity of the biomedical research enterprise (Lundh et al. 2017; Cherla et al. 2019; Waqas et al. 2019; Graham et al. 2022). Numerous studies and subsequent meta-analyses have demonstrated that fCOI is associated with the publication of results favorable to industry (Lundh et al. 2017; Cherla et al. 2019; Als-Nielsen et al. 2003). This association has been well validated in multiple clinical areas (Perlis et al. 2005; Lopez et al. 2015; Riaz et al. 2016). Emerging research also indicates that fCOI may be associated with increased adverse event rates for drug products developed in industry contexts (Graham et al. 2022). Although the general deleterious effects of fCOI are well established, research needs to assess the differential effects of different types of fCOI to guide more effective policy making. For example, it may be possible that industry employment presents a greater risk to the integrity of biomedical research than do speaking fees, which would mean they should be reported, understood, and regulated differently. Current studies that evaluate the effects of fCOI tend to treat fCOI as a dichotomous variable (present or absent) for each study of interest (Ahn et al. 2017; Bighelli et al. 2020; Raman et al. 2018; Flacco et al. 2015). Similarly, numerous studies also collapse fCOI and industry funding declarations into a single variable (van Lent, Overbeke, and Out 2014; Etter, Burri, and Stapleton 2007; Bond et al. 2012; Flacco et al. 2015). A notable exception is research that has assessed industry employment of study authors in isolation (Ahmer et al. 2005; Bond et al. 2012; Rattinger and Bero 2009). However, given that many COI policies differentiate between different types of industry relationships beyond employment, a better understanding of the effects of specific types of relationships is necessary to support effective risk mitigation policies. These studies have been vital for determining that there is a relationship between industry funding practices and target outcomes, and they make clear the need for research designs that can identify what types of fCOI pose the greatest risks to biomedical research. However, the lack of empirically derived fCOI taxonomies as well as the lack of appropriate prevalence data may stymie research in this area. This study aims to support future research into the nature and effects of fCOI by providing data on the prevalence of relationship types and payors in a broad sample of disclosed relationships.

Literature review

Several previous efforts have sought to document the prevalence of self-reported fCOI in the biomedical literature (Bellomo, Hwang, and Corriere 2019; Scott, Majdik, and Clark 2020; Grundy et al. 2018; Hampson et al. 2007; Jagsi et al. 2009; Lerner et al. 2012; Lopez et al. 2017). However, these studies were generally not designed to establish empirically-grounded taxonomies of reported fCOI across clinical areas. For example, Grundy et al. and Lerner et al. document the prevalence of published articles with at least one reported fCOI, but they do not distinguish among fCOI types or assess the frequency of fCOIs per author or article. Bellomo, Hwang, and Corriere report data on the number of disclosed fCOI per author, but again do not stratify fCOI by type. The lack of an empirically-grounded taxonomy leaves researchers without a comprehensive framework for evaluating the differential effects of different types of fCOI and leads to significant bodies of research that does not examine potential differences.

Prevalence studies that do stratify fCOI by type tend to be focused on a single clinical area such as oncology or vascular surgery (Jagsi et al. 2009; Lopez et al. 2017; Hampson et al. 2007). Importantly, stratification by type does not necessarily mean that identified types were empirically derived. For example, Hampson et al.’s taxonomy is driven by the categories made available on the American Society of Clinical Oncology (ASCO) disclosure form and Scott et al.’s study is based on the ICMJE disclosure form categories. These taxonomies were developed by expert committees to guide policy and disclosure practices, and may not be representative of the full range of reported fCOI. The prevalence data produced based on them may then conflate, collapse, or ignore fCOI types not considered by relevant policy committees. Additionally, fCOI prevalence studies typically use published articles as the unit of analysis. This is appropriate for determining what proportion of the articles has fCOI. However, because fCOI is asymmetrically distributed in the literature, article based studies may not provide an appropriate foundation for an analysis of what fCOI types are most commonly disclosed. While knowing how many studies report fCOI is critically important for research in this area, it is also essential for researchers to have a comprehensive understanding of what fCOI are reported. Finally, it is notable that few fCOI prevalence studies report data on the distribution of industry payors. A solid empirical foundation for research on fCOI requires foundational research into the prevalence of fCOI types and associated payors. That foundation will support more effective research on COI and allow for the tailoring of policy making to the actual patterns and risks of existing COI.

Methods

The primary goal of this study was, therefore, to determine the nature of self-reported financial relationships, including the types of relationships and associated payors. We aimed to document the prevalence of commonly and uncommonly occurring fCOI types and payors. Most research in this space uses published articles as the unit of analysis, which obscures the prevalence of these types of relationships and payors because fCOI disclosures are asymmetrically distributed across articles. Therefore, relationship disclosures were selected as the primary unit of analysis. For the purposes of this study, we define an individual “relationship disclosure” as a statement linking an author and payor and typically including a description of the type of relationship. Based on this definition, “JQP has received speaking fees from Pfizer and GSK” would constitute two individual relationship disclosures. A prospective target sample size of N ≥ 6,085 relationships was established to achieve a 95% confidence level for fCOI type prevalences as low as 1% (Hajian-Tilaki 2011). The data extracted for this study came from a dataset of 15,374 PubMed-indexed articles known to have reported fCOI (Scott, Majdik, and Clark 2020). In 2017, PubMed added a new data field to its XML-structure for conflicts of interest statements (Collins 2017). Since then, PubMed-indexed journals have the option of providing fCOI disclosure statements alongside other article metadata. In 2019, Scott, Majdik, and Clark extracted all 274,245 PubMed-indexed articles with PubMed-indexed fCOI disclosure statements. The 15,374 are the articles where actual disclosures were identified, as opposed to statements of “no conflicts reported” or similar. A random sample of 1,000 articles was extracted from the 15,374 for possible analysis, using the slice_sample function from dplyr 1.0.7, an R statistical computing library (Wickham, n.d.). The number of reported fCOI per article varies widely. Therefore, we selected a sample of 1,000 to assure that even if all randomly selected articles had relatively low reported fCOI rates, we would still achieve the target fCOI sample size of N ≥ 6,085. Each disclosed relationship was collected until we reached the target sample size (N ≥ 6,085 relationships), and all relationships were included for each article assessed.

Content-analytic methods have been used successfully in the past to scope fCOI among members of patient advocacy organizations (Brems and McCoy 2019). For this study, the research team categorized different terms for identical relationship types. While popular frameworks such as the International Committee of Medical Journal Editors (ICMJE) guidelines have been developed to guide disclosure requirements, these guidelines change over time and are not universally adopted. Additionally, even when such recommendations guide disclosure practices, authors may and do deviate from established guidelines (Drazen et al. 2009). Thus, our interactive content-analytic approach was designed to first surface a fuller range of disclosure statements and then to collapse linguistic variances into meaningful categories (Krippendorff 2018). This includes both simple differences in phraseology such as “are employed by,” vs. “are employees of” as well as more complex categories such as those pertaining to stock ownership or intellectual property rights. Global biomedical research occurs in many different economic, legal, and contractual contexts that may lead to different ways of expressive similar relationships. Shareholding may be described in terms of “stocks,” “shares,” or “equity interests,” whereas patents can be “owned,” “licensed,” or “applied for.” Ultimately, for each disclosed relationship, two raters collected core relationship data including payee text (typically author name or initiations), relationship text (e.g., “employed” “received fees”), and payor text. Relationship text was then assigned a classification representing the fCOI type. The raters began their work collaboratively to refine and norm data collection decision practices. fCOI types were identified iteratively in consultation with the full project team. After this first phase of coding and norming, the coders both independently assessed 153 individual relationships listed in five disclosure statements. The initial exercise achieved near perfect reliability for fCOI type (κ = 0.944) and very good reliability for sponsor identification (κ = 0.798). After additional discussion and reconciliation, a second reliability exercise using 134 individual relationships reported in 10 disclosure statements achieved κ = 0.935 for fCOI type and κ = 0.884 for sponsor text. In the second interrater reliability exercise, nine fCOI types were identified (i.e., advisory, consulting, employment, fee-unspecified, patent, research, speaking fees, travel fees, and employment management). Agreement was 100% for all categories, except for fee-unspecified. The raters agreed on 12 of 19 instances. Given the high degree of reliability across coding categories, the raters subsequently began to independently code the remaining data. During this time, the raters met periodically to discuss results and continue to norm data collection practices, especially when evaluating fee-unspecified cases and when confronting new fCOI categories.

We cleaned and harmonized the payor data prior to analysis, because payor labeling in the disclosure statements tends to be inconsistent. For example, it is not uncommon to find even different employee authors from the same study referring to their employer in different ways (e.g., GlaxoSmithKline, Glaxo, and GSK). Preparing these data required making a number of decisions about how to treat complex multinational entities, and we expected to treat multinational conglomerates as the same company. We, therefore, focused on primary company names wherever possible, filtering out information on corporate structures (Inc., LLC., Ltd., GmbH., etc) and geographic locations. For example, all instances Pfizer, Inc., Pfizer Ltd., and Pfizer India, and Ltd became “Pfizer.” We also identified common company abbreviations in the payor text (e.g., GSK, BMS, AZ) and expanded those names prior to analysis. We did not try to identify parent or subsidiary payors so as to further collapse the data in most cases with two exceptions: We identified all reports of financial relationships with “Merck” as “MSD” to follow the globally more common naming convention, and we collapsed all individually identified NIH institutes or centers into a single “NIH” category. Finally, we also classified all payors as industry, government funding agency, university or academic medical center, patient advocacy organization (PAO), professional medical organization (PMO), or other. The industry category was assigned to any for-profit entity regardless of sector, and other included primarily private charitable foundations not classifiable as PAO or PMO, private individuals, and unclassifiable fCOI (i.e., those arising from unclear abbreviations).

Results

The results of this analysis included 6,165 reported relationships distributed across 378 articles in 190 journals. This includes 32 articles in Plos One, 20 in BMJ Open, 14 in Oncotarget, and 13 in PNAS. The remaining journals were each represented fewer than 10 times in the data. The collected journals represent a wide range of general medical journals (e.g., The BMJ, Annals of Internal Medicine, Plos One, The New England Journal of Medicine) and a variety of subspecialty journals in psychiatry, oncology, gastroenterology, sports medicine, nephrology, endocrinology, rheumatology, and so on. All articles included were published between 2015 and 2019, with an average publication date in 2017. There were seven articles published in 2015, 84 in 2016, 96 in 2017, 176 in 2018, and 15 published in 2019. Complete journal publisher information, publisher location, open access status and metrics are available in the supplementary material and were collected from the open-access Scimago database, which is more comprehensive than the proprietary Scopus and Web of Science alternatives (Falagas et al. 2008). Eleven of the selected journals were not indexed with Scimago. For these journals, we were able to retrieve publisher, location, and open access data, but not comparable impact metrics. The represented journals were published in 14 countries by 65 unique publishers. Publisher data represents those indexed by Scimajo or reported on the journal masthead. It does not always account for imprints owned by parent publishers. The majority of journals are published in the UK (78), the US (47), New Zealand (23), and Switzerland (10). The most common publishers were BMJ Publishing Group (23), Dove Medical Press (21), Springer (20), Nature Publishing Group (13), Sage (13), and Taylor and Francis (13). A majority of the journals included in this study were open access (104 of 179, 58%). The SJR prestige score is analogous to more commonly reported impact factors and has been found to closely correlate with similar metrics provided by proprietary companies (Falagas et al. 2008). Of the Scimajo indexed journals (179), the latest available SJR scores varied widely, ranging from 0.0 to 19.9 (m = 1.8), and H-indices ranged from 2 to 1,030 (m = 101.2).

Our analysis identified 21 fCOI types reported in the literature. However, only 9 of those fCOI types occur at a frequency of greater than 1% of reported fCOI. These types included research funding (24.8%), speaking fees (20.8%), consulting fees (18.8%), advisory relationships (10.9%), industry employment (7.6%), unspecified fees (4.8%), travel fees (3.2%), stock holdings (3.1%), and patent ownership (1%). fCOI types occurring at less than 1% included provision of educational services, company ownership, receipt of royalties, editorships, receipt of research supplies, management-level positions of employment, other types of intellectual property, travel fees, other fee types, awards, endowed chairs, and serving as an expert witness.

Additionally, our analysis identified 1,077 unique payors, 22 at greater than 1% prevalence. The ten most common payors included Pfizer (4%), Novartis (3.9%), MSD (3.8%), Bristol Myers Squibb (3.2%), AstraZeneca (3.1%), GSK 3%), Boehringer Ingelheim (2.9%), Roche (2.8%), Eli LIlly (2.5%), and AbbVie (2.4%). As the top 10 payors indicate, the most common payor type was industry at 91.2% of reported fCOI. 4.1% of payors were classified as other, 3% as agency or research council, and a combined 1.7% were universities or academic medical centers, patient advocacy organizations, or professional medical organizations. Through comparing payor and fCOI type data, we can also learn about important differences in research funding profiles by each company. Figure 1 displays the proportion of top 5 most prevalent fCOI types for each of the top 10 most prevalent companies. From this, we can see that speaking fees is the most prominent payment type for most of the top 10 companies. However, AbbVie and GlaxoSmithKline were notably distinct in that their dominant author relationships were research funds and author employment, respectively. Finally, we also assessed fCOI frequencies by payor type (Table 1). The data showed that industry was the dominant payor in almost all cases, with the exception of patent holding. Individuals often held their own patents. Research was notable for being industry involved in fewer than 90% of cases. As one would expect, research also had, by far, the highest rate of agency-associated fCOIs at 9.5%.

Figure 1. Frequency of fCOI type among top 10 payors.

Table 1. Number of percent of payor type by fCOI type for fCOI types ≥ 1% prevalence.

	Industry	Other	UAMC	Agency	PMO	PAO
Advisory	640 (94.7%)	23 (3.4%)	5 (0.7%)	3 (0.4%)	3 (0.4%)	2 (0.3%)
Consulting fees	1139 (98.1%)	8 (0.7%)	2 (0.2%)	6 (0.5%)		6 (0.5%)
Educational	60 (100%)
Employment	421 (90.1%)	18 (3.9%)	8 (1.7%)	15 (3.2%)	1 (0.2%)	4 (0.9%)
Fee-unspecified	275 (92%)	14 (4.7%)	6 (2%)	3 (1%)	1 (0.3%)
IP-patent	1 (1.6%)	62 (96.9%)	1 (1.6%)
Research	1258 (82.2%)	69 (4.5%)	22 (1.4%)	145 (9.5%)	7 (0.5%)	29 (1.9%)
Speaking fees	1269 (98.9%)	10 (0.8%)		3 (0.2%)	1 (0.1%)
Stock	189 (99.5%)	1 (0.5%)
Travel fees	185 (93.9%)	5 (2.5%)	2 (1%)	3 (1.5%)	1 (0.5%)	1 (0.5%)

Discussion

This study had two principle aims: (1) to develop an empirically-grounded taxonomy of self-reported fCOI, and (2) to provide a novel, foundational understanding of the prevalence of fCOI types and payors in a range of general medical and subspecialty journals. Our empirically-derived taxonomy includes 10 primary fCOI types that occurred at prevalences of greater than 1%. This taxonomy overlaps with preexisting taxonomies and is largely confirmatory. That is, it is now clear that commonly provided taxonomies like those from ICJME or ASCO are generally represented of self-reported disclosure types. Additionally, this study provides novel information on the prevalence of fCOI types and funders. Remuneration for research, consulting, and speaking are the most common fCOI types across the sampled disclosure statements. These findings suggest that future research should be reoriented to pay additional attention to these areas when studying fCOI types comparatively or in isolation. Although industry employment is the most well-studied fCOI type, it only comprises 7.6% of disclosed fCOIs. The overall effects of industry employment on biomedical research may be less substantial than effects arising from research funding, speaking fees, consulting fees, and advisory relationships, and a focus only on employment ignores the majority of relationships.

Previous research has found that research funding relationships may not present the same kinds of risks as do other forms of fCOI (Scott et al. 2022), and the results of this study show the diverse range of other, less-well studied fCOI that merit further study. Specifically, whereas industry employment, stock holdings, and other personal fCOI have been associated with higher adverse event rates, research relationships have been associated with lower adverse event rates (Scott et al. 2022). Research-related fCOI tends to associate with government funding agencies, patient advocacy organizations, and other non-industry payers. Research grants from industry are frequently awarded to universities and academic medical centers. They offer financial benefits to authors, but employer policies governing use of funds and research conduct may reduce the overall risks associated with the financial relationships. Research is needed that evaluates the distinctive functioning of different forms of fCOI, and this study provides an empirical foundation for that research. Finally, fCOI governance in biomedical publishing focuses primarily on authors. However, additional data on company-author relationships can provide useful information that might lead to effective strategies for more effective researcher practice and more effective oversight, which together can cultivate more ethically and pragmatically sound research. For example, author employment in industry associates with up to an 8.33 fold increase in the likelihood that a study will return results favorable to industry (Ahmer et al. 2005). If these findings were confirmed through additional comparative research on the effects of different fCOI types, it might indicate that greater attention should be paid to scrutinizing companies who invest more in research produced by employee authors.

Limitations

The data presented in this study are grounded in disclosure statements from journals that participate in the PubMed fCOI reporting program. Not all journals participate in this scheme. While the data presented here represent a broad range of medical journals by subspecialty, prestige, and publisher, it may be possible that journals that do not participate have different fCOI distributions that could affect aggregate numbers across a broader sample of journals. To the best of our knowledge, there is no comprehensive audit of which PubMed-indexed journals report fCOI data. Our general impression is that reporting journals are more likely to be open access. There are 30,000 journals currently indexed with PubMed and 12,500 open access journals indexed with the Directory of Open Access Journals. Even if all of these were indexed with PubMed (which they are not), that would mean that fewer than 42% of PubMed-indexed journals are open access. Yet, 55% of the journals included in this study are open access. Prior research found that disclosures in journals that accept reprint fees but typically not open access tend to have greater numbers of reported relationships overall (Scott, Majdik, and Clark 2020). It is unclear if this would translate into differences in prevalence by fCOI type. Additionally, the results presented here are limited to self-reported fCOI in published disclosures statements. Published disclosure statements do not always accurately reflect all industry relationships held by authors (Rasmussen et al. 2015). Finally, when relationships are reported, author-provided data may be incomplete. For example, in this study, 4.8% of the identified fCOIs were for fees of an unspecified nature. Incomplete fCOI disclosure information may affect both prevalence estimates and future research on differential risks.

Conclusion

This study provided novel, multi-domain prevalence data on self-reported fCOI in biomedical research. As such, the results can productively inform future research to assess the differential effects of various types of fCOI. The findings indicate what fCOI types should be the next investigational target of interest and may support analyses of aggregate effects of varying fCOI types. The findings point to the need to assess the effects of research funding, speaking fees, and consulting fees given that these types are the most commonly reported. This insight is especially important for research funding in that fCOI often involve non-industry payors and indirect remuneration through university and academic medical centers. Finally, this study also provided new information on which companies are most commonly involved with fCOI and what types of fCOI authors usually have. These data may contribute to research practice and the development of governance strategies beyond the author-focused approaches currently in use.

Data availability statement

Data are available upon request.

Disclosure statement

There are no conflicts of interest to report.

Additional information

Funding

This work was supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Number R01GM141476.