https://orcid.org/0000-0001-7413-5248
ABSTRACT
A systematic literature review was conducted to describe in a historical perspective the evolution of studies concerning HPV vaccination. The search identified 794 articles of which 568 were included. The first article was published in 2001, and the maximum annual number of publications was reached in 2014. The average number of authors per paper was 8.8. Papers originated from 49 different countries, with the USA accounted for the maximum number of publications (n = 217). Efficacy (46.5%) and safety (31.0%) were the most prevalent objectives. Clinical trials constituted the largest group of methods (37.9%). Chronological trends did not reveal any lasting curve-crossings, indicating that the priority topics have remained the same. The geographical origin of these studies raises questions about the transposability of the results to populations where HPV vaccination has been studied only a little. This study could help guide future research to less-studied research objectives, particularly for vaccines.
Introduction
Human papillomavirus (HPV) infection is the most common sexually transmitted infection in the world: 70% of men and women will be infected with at least one type of HPV in their lifetime. In most cases, this infection occurs within the 5 first years of sexual activity.Citation1 To date, more than 120 types of HPV have been described, among which fifteen are considered high-risk types (oncogenic HPV types).Citation2,Citation3 HPV can affect both men and women, and cause oropharyngeal, head & neck, anal, penile, vulvar and cervical cancers.Citation4 HPV is found to be present in 99.7% of cervical carcinomas, with a prevalence of 73% for the HPV16 and HPV18 types.Citation5 In 2002, cervical cancer accounted for 493,000 new cases and 273,000 deaths.Citation6 In 2018, 569,000 new cases were recorded with 311,000 deaths worldwide.Citation7
In 1974, German virologist Harald Zur Hausen and his team discovered that HPV was responsible for cervical cancer, leading to Zur Hausen receiving a Nobel Prize in 2008.Citation8 His work paved the way for the development of a vaccine. Research in this field has grown exponentially, ultimately leading to the introduction of HPV vaccines. Between 2006 and 2007, these vaccines were incorporated into the vaccination schedules of most developed countries. There are currently three vaccines on the market. The quadrivalent vaccine (6, 11, 16, 18 types) was licensed in 2006; the bivalent (16, 18 types) in 2007 and the nonavalent vaccine (6, 11, 16, 18, 31, 33, 45, 52, 58 types) in 2014.Citation9 The nonavalent vaccine is the successor to the quadrivalent by adding five more HPV genotypes antigens such that no market offers all three vaccines: some markets distribute the quadrivalent and not the nonavalent, while others markets have replaced the quadrivalent with the nonavalent.
The growing medical literature on this topic allows for the publication of a lot of state-of-the-art reports, but also provides an opportunity to carry out a bibliometric study on the subject. Bibliometric outcomes are used to explore, organize and analyze large volumes of historical data to display trends toward a theme in the scientific literature, using statistical analysis to measure scientific output, to show how knowledge on a medical field is built, to question the topic of interest to the scientific community, to shed comparative light on current debates, to highlight gaps in certain areas of research, and to map out the present science in order to suggest the direction of future research.Citation10,Citation11Citation12
Considered a new social phenomenon, vaccine hesitancy has become a common problem worldwide, particularly in the European and the Western Pacific regions.Citation13 A review of the determinants of HPV vaccine hesitancy in Europe showed that the barriers to vaccination include lack of information, concerns about potential side effects, and mistrust of health authorities, healthcare workers, and new vaccines.Citation14 In the current context of controversy and debates on vaccination, especially against HPV, to the best of our knowledge, no detailed bibliometric study on HPV vaccination has ever been carried out.
The objective of this study was to describe the evolution of studies on HPV vaccination in a historical perspective. This study focused on research methods, study population, number of authors, country of origin, publication journals, impact factor, funding and disclosure of interest.
Methods
A systematic review was conducted until the 05/23/19 with no lower limit, using the Medline database via the Pubmed interface. All original studies on HPV vaccination published in English or French were eligible. Animal studies, research methodology study (such as dosage, statistic methods or assessment tools), articles without abstract or those introducing only inclusion data were excluded.
The following search query was used: “Papillomavirus Vaccines”[Majr] AND ((Case Reports[ptyp] OR Clinical Study[ptyp] OR Clinical Trial[ptyp] OR Clinical Trial, Phase I[ptyp] OR Clinical Trial, Phase II[ptyp] OR Clinical Trial, Phase III[ptyp] OR Clinical Trial, Phase IV[ptyp] OR Comparative Study[ptyp] OR Controlled Clinical Trial[ptyp] OR Evaluation Studies[ptyp] OR Meta-Analysis[ptyp] OR Multicenter Study[ptyp] OR Observational Study[ptyp] OR Pragmatic Clinical Trial[ptyp] OR Randomized Controlled Trial[ptyp] OR Retracted Publication[sb] OR Retraction of Publication[sb] OR Validation Studies[ptyp] OR systematic[sb]) AND (English[lang] OR French[lang])).
The selection was made on the basis of title and abstract by two researchers (MP and PF), with consensus in case of disagreement. For each study included, the following data were extracted: year of publication, type of vaccine studied, aim of the study, type of method, population studied, number of authors, country of origin, publication journal, impact factor, funding and disclosure of interest.
Data on vaccine type were sorted into three groups: bivalent, quadrivalent, nonavalent. Data on research methods were sorted into ten groups: vaccination/vaccine, vaccination schedule, vaccination policy, cost, intervention to increase HPV vaccination coverage, efficacy, safety, patient knowledge/opinions and attitudes, vaccination intention, healthcare professional knowledge/opinions and attitude. The research methods were sorted into 13 groups: case study/case series, cross-sectional study, ecological study, cohort study, case-control study, uncontrolled study, comparative trial, medico-economic study, qualitative study, systematic literature review, quantitative meta-analysis, mathematical modeling and unspecified method. The study population was sorted by sex and specific profile: healthcare professional, parents, student (school/university), and specific sub-population (HIV+ person, men who have sex with men, pregnant women, patients with autoimmune disease).
The number of authors was sorted into five groups: [1–2 authors], [3–6 authors], [7–12 authors], [13–24 authors], [>24 authors]. These groups were formed to meet the Vancouver standards, where more than 6 authors are followed by “et al”, and to meet the National Library of Medecine (NLM) standards which stop listing authors beyond 24. When a study group was listed as an author in an article, that group was considered as a single author for all members of the group.
When a country name appeared in the title of the article or in its abstract, that country was considered to be the country of origin of the article. Where more than one country or no country was mentioned, the country of origin was considered to be that of the corresponding author. When the corresponding author had more than one country mentioned, the country of origin was considered to be his or her e-mail’s country domain name.
Impact factors were considered using the Journal Citation Reports corresponding to the year of publication of the article.
Funding information was collected from full-length articles and using the Clinicaltrials.gov database when declared therein. Funding information was restricted to what was declared by the authors, even when the vaccine supply by the manufacturer could have been omitted by the authors in the funding declared. Disclosure of interest was also collected from full-length articles, restricted to author declarations.
Analysis of data was performed using IBM SPSS statistics version 24 (IBM corporation, New York, USA) and Excel (Microsoft, Redmond, Washington, USA).
Results
We identified 794 results using PubMed on the 05/23/19, published between 1990 and 2019. We excluded 226 articles: therapeutic vaccine (n = 52), no abstract (n = 46), research methodology (n = 40), irrelevant (n = 36), expert opinion (n = 16), animal or in vitro trial (n = 15), clinical trial protocol (n = 14), only inclusion data (n = 6) and duplicate records (n = 1). As a result, 568 articles could be included, with publication dates ranging from 2001 to 2019. The characteristics of the studies are presented in .
Table 1. Characteristics of the studies (N = 568)
The number of studies published each year ranged from 2 studies in 2001 to 69 articles in 2014. According to their abstracts, the first publication dealing explicitly with the bivalent vaccine was in 2004, the first with the quadrivalent vaccine was in 2005 and the first with the nonavalent vaccine was in 2012. The maximum number of publications was reached in 2014 for the bivalent vaccine (n = 28), in 2012 for the quadrivalent vaccine (n = 21) and in 2015 for the nonavalent vaccine (n = 8). There were only 23 studies on the nonavalent vaccine (n = 131 for the bivalent and n = 175 for the quadrivalent).
The average number of authors per article was 8.81 (ranging from 1 to 44). Eight articles had a single author. One research group was mentioned in the authors for 60 studies.
Objectives of the studies
Among the ten types of objectives identified, the first publications were issued in 2001 for studies on efficacy and safety, in 2006 for studies on patient knowledge/opinions and attitudes, in 2007 for studies on health professionals’ knowledge/opinions and attitudes, and in 2008 for studies on vaccination intention. HPV-induced cancers were specifically assessed in 33 studies. They were: cervical cancer (n = 28), vaginal cancer (n = 6), oropharyngeal cancer (n = 6), anal cancer (n = 5), vulvar cancer (n = 5) and penis cancer (n = 4).
Publications peaked in 2011 and 2013 for studies of vaccination intention (n = 7), in 2014 for efficacy (n = 36) and safety (n = 21) studies, in 2015 for studies of patient knowledge, opinions and attitudes (n = 31) and in 2017 for studies of health professionals’ knowledge (n = 6) (see ).
Figure 1. Historical evolution of study objectives
Methods of the studies
Among the thirteen types of studies identified, the first publications appeared in 2001 for controlled or uncontrolled clinical trials, in 2006 for cross-sectional studies, in 2007 for systematic reviews/meta-analyses, as well as for cohort and qualitative studies, in 2008 for medico-economic studies and case studies and case series.
Peak publication rates were reached in 2008 for medico-economic studies (n = 7), in 2012 for case studies or case series (n = 7), in 2014 for systematic literature reviews/meta-analyses (n = 15), in 2015 for controlled/uncontrolled clinical trials (n = 27), for cross-sectional studies (n = 14), and in 2017 for qualitative studies (n = 3). For cohort studies, there were two peaks in 2014 and 2016 (n = 10) (see ).
Figure 2. Historical evolution of study methods
Study population
The first publications on women were issued in 2002, and the first on men in 2006. The peak in publications was reached in 2014 for women’s studies (n = 47) and in 2015 to 2017 for men’s studies (n = 13).
Among the four specific population profiles, the first publications were in 2006 for studies of a particular sub-population (HIV+ patients, male homosexual patients, patients with autoimmune diseases, pregnant women), in 2008 for school-based studies and studies of healthcare professionals, and in 2009 for studies of parents.
The peak in publications was reached in 2012 for school-based studies (n = 7), in 2014 for sub-population-specific studies (n = 11), in 2017 for health professional studies (n = 6), and for parent studies (n = 9).
Country of origin
The publications came from 49 different countries, each having published between 1 article (13 countries) and 217 articles (1 country: USA). Thirty-five countries published fewer than 10 articles. Eight countries published between 10 to 20 articles. Six countries published more than 20 articles: Australia (n = 23), Italia (n = 24), China (n = 25), United Kingdom (n = 37), Canada (n = 43), and USA (n = 217).
Journal of publication
The articles reviewed came from 197 different journals and 129 journals had only one publication relative to HPV vaccination. The two journals with the most publications were Vaccine and Human Vaccines & Immunotherapeutics (see ).
Figure 3. Total number of publications in the most prolific journals on HPV vaccination (with NEJM and Lancet for information)
Impact factor
The average impact factor found was 4.766, ranging from 0.323 (Archivos Argentinos De Pediatria) to 59.558 (New England Journal of Medecine). The coefficient of determination between the average impact factor and the number of publications per country was R2 = 1.6E-10. The coefficient of determination between the impact factor and the number of authors was R2 = 0.16. The evolution of the average value of the impact factors is shown in . The distribution of the impact factor per country is shown in Appendix 1 and the average impact factor according to the type of study is shown in .
Table 2. Conflict of interest, funding and impact factor by type of study
Figure 4. Evolution of the average value of the impact factors
Funding and disclosure of interest
The most frequently declared conflicts of interests were with manufacturers such as Merck (n = 208 studies), GlaxoSmithKline (n = 180), Pasteur (n = 86), Roche(n = 34), Pfizer (n = 34), CSL (n = 25) and Novartis (n = 21). For four articles, at least one author was a member of an industrial company that could be involved in a conflict, without being in the “declaration of conflict of interest” section. These articles were therefore sorted as “unknown conflicts”. For one article, the authors stated that they had received grants without considering them as a conflict, the article was therefore classified as without conflict. shows the historical evolution of funding and disclosure of interest.
Figure 5. Evolution of funding (left) and evolution of disclosure of interest (right)
Among the countries with more than 20 publications, the country with the highest number of conflicts reported in studies was Australia (60.9%), followed by the USA (49.3%), Canada (46.5%), China (40.0%), the UK (35.1%) and Italy (8.3%). Funding and conflicts according to the type of study are presented in .
Discussion
Our results show that the number of studies on efficacy and safety remains predominant although it does now seem to be slowing down, while the number of studies on practices, opinions and intentions is growing, highlighting the current pragmatic challenge of HPV vaccination. The majority of studies continue to aim for a high level of evidence through clinical trials and literature reviews. The emergence of literature reviews and their multiplication would seem logical and appropriate in parallel with the growth of the corpus of original studies. The expansion of literature reviews does not appear to be at the expense of the slowing down of original studies.
The evolution of scientific output on HPV vaccination appears to be in line with the growth of general scientific output. In 1963, in his work on the exponential growth of science, Price noted that all the scientific journals founded since the very first, as early as 1665, had produced more than 6 million articles in the previous 300 years, with scientific output doubling every 15 years since 1665.Citation15 In 2005, Druss showed that in only 23 years, from 1978 to 2001, more than 8 million articles had been published on Medline.Citation16 The results of our study demonstrated this growth in specific human papillomavirus vaccine research, with an average increase in the number of publications of 28% per year between 2001 and 2018, peaking at 150% between 2006 and 2007.
The results of this bibliometric study show a high degree of momentum in this research topic (N = 794). As a comparison, the same search query adapted to Meningococcal Vaccines found 521 articles in PubMed on 05/23/19, 412 articles when adapted to Rotavirus Vaccines and 134 when adapted to Herpes Zoster Vaccine. Moreover, the good level of evidence and the pragmatic approach seem consistent in responding to the current controversy.
The chronological trends do not reveal any sustained crossover, indicating that the priority topics remain the same. The majority of the objectives (46.3%) concerned vaccine efficacy (combining immunogenicity studies, efficacy on infection, on precancerous lesions, on cancer, on mortality and on quality of life), followed by safety studies (30.8%) with an almost constant increase in the number of publications until 2014. These proportions seem logical given that HPV vaccines are still a relatively recent innovation, the majority of the studies published were clinical trials. The least treated subjects concern the knowledge and practices of health professionals (6.3%). In the middle of the table there are questions directly concerning patients (knowledge/opinions, behaviors and vaccination intention), the majority of which were published between 2011 and 2016. These curves could be reversed in the years to come, as efficacy studies are most often conducted early in the history of a medical innovation. The efficacy studies that have yet to be published are likely to be long-term efficacy studies, particularly on the occurrence of cervical cancer and mortality, with a delay of several decades being required to access these data.Citation17
In terms of methods, the chronological evolution clearly corresponds to the specific natural evolution of a health intervention, from fundamental studies to clinical studies, then to pragmatic studies and finally to work aimed at understanding the behaviors and modalities of appropriation. The study methods used follow the same logic as the research questions. Since efficacy and safety studies are mostly clinical trials, clinical trials have largely dominated the scientific literature on HPV vaccines from the outset. More systematic literature reviews and meta-analyses appeared even later, the majority of which were published after 2013.
The evolution of publications can also be seen in the light of the emergence of a general mistrust toward vaccines. In France, since 2010, HPV vaccination coverage has been low, with a complete vaccination rate in 2015 of less than 14% and 1 dose administered in 2016 to only 19% of the target population.Citation17,Citation18 This may be partly explained by concerns and controversy surrounding vaccines and in particular the risk of vaccine-related autoimmune diseases. In our study, case studies increased until 2012, with some reporting isolated improvements after vaccination but most suggesting a possible association between the HPV vaccine and the subsequent onset of disease. The association between autoimmune diseases and HPV vaccination was disproved in 2015 by a large international observational study of a cohort of more than 2 million girls in an exposed/unexposed model.Citation19 The surge in publications on cohort studies, literature reviews, clinical trials, and safety studies between 2013 and 2015 may be partly related to attempts to find a clear answer to these concerns about iatrogenic management.
The total number of authors has increased steadily until 2014, with no clear rise in the number of authors per article. In 1978, the ICMJE group established rules to standardize medical production, and updated them in 1997 by redefining the criteria for authorship.Citation20 In 1993, in an editorial for the British Medical Journal, Epstein raised the problem of the increasing number of authors per article observed, and suggested that only the first and last author of studies be cited.Citation21 However, these policies have not shown any effect on the number of authors found in medical publications over time,Citation22,Citation23 with an almost constant increase in this average number.Citation23,Citation24,Citation25,Citation26Citation27 This study stands out with a stable trend in the average number of authors, from 2008. However, while the average number of authors did not increase in this study, it was higher than in other studies of this type. The high average number of authors at the start of HPV research may explain why we did not find an increasing trend.
In a 2018 study focusing on medical publications from different countries between 1995 and 2015, the USA stood out with 4.19 million publications, against 0.91 million for the second country, China.Citation28 Our study shows no exception, with the USA being the country involved in the largest number of publications. The large number of clinics and research centers providing publications in Anglo-Saxon countries may partly explain such a geographical distribution of publications. Another possible explanation for this distribution is the difficulty encountered by authors from less wealthy countries in promoting their work and getting it published.Citation29
The publications came from 197 different journals, more than a third (37.5%) came from journals that had published more than 10 articles related to HPV vaccination, and more than a fifth came from two journals, “Human Vaccines & Immunotherapeutics “and” Vaccine “. These data illustrate Bradford’s law, a small core of journals have as many papers on a given subject as a much larger number of journals.Citation30 This can be explained by the fact that the number of citations of a publication is lower if it does not belong to one of these core journals. This phenomenon is also illustrated in other areas of clinical research.Citation31,Citation32,Citation33,Citation34
There is no correlation between the quantity of publications produced per country and the average value of the impact factor (R2 = 1.6E-10). The correlation between the number of authors of a publication and the impact factor of the journal in which it was published seems negligible (R2 = 0.16). This negligible correlation shows that there is no need to have a larger number of authors to produce a better quality publication. The correlation figure found is consistent with that previously described in the literature.Citation27 The impact factor of papillomavirus-related publications appears to remain stable over time. Although currently decried because of its misuse,Citation35,Citation36,Citation37 and the fact that new indicators for measuring the visibility of a journal have emerged, such as “Altmetrics” or the “H index,Citation38,Citation39 the impact factor remains one of the most widely used methods to assess the quality of a medical journal.Citation40
Public funding of studies seems to gradually join private funding. Funding was present for more than 75% of studies, although since 2008, there seems to have been an increase in the number of studies declared unfunded. In 2007, Reed et al demonstrated the association between a study of better quality and the presence of funding.Citation41 Few recent studies can explain the changes in the distribution of funding found here. It is important to note that the study carried out here looked at the sources of funding and not the amount of money invested. The public amount invested may actually be less than the private one. In a study from 2015, the United States remained the country investing the most in research, but with a notable drop in public investments. At the same time, other regions of the world were increasing their share of public funding, notably China and the European Union.Citation42
A rise in the declaration of conflicts of interest was observed. The presence of a conflict appears to be decreasing slightly, but remains at 50%. In 1984, the New England Journal of Medicine set up a policy of transparency, obliging authors wishing to have their manuscripts published in their review to declare their conflicts of interest.Citation43 Since then, many journals have also included a declaration of conflicts in their publication criteria, and made this growing trend toward transparency achievable.Citation44,Citation45 This can be interpreted as a response by the scientific community to the current climate of mistrust in research and vaccination.
Strengths and limitations
We did not find any bibliometric study exploring the objectives of studies on HPV vaccination, which underlines the novelty of this study but does not allow us to compare our results and to verify its external validity. From a methodological point of view, this literature review attempted to comply with the PRISMA recommendations. Data extraction was carried out by two researchers in a systematic manner and all abstracts identified were read exhaustively in order to exclude irrelevant articles and generate a ranking of research questions, study types and study populations. However, there are a number of limitations in terms of the methodology. To our knowledge, there are no international guidelines on bibliometric methods, and existing bibliometric studies seem to be diverse.Citation46 The closest Equator Network guidelines to our method could be the PRISMA guidelines for systemic review. However, our aim was not to carry out an exhaustive inventory on a specific topic but to outline trends on a major research theme. We focused on the Medline® database and no other database such as Embase® or gray literature was used. We made this choice because our aim was to identify trends and their dynamics in this area of research and not to provide an exhaustive quantification of the scientific output. The presence in the search query of the occurrence “French” may lead to an overestimation of the number of publications from this country compared to others. The social sciences are not always well represented in Medline® either. However, this database was chosen because it is the most easily accessible and is the indispensable source for biomedical publications.Citation47 Although it goes further than bibliometrics based on keyword indexing, the fact that the collection is based on abstracts may have led to several biases. Some abstracts were not available and others were more or less incomplete because some of the data searched for were missing. Depending on the different works, more than 50% of the abstracts could be incomplete,Citation48 with a level of inconsistency between the abstract and the full article measured from 4 to 78%,Citation49 25% containing omissionsCitation50 and 18 to 68% deficient.Citation51 In a study of physiotherapy, only 44.5% of abstracts fully reported the criteria for the study population, compared with 89.5% of full articles.Citation52 In another study, 75% of the studies had at least one inconsistency in their abstract.Citation53 It is apparent that these gaps and inconsistencies most often ocur to the detriment of the paragraphs on methods and results,Citation54 not respecting the PRISMA-for-Abstract checklist,Citation55,Citation56 with often erroneous numbers reported.Citation57 In our study, however, these latter biases remain limited because we did not look at the results of the studies included. Finally, we did not collect the samples sizes of the studies, which could have been a significant limitation if we had examined the results of the studies. We made this choice because comparisons between samples sizes from clinical trials, meta-analyses, qualitative studies and case studies would have been hazardous.
Conclusion
This study illustrates the exponential dynamics of the production of publications on HPV vaccine research. The good level of evidence and the pragmatic approach seem consistent in responding to the current controversy. The increase observed in the declarations of conflicts and funding probably reflects a societal dynamic for transparency. Some topics still seem to be underdeveloped, such as the nonavalent vaccine or the knowledge and practices of healthcare professionals. The geographical origin of these studies raises questions about the transferability of the results to populations where this vaccination has been less studied.
Disclosure of potential conflicts of interest
The authors have nothing to disclose.
Additional information
Funding
References
- Chesson HW, Dunne EF, Hariri S, Markowitz LE. The estimated lifetime probability of acquiring human papillomavirus in the United States. Sex Transm Dis. 2014;41:660–64.
- Bernard H-U, Burk RD, Chen Z, van Doorslaer K, Zur Hausen H, de Villiers E-M. Classification of papillomaviruses (PVs) based on 189 PV types and proposal of taxonomic amendments. Virology. 2010;401:70–79.
- Muñoz N, Bosch FX, de Sanjosé S, Herrero R, Castellsagué X, Shah KV, Snijders PJF, Meijer CJLM. Epidemiologic classification of human papillomavirus types associated with cervical cancer. N Engl J Med. 2003;348:518–27.
- WHO. Human papillomavirus (HPV) [Internet]. WHO. [ cited 2020 Jul 10]; Available from: http://www.who.int/immunization/diseases/hpv/en/
- Walboomers JM, Jacobs MV, Manos MM, Bosch FX, Kummer JA, Shah KV, Snijders PJ, Peto J, Meijer CJ, Muñoz N. Human papillomavirus is a necessary cause of invasive cervical cancer worldwide. J Pathol. 1999;189:12–19.
- Parkin DM, Bray F, Ferlay J, Pisani P. Global cancer statistics, 2002. CA Cancer J Clin. 2005;55:74–108.
- Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68:394–424.
- Zur Hausen H. Papillomaviruses in the causation of human cancers — a brief historical account. Virology. 2009;384:260–65.
- WHO. Countries using hpv vaccine [Internet]. WHO. [cited 2020 Apr 10]; Available from: https://www.who.int/immunization/diseases/hpv/decision_implementation/en/
- Cole FJ, Eales NB. The history of comparative anatomy: part 1 - A statistical analysis of the literature. Sci Prog. 1917;11:578–96.
- Wouters P. Eugene Garfield (1925–2017). Nature. 2017;543:492–492.
- Garfield E. Citation analysis as a tool in journal evaluation. Science. 1972;178:471–79.
- Larson HJ, de Figueiredo A, Xiahong Z, Schulz WS, Verger P, Johnston IG, Cook AR, Jones NS. The state of vaccine confidence 2016: global insights through a 67-country survey. EBioMedicine. 2016;12:295–301.
- Karafillakis E, Simas C, Jarrett C, Verger P, Peretti-Watel P, Dib F, De Angelis S, Takacs J, Ali KA, Pastore Celentano L, et al. HPV vaccination in a context of public mistrust and uncertainty: a systematic literature review of determinants of HPV vaccine hesitancy in Europe. Hum Vaccin Immunother 2019;15:1615–27.
- Price DJDS. Little science, big science. New-York: Columbia University Press; 1963.
- Druss BG, Marcus SC. Growth and decentralization of the medical literature: implications for evidence-based medicine. J Med Libr Assoc. 2005;93:499–501.
- Gaudelus J. Vaccination contre le papillomavirus : bilan et nouvelles extensions [Papillomavirus vaccination: assessment and new extensions]. Rev Prat. 2019;69:7–12.
- Haut Conseil de la Santé Publique. Avis relatif à la place du vaccin GARDASIL 9® dans la stratégie actuelle de prévention des infections à papillomavirus humains du 10 février 2017 [Notice regarding the place of GARDASIL 9® vaccine in the current human papillomavirus infection prevention strategy of February 10, 2017] [Internet]. 2017. [cited 2020 Mar 26]. Available from: https://www.hcsp.fr/explore.cgi/avisrapportsdomaine?clefr=602
- ANSM. Vaccins anti-HPV et risque de maladies auto-immunes : étude pharmacoépidémiologique [HPV vaccines and risk of autoimmune diseases: pharmacoepidemiological study] [Internet]. 2015 [cited 2020 Mar 26]. Available from: https://ansm.sante.fr/var/ansm_site/storage/original/application/ea5e12b9c18ae41c2b8163ae5d7cb6f3.pdf
- Uniform requirements for manuscripts submitted to biomedical journals: writing and editing for biomedical publication. J Pharmacol Pharmacother. 2010; 1:42–58. Available from: https://pubmed.ncbi.nlm.nih.gov/21808590/
- Epstein RJ. Six authors in search of a citation: villains or victims of the Vancouver convention? BMJ. 1993;306:765–67.
- Shaffer E. Too many authors spoil the credit. Can J Gastroenterol Hepatol. 2014;28:605.
- McDonald RJ, Neff KL, Rethlefsen ML, Kallmes DF. Effects of author contribution disclosures and numeric limitations on authorship trends. Mayo Clin Proc. 2010;85:920–27.
- Levsky ME, Rosin A, Coon TP, Enslow WL, Miller MA. A descriptive analysis of authorship within medical journals, 1995–2005. South Med J. 2007;100:371–75.
- Hong SJ, Yoon DY, Cho YK, Yoon SJ, Moon JY, Baek S, Lim KJ. Characteristics and quality of radiologic randomized controlled trials: a bibliometric analysis between 1995 and 2014. Am J Roentgenol. 2016;206:917–23.
- Baek S, Yoon DY, Cho YK, Yun EJ, Seo YL, Lim KJ, Choi CS. Trend toward an increase in authorship for leading radiology journals. Am J Roentgenol. 2015;205:924–28.
- Chow DS, Ha R, Filippi CG. Increased rates of authorship in radiology publications: a bibliometric analysis of 142,576 articles published worldwide by radiologists between 1991 and 2012. Am J Roentgenol. 2015;204:W52–7.
- Fontelo P, Liu F. A review of recent publication trends from top publishing countries. Syst Rev. 2018;7:147.
- Winnik S, Speer T, Raptis DA, Walker JH, Hasun M, Clavien P-A, Komajda M, Bax JJ, Tendera M, Fox K, et al. The wealth of nations and the dissemination of cardiovascular research. Int J Cardiol. 2013;169:190–95.
- Rostaing H La bibliométrie et ses techniques [Internet]. 2017 [cited 2019 Nov 28]; Available from: https://hal.archives-ouvertes.fr/hal-01579948/document
- Shuaib W, Khan MS, Shahid H, Valdes EA, Alweis R. Bibliometric analysis of the top 100 cited cardiovascular articles. Am J Cardiol. 2015;115:972–81.
- Brinjikji W, Klunder A, Kallmes DF. The 100 most-cited articles in the imaging literature. Radiology. 2013;269:272–76.
- Jiang Y, Hu R, Zhu G. Top 100 cited articles on infection in orthopaedics: A bibliometric analysis. Medicine. 2019;98:e14067.
- Chen X, Yang K, Xu Y, Li K. Top-100 highest-cited original articles in inflammatory bowel disease: A bibliometric analysis. Medicine. 2019;98:e15718.
- Sinowatz F. The impact of the impact factor. Anat Histol Embryol. 2016;45:159–60.
- Rawat S. How is impact factor impacting our research? Biomed J. 2014;37:415.
- Heneberg P. From excessive journal self-cites to citation stacking: analysis of journal self-citation kinetics in search for journals, which boost their scientometric indicators. PLoS ONE. 2016;11:e0153730.
- Thelwall M, Haustein S, Larivière V, Sugimoto CR. Do altmetrics work? Twitter and ten other social web services. PLoS ONE. 2013;8:e64841.
- Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci. 2005;102:16569–72.
- Grzybowski A, Patryn R. Impact factor: universalism and reliability of assessment. Clin Dermatol. 2017;35:331–34.
- Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298:1002.
- Moses H, Matheson DHM, Cairns-Smith S, George BP, Palisch C, Dorsey ER. The anatomy of medical research: US and international comparisons. JAMA. 2015;313:174.
- Relman AS. Dealing with conflicts of interest. N Engl J Med. 1984;310:1182–83.
- Voineskos SH, Coroneos CJ, Ziolkowski NI, Kaur MN, Banfield L, Meade MO, Chung KC, Thoma A, Bhandari M. A systematic review of surgical randomized controlled trials: part 2. Funding source, conflict of interest, and sample size in plastic surgery. Plast Reconstr Surg. 2016;137:453e–61e.
- Ivanov A, Kaczkowska BA, Khan SA, Ho J, Tavakol M, Prasad A, Bhumireddy G, Beall AF, Klem I, Mehta P, et al. Review and analysis of publication trends over three decades in three high impact medicine journals. PLoS ONE. 2017;12:e0170056.
- Ellegaard O, Wallin JA. The bibliometric analysis of scholarly production: how great is the impact? Scientometrics. 2015;105:1809–31.
- Ramos JM, Gutiérrez F, Masía M, Martín-Hidalgo A. Publication of European Union research on infectious diseases (1991–2001): a bibliometric evaluation. Eur J Clin Microbiol Infect Dis. 2004;23:180–84.
- Duyx B, Swaen GMH, Urlings MJE, Bouter LM, Zeegers MP. The strong focus on positive results in abstracts may cause bias in systematic reviews: a case study on abstract reporting bias. Syst Rev. 2019;8:174.
- Li G, Abbade LPF, Nwosu I, Jin Y, Leenus A, Maaz M, Wang M, Bhatt M, Zielinski L, Sanger N, et al. A scoping review of comparisons between abstracts and full reports in primary biomedical research. BMC Med Res Methodol. 2017;17:181.
- Ward LG, Kendrach MG, Price SO. Accuracy of abstracts for original research articles in pharmacy journals. Ann Pharmacother. 2004;38:1173–77.
- Pitkin RM, Branagan MA, Burmeister LF. Accuracy of data in abstracts of published research articles. JAMA. 1999;281:1110–11.
- Nascimento DP, Costa LOP, Gonzalez GZ, Maher CG, Moseley AM. Abstracts of low back pain trials are poorly reported, contain spin of information, and are inconsistent with the full text: an overview study. Arch Phys Med Rehabil. 2019;100:1976–1985.e18.
- Lehmen JA, Deering RM, Simpson AK, Carrier CS, Bono CM. Inconsistencies between abstracts and manuscripts in published studies about lumbar spine surgery. Spine. 2014;39:841–45.
- Jia P-L, Xu B, Cheng J-M, Huang X-H, Kwong JSW, Liu Y, Zhang C, Han Y, Xu C. Assessment of the abstract reporting of systematic reviews of dose-response meta-analysis: a literature survey. BMC Med Res Methodol. 2019;19:148.
- Beller EM, Glasziou PP, Altman DG, Hopewell S, Bastian H, Chalmers I, Gøtzsche PC, Lasserson T, Tovey D, PRISMA for Abstracts Group. PRISMA for Abstracts: reporting systematic reviews in journal and conference abstracts. PLoS Med. 2013;10:e1001419.
- Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62:e1–e34.
- Gøtzsche PC. Believability of relative risks and odds ratios in abstracts: cross sectional study. BMJ. 2006;333:231–34.
Appendix 1.
Impact factor distribution per country
