https://orcid.org/0000-0002-6363-9202
As scientific output has exponentially increased,Citation1 so too has an interest in its quantification via bibliometric measures. Traditionally, citation counts have been utilized to discern an article’s scientific impact – a larger number of citations is construed to indicate the publication has had a significant influence on its field. However, there are notable shortcomings with this simple measure.Citation2 All citations are considered uniformly important and positive, an assumption that may not be consistently accurate. Furthermore, citation counts cannot capture the novelty or revolutionary nature of published findings.
Intending to quantify such paradigm shifts, Wu and colleagues described and validated the disruption index, a measure of the extent to which an article presents “something new that eclipses attention to previous work upon which it has built.”Citation3 In effect, this metric assesses the degree of innovation an article introduces to its field. An article’s disruption score is scaled from −1 to +1, with scores less than 0 entailing articles that are developmental and score greater than 0 entailing articles that are disruptive. Disruptive articles are defined as those that alter the foundation of a field, whereas developmental articles are defined as those that continue to build upon the foundation.Citation3,Citation4
The formula to calculate the disruption score is detailed in . To illustrate with an example, suppose article A has 100 citations. 70 articles exclusively cited article X, while 30 cited article X and another article that was in its reference list. Additionally, a further 20 articles exist that did not cite article X but cited one of its references. Article X’s disruption score would be calculated as (70–30)/(70 + 30 + 20) = 0.33. Aside from manual calculation, there is a publicly accessible dataset provided by Wu and colleagues that has collated disruption scores for 19,495,814 papers published between 1830 and 2021.Citation3,Citation5
Figure 1. Illustration of the calculation of disruption scores. Papers may exclusively cite the article of interest (a), cite both the article of interest and a referenced article (b), or exclusively cite a referenced article (c). The disruption score is scaled from −1 to +1, with the former indicating developmental papers and the latter indicating disrupting papers.
Investigators have explored the disruption index for publications across multiple medical specialties.Citation6–10 Considering the vast body of medical knowledge, their findings highlight seminal texts that should be useful for both experienced and young scholars in their fields. Interestingly, a substantial proportion of these studies were case series or case reports rather than randomized controlled trials, suggesting smaller-scale studies are often the most disruptive. The authors of this editorial encourage further identification of disruptive texts in other specialties to characterize better the innovation that defines medicine.
The disruption index is not without its weaknesses. One concern is that it identifies publications that may impact a smaller research community but not the broader field.Citation6 This leads to circumstances where an ostensibly disruptive paper may have few citations and hence limited meaningful impact. Furthermore, analogous to citation count, the disruption score evolves for publications throughout their lifetimes and must continuously be reassessed. In general, the measure remains underexplored in medicine, and further investigation would be required to examine its relationship with traditional bibliometrics such as the h-index.Citation11
Despite its flaws, the disruption index provides a unique lens to view scientific research, similar to Altmetric scores.Citation12 Therefore, the authors of this editorial stress that disruption scores are best utilized as an adjunct to other article metrics. Employing multiple perspectives will enable the most comprehensive understanding of a publication’s scientific impact and innovation.
Disclosure Statement
No potential conflict of interest was reported by the author(s).
Additional information
Funding
References
- Bornmann L, Haunschild R, Mutz R. Growth rates of modern science: a latent piecewise growth curve approach to model publication numbers from established and new literature databases. Humanit Soc Sci Commun. 2021;8(1):224. doi:10.1057/s41599-021-00903-w.
- Aksnes DW, Langfeldt L, Wouters P. Citations, citation indicators, and research quality: an overview of basic concepts and theories. Sage Open. 2019;9(1):2158244019829575. doi:10.1177/2158244019829575.
- Wu L, Wang D, Evans JA. Large teams develop and small teams disrupt science and technology. Nature. 2019;566(7744):378–382. doi:10.1038/s41586-019-0941-9.
- Funk RJ, Owen-Smith J. A dynamic network measure of technological change. Manag Sci. 2017;63(3):791–817. doi:10.1287/mnsc.2015.2366.
- Wu L, Wang D, Evans JA. Data from: replication data for: large teams develop and small teams disrupt science and technology. 2021;DRAFT VERSION. Deposited. 2021 March 26;16(28):10.079. doi:10.7910/DVN/JPWNNK.
- Becerra AZ, Grimes CE, Grunvald MW, et al. A new bibliometric index: the Top 100 most disruptive and developmental publications in colorectal surgery journals. Dis Colon Rectum. 2022;65(3):429–443. doi:10.1097/DCR.0000000000002118.
- Grunvald MW, Williams MD, Rao RD, O’Donoghue CM, Becerra AZ. 100 disruptive publications in breast cancer research. Asian Pac J Cancer Prev. 2021;22(8):2385–2389. doi:10.31557/APJCP.2021.22.8.2385.
- Khusid JA, Gupta M, Sadiq AS, Atallah WM, Becerra AZ. Changing the Status Quo: the 100 most-disruptive papers in Urology? Urology. 2021;153:56–68. doi:10.1016/j.urology.2020.10.073.
- Hansdorfer MA, Horen SR, Alba BE, Akin JN, Dorafshar AH, Becerra AZ. The 100 most-disruptive articles in plastic and reconstructive surgery and sub-specialties (1954-2014). Plast Reconstr Surg Glob Open. 2021;9(3):e3446–e3446. doi:10.1097/GOX.0000000000003446.
- Becerra AZ, Aquina CT, Hayden DM, Torquati AF. The top 100 most disruptive publications in academic surgery journals: 1954–2014. Am J Surg. 2021;221(3):614–617. doi:10.1016/j.amjsurg.2020.07.037.
- Ali MJ. Forewarned Is Forearmed: The h-Index as a Scientometric. Semin Ophthalmol. 2021;36(1–2):1. doi:10.1080/08820538.2021.1894889.
- Ali MJ. Understanding the Altmetrics. Semin Ophthalmol. 2021;36(5–6):351–353. doi:10.1080/08820538.2021.1930806.