Measuring the social impact of contemporary dysphagia research: an altmetric analysis

ABSTRACT

This study aimed to (a) identify characteristics of dysphagia research receiving online attention; (b) determine associations between altmetric scores and traditional research metrics; (c) establish differences in altmetric scores between open access and closed access research and funded and non-funded research. Altmetric Explorer was searched on 7th October 2019 to identify contemporary (January 2014 to January 2019) articles with the keyword ‘dysphagia’. Data from 100 articles with the highest Altmetric Attention Score (AAS) were exported for analysis. Data extracted included journal name; first author profession; country; study design; population studied; publication year; journal impact factor; citations; downloads; funding and access status. Most of the 100 articles (AAS 19–317) focused on adult populations (95%). Nearly half of study designs were systematic reviews (27%) or randomized control trials (18%). The Dysphagia journal published the most articles (34%) and nearly one-third of first authors (31%) were based in USA. The most studied population was neurological (30%). There was no association between altmetric scores and traditional metrics. A significant difference in altmetric scores (U = 650.50, p = 0.045, p < 0.05) was found between the earlier time-period (2014–2016) (median AAS = 29) and later time-period (2016–2019) (median AAS=36). A significant difference in altmetric scores was identified between open (median = 33) and non-open access research (median = 29) (U = 1030.50, p = 0.048). Altmetric scores provide an innovative article level metric capturing public interest in dysphagia research. As altmetric scores do not correlate with traditional metrics, improved understanding of the type of dysphagia research that has social impact is imperative to guide researchers and clinicians.

KEYWORDS:

• Dysphagia
• swallowing
• altmetrics
• social impact
• research metrics

Introduction

Research excellence is vital to inform policy, improve clinical practice, and influence university hiring, promotion, and funding. Traditionally, significant research has been identified using bibliometrics including citation counts, h-indices, journal impact factor (JIF) and usage statistics (e.g., number of downloads), which evaluate research impact within publishing and academic communities (Gunn, 2013). However, these measurements accumulate slowly; the ‘citation latency’ of published articles can be several years, even for those with high impact (Wang, 2013). They measure impact among a narrow audience, essentially only capturing readership within the academic community, and fail to reflect research impact on other sections of society (Hammarfelt, 2014). Conventional metrics are susceptible to manipulation; authors may cite their own research or groups of researchers may excessively cite each other’s work (Masic, 2013). Research may also be cited in a negative light or in a way that does not reflect intellectual influence (Bornmann & Daniel, 2008). Therefore, citation counting alone may no longer be the most effective means to assess research impact, especially in the current context of ever-expanding digital publishing.

Altmetrics (short for alternative metrics) measure both the volume and nature of online attention surrounding research findings. The breakthrough of altmetrics in 2010 has provided a measure of societal research impact, by quantifying the online attention generated by an individual article. Altmetrics are based on the number of mentions an article receives on various online sources including social media platforms (Twitter, Facebook), Wikipedia, Google⁺, mainstream news outlets, scientific blogs, Pinterest, YouTube, policy documents and reference managers (e.g., Mendeley) (Melero, 2015). These sources allow the impact of scholarly work and the level of public engagement in the wider society to be assessed, no longer restricting impact measurement to academia (Bornmann et al., 2019). Altmetrics have a key advantage in accumulating in real time, allowing for an almost instantaneous measurement of impact, as opposed to the delay in citation accumulation (Warren et al., 2017). Also, subjects more relevant and relatable to the public have been found to have an edge in accumulating greater online attention, compared to traditional scientific articles (Barbic et al., 2016). Altmetrics, therefore, have the valuable potential to gauge the research that the wider public, including patients and families, clinicians, news broadcasters and policy makers are most interested in. Altmetric information is also increasingly being considered in decisions for the awarding of tenure, university promotions, hiring and funding, (Khodiyar et al., 2014). Furthermore, altmetrics hold a vital advantage in the debate surrounding open access research (Cintra et al., 2018). Open access (OA) articles (free of charge, copyright, and licensing restrictions) have the potential to reach a much wider audience beyond academia which can create greater online research impact (Shema et al., 2014). Sharing research through social media platforms allows both readers and researchers to avoid issues regarding data restriction and pay walls inhibiting public access (Holmberg et al., 2019). Research investigating the influence of OA on altmetrics is beginning to emerge and has currently demonstrated the existence of an OA Altmetric Advantage, i.e., OA articles garner greater online attention (Cintra et al., 2018; Holmberg et al., 2019). This gives a clear indication of the value that social media can also provide for those in dysphagia research, both in promoting work but also in interacting with a greater and more varied audience.

As dysphagia research being shared online is much more accessible, it has clear potential to influence clinical practice. It is therefore imperative to determine the quality and nature of articles being shared online across society. Numerous healthcare disciplines including dentistry, nursing, radiology, oral cancer, emergency medicine, orthodontics and stroke have already explored the characteristics of research within their speciality with social impact (Barbic et al., 2016; Dardas et al., 2018; Delli et al., 2017; Hassona et al., 2019; Kim et al., 2019; Livas & Delli, 2018; Rosenkrantz et al., 2017). Through investigating differences in trends between alternative and citation metrics within their discipline, these studies, to date recommend that traditional citation metrics and altmetrics should be viewed as complementary analysis tools to be used side-by-side. To remain advanced in a growing digital age, it is pertinent that we assess to what extent altmetrics can measure the broader impact of dysphagia research into the mainstream. An understanding of the type of dysphagia research being shared online and what platforms it is being shared on is imperative. Furthermore, an exploration into the factors influencing the altmetric scores of dysphagia research is needed, in order to gain insight into the potential meaning and value altmetrics hold for future research in this area.

This study aims to (a) identify the characteristics of contemporary dysphagia articles that received the most online attention; (b) determine any association between the traditional research metrics and the altmetric scores assigned to these dysphagia articles and (c) establish any differences in altmetric scores between articles that are open access versus pay-for-access or between funded research versus non-funded research.

Methods

Search engine

Altmetric Explorer (Altmetric, London, UK) is a web-based application which was used to search sources of dysphagia research output, to yield the most-relevant and up-to-date picture of the online activity and discussion generated by these articles. Altmetric Explorer has been described in previous literature as the most comprehensive service for monitoring alternative metrics (Haustein et al., 2015; Robinson-Garcia et al., 2014). This company track, in real time, the online attention published material receives on the following sources (Figure 1); (‘Altmetric. Our sources’, 2019).

Figure 1. The online platforms used to share, discuss, and engage in research. The online attention published material receives on these platforms is tracked, in real time, by Altmetric Explorer, to produce alternative metrics detailing the nature and extent of the online attention.

Search strategy

A systematic search of Altmetric Explorer (Altmetric, London, UK) was conducted on 7 October 2019, using the search term ‘dysphagia’ to identify the top 100 articles with the highest AAS discussing any topic specific to dysphagia. Dysphagia’ was selected as the sole search term. The search engine, Altmetric Explorer, does not permit a search with multiple terms. Thus, the search was restricted to a single word (i.e., dysphagia). Furthermore, pilot searches established that articles generated under alternative terms such as ‘swallowing’ or ‘deglutition’ yielded unsuitable articles (e.g., sword swallowing) which did not discuss swallowing from a medical perspective. Authors decided not to focus exclusively on oropharyngeal dysphagia and to include oesophageal dysphagia for the purpose of this study. This was due to the growing awareness of the strong inter-relationship between all phases of swallowing and the increasing number of research studies which include both oropharyngeal and oesophageal phases of swallowing.

The decision to analyse 100 articles was in keeping with altmetric studies in other healthcare disciplines, for example, nursing, dentistry, and oral cancer research (Dardas et al., 2018; Hassona et al., 2019; Kim et al., 2019). The decision to analyse the top 100 articles regardless of the publishing journal was to ensure that dysphagia research published in journals pertaining to other disciplines such as nursing, medicine, and dentistry, were not overlooked. To capture contemporary research, a timeframe of five years (2014–2019) was established. An upper time limit of January 2019 was selected to allow sufficient citations to accumulate while also accounting for the significant growth of social media in recent times (Haustein et al., 2015).

Inclusion criteria for studies included (1) English studies or studies with English translations, (2) full text, original and review articles (3) publications between January 2014 and January 2019. Exclusion criteria included (1) conference abstracts (2) meeting reports (3) editorials (4) errata (5) research in progress.

From the top 100 articles identified within the selected timeframe with the highest AAS, data regarding the following was extracted: article title, AAS, year of publication, journal title, number of citations, number of downloads, first-author country of origin, and first author speciality.

An article is assigned a weighted score using an automated algorithm, referred to as the Altmetric Attention Score (AAS), based on the different platforms that mention it (Davies, 2015). The weight of a mention type corresponds to the relative reach and value of the online source. For example, news articles receive greater weight than blog posts, which have greater weight than Twitter or other social media posts, due to audience size and the level of attention generated by that score. The AAS is displayed in the centre of the ‘Altmetric ‘donut’, made up of the colours that reflect the sources and quantity of online attention, (Davies, 2015) (see Figure 2).

Figure 2. The Altmetric Donut – The sources monitored by Altmetric Explorer are listed in the above figure accompanied by their representative colour. To the right, the altmetric donut depicts the sources (colour of stripes) and the frequency of mentions (thickness of the stripes) with the altmetric attention score located in the centre of the donut.

Furthermore, information was gathered in relation to the online platforms contributing the most to the AAS of each article, study designs of articles, clinical population and age group studied, (adults or paediatrics) as well as open research characteristics including article access (i.e., open access versus subscription requirement) and study funding, (i.e., whether the study was funded or not).

Details were also extracted regarding the geographic dissemination of individual’s sharing dysphagia research and the breakdown of mentions on news, blogs, policy documents, Twitter, Facebook, Wikipedia and so on, for each article.

Information pertaining to article title, AAS, year of publication, journal title, geographic dissemination and the number of mentions received on the various online platforms listed above, was obtained from the Altmetric Explorer webpage. On the webpage, electronic links are provided to the individual web pages for each article. On following these links, the article content was manually reviewed to determine the study design, research topic, article access type, study funding, first author speciality and first author country of origin.

Citation data for each article was also obtained from Altmetric Explorer. Data regarding the number of downloads was retrieved from ‘ScienceDirect.Com’ (2019) within the period from October to November 2019. The JIF for each journal identified was accessed via Incites Journal Citation Reports (2019).

Data analysis

Information selected for analysis was exported onto an electronic spreadsheet. Analysis was carried out with IBM SPSS Statistics 25 (SPSS, Chicago, IL, USA). Descriptive statistics (medians, frequencies, and ranges) were generated to detail the characteristics of the factors contributing to altmetric scores. The Mann–Whitney U test was used to investigate differences in AASs between articles published in early years (2014–2016) and more recent years (2017–2019) and to investigate difference among AASs and articles that were OA or pay-for-access and articles that were funded or not funded. Associations between altmetrics and traditional metrics (i.e., JIFs, downloads, citation counts) were analysed using Spearman’s correlation coefficient. Correlation strength was interpreted using Munro’s (2005) correlation parameters as follows; 0.00–0.25 (little if any); 0.26–0.49 (low); 0.50–0.69 (moderate); 0.70–0.89 (high); 0.90–1.0 (very high). A P value < .05 was considered statistically significant with confidence intervals taken at 95%. Non-parametric tests were used due to the skewed nature of AAS data set.

Results

Search output

The search word, ‘dysphagia’ yielded 1740 outputs in Altmetric Explorer published during the period from January 2014 to January 2019. The dysphagia articles with top 100 AASs were selected. Supplementary Tables 1 and 2 summarize the title, AAS, publication year, publishing journal, JIF, number of citations, number of downloads, country of origin and profession of first author, study design, clinical populations, access type, funding status and social media account contributing the greatest to the AAS for the 100 articles analysed.

Characteristics of the contemporary dysphagia research with the top 100 AASs

Of the 100 articles identified, the majority were published in 2018 (n = 35), followed closely by 2017 (n = 29) (Figure 3). A statistically significant difference in AAS was seen in articles published in the latter half of study time frame (2017–2019) compared to the earlier half of the time period (2014–2016), (U = 650.50, p = 0.045). Based on the descriptive statistics, articles published in the latter half of study time frame (2017–2019) had higher median AASs (median = 36; range = 19–317) compared with articles published in the earlier half of the time period, (2014–2016) (median = 29; range = 20–110). First authors were based in twenty different countries of origin. 31% originated from the USA, (AAS median = 34, range from 20 to 203), followed by 18% from Australia (AAS median = 28, range from 19 to 146), 10% from the United Kingdom (AAS median = 28, range 25–108) and 7% from Canada, (AAS median = 30, range 19–77).

Figure 3. Distribution of the year of publications for the 100 dysphagia articles with the highest altmetric attention scores analysed in this study.

The 100 articles analysed were published in 51 different journals belonging to various healthcare disciplines (see supplementary Table 3). Dysphagia published the greatest quantity of articles (n = 34; median AAS = 28; range = 19–203), followed by Stroke (n = 7; Median AAS = 29; range = 21–77), Clinical Interventions in Ageing (n = 5; median AAS = 36; range = 21–38), and BMC Cancer (n = 4; median AAS = 28; range = 19–29). Table 1 highlights the top 10 publishing journals (those that published the greatest number of articles of the 100 selected), including their number of mentioned outputs, median and range AAS, and JIF (based on the most recent published impact factors, for the year 2018).

Table 1. Characteristics of the top 10 journals which published the greatest number of articles in descending order.

Journal title	Mentioned outputs	AAS median (min/max)	Journal impact factor (for 2018)
Dysphagia	34	28 (19–203)	3.034
Stroke	7	29 (21–77)	6.046
Clinical Interventions in Ageing	5	36 (21–38)	2.585
BMC Cancer	4	28(19–29)	2.933
ASHA Leader	3	28(27–52)	0.1
Nature Reviews Gastroenterology and Hepatology	3	21(20–50)	23.57
International Journal of Speech Language Pathology	2	33(21–45)	1.28
American Journal of Speech Language Pathology	2	35.5 (29–42)	1.321
Diseases of the Oesophagus	2	52.5 (33–72)	2.323
BMC Geriatrics	2	119.5 (53–186)	2.818

Regarding study designs used, the majority were systematic reviews (n = 27; median AAS = 29; range = 19–146), followed by randomized control trials, (n = 18; median AAS = 32; range = 20–318), cohort studies (n = 14; median AAS = 34; range = 20–100) and cross-sectional design (n = 11; median AAS = 34.5; range = 25–58). A full breakdown of study design can be seen in Figure 4.

Figure 4. Percentage breakdown of the study designs of the dysphagia articles with the top 100 altmetric attention scores selected for analysis in this study.

Speech and language therapists were the first authors of 47% of articles, (median AAS = 29; range 19–203) and a medical doctor was the first author of 43% of articles (median AAS = 29; range = 20–318). The remaining professions of first authors contributing to 10% of the overall score included Occupational Therapists, Dietetics, Biomedical Scientists, Nurses, Clinical Psychologists, Physiotherapists and Pharmacists. The most common clinical population studied was neurology (n = 30; median AAS = 28; range = 19-318), followed closely by a mixed clinical population i.e., articles involving heterogeneous clinical populations (e.g., stroke, head and neck cancer and dementia), (n = 26; median AAS = 30; range = 19–146) and then older populations (n = 12; median AAS = 24.5; range = 20–186). The full breakdown of the populations identified is illustrated in Figure 5.

Figure 5. Percentage breakdown of the different clinical populations studied in the 100 dysphagia articles with the highest almetric attention scores selected for analysis in this study.

Of these 100 articles, 95% were based on topics regarding adult swallowing (median AAS = 29; range 19–318), while 5% dealt with paediatric swallowing issues, (median AAS = 34; range = 29–100).

Sources contributing to the altmetric attention scores received by the 100 articles

The median AAS obtained by the 100 dysphagia articles was 29 with a range from 19 to 317. Table 2 highlights the median and ranges of AASs obtained by altmetric studies in other healthcare disciplines as a benchmark by which to compare the results of this study.

Table 2. ASS median and ranges obtained by altmetric studies in other healthcare disciplines. These scores provided a comparison for the results obtained in this study (AAS = Altmetric Attention Score).

Discipline	Median AAS	ASS ranges	Methodology	Journals Included
Dysphagia	29	19–317	Top 100 articles with highest AAS	Any journals with dysphagia articles
Nursing (Dardas et al., 2018)	248	138–649	Top 100 articles with highest AAS	Nursing journals only
Dentistry (Delli et al., 2017)	119	69–1307	Top 100 articles with highest AAS	Dental journals only
Orthodontics (Livas & Delli, 2018)	8	5–196	Top 200 articles with highest AAS	Orthodontic journals only
Stroke (Kim et al., 2019)	657	454–3215	Top 50 articles with highest AAS	Neurology journals only
Radiology (Rozenkrantz et al., 2017)	0	0–264	All radiology journals published in 2013	Radiology journals only
Oral Cancer (Hassona et al., 2019)	455	21–1307	Top 100 articles with highest AAS	Any journals with oral cancer articles

Twitter contributed the greatest to the AAS of most articles, (n = 77 – number of articles which twitter mentions made up the majority of the AAS; median twitter mentions per article = 45; range 0–223), followed by news outlets (n = 12 – number of articles which news mentions made up the majority of the score; median news stories per article = 10; range = 0–38), and then Facebook, (n = 9; median = 2; range = 0–2). Table 3 illustrates the full online attention breakdown per media source for all articles analysed.

Table 3. Descriptive statistics for mentions per media platform for the articles analysed in the present study (statistics generated by Altmetric Explorer).

Source	Median mentions	Range	Total mentions	Percentage of articles with at least one mention
News mentions	10	0–38	193	34%
Blog mentions	0	0–2	23	20%
Policy statements	0	0–3	3	3%
Tweets	45	0–223	3916	99%
Peer review	0	–	0	–
Facebook	2	0–61	398	78%
Wikipedia	0	0–4	8	4%
Google+	0	0–3	18	13%
LinkedIn	0	–	0	–
Mendeley readers	44	0–331	5809	100%
CiteULike	0	0–1	3	3%
F100 mentions	0	0–1	1	1%
Reddit mentions	0	0–1	1	1%
Q & A threads	0	–	0	–
Videos	0	–	0	–
Syllabi mentions	0	–	0	–

Although news outlets contributed the greatest to the AAS of only 12 articles, overall, 34 articles received at least one mention on a news outlet. Of these 34 articles, the majority were RCTs (17 articles), and mainly discussed neurology populations (11 articles) followed by mixed clinical populations (7 articles). Table 4 describes the 12 articles in which news outlets contributed to the majority of the article’s AAS.

Table 4. Articles in which news outlets contributed to the majority (minimum 50%) of their altmetric attention score (AAS).

Article title	AAS	Citation score	Access	Funding
Pharyngeal electrical stimulation for early decannulation in tracheotomised patients with neurogenic dysphagia after stroke (PHAST-TRAC): a prospective, single-blinded, randomized trial. (Dziewas et al., 2018)	318	11	Pay-for-Access	Funded
Volumetric Changes to the Pharynx in Healthy Aging: Consequence for Pharyngeal Swallow Mechanics and Function. (Molfenter, Lenell, & Lazarus, 2019)	203	6	Pay-for-Access	Funded
Assessing Hyolaryngeal Excursion: Comparing Quantitative Methods to Palpation at the Bedside and Visualization During Videofluoroscopy. (Brates, Molfenter, & Thibeault, 2019)	200	1	Pay-for-Access	Funded
Short-Term Effects of Cold Liquids on the Pharyngeal Swallow in Preterm Infants with Dysphagia: A Pilot Study. (Ferrara et al., 2018)	100	0	Pay-for-Access	No funding
Videofluoroscopic Validation of a Translational Murine Model of Presbyphagia. (Lever et al., 2015)	74	17	Pay-for-Access	Funded
An abuse-deterrent, microsphere-in-capsule formulation of extended-release oxycodone: alternative modes of administration to facilitate pain management in patients with dysphagia. (McCarberg et al., 2016)	73	8	Pay-for-Access	Funded
Economic and survival burden of dysphagia among inpatients in the United States. (Patel et al., 2017)	72	16	Pay-for-Access	No Funding
Dysphagia after Occipitocervical Fusion. (Hong & Lim, 2017)	60	9	Pay-for-Access	No funding
The Physiology of Eructation. (Lang, 2016)	60	5	Pay-for-Access	No Funding
Utility of Pulse Oximetry to Detect Aspiration: An Evidence-Based Systematic Review. (Britton et al., 2018)	53	3	Pay-for-Access	No funding
Effects of expiratory muscle strength training on oropharyngeal dysphagia in subacute stroke patients: a randomized controlled trial. (Park, Oh, Chang, & Kim, 2016)	48	20	Pay-for-Access	No Funding
The effects of a xanthan gum-based thickener on the swallowing function of patients with dysphagia. (Rofes, Arreola, Mukherjee, Swanson, & Clavé, 2014)	33	45	Open Access	Funded

Three of the 100 articles were mentioned in policy documents. These articles are described in Table 5.

Table 5. The three articles mentioned in policy during the selected study time frame (January 2014 to January 2019), including altmetric attention scores (AAS), citation data, access, and funding type.

Article title	Policy source	AAS	Citation data	Access	Funding
The effects of a xanthan gum-based thickener on the swallowing function of patients with dysphagia. (Rofes et al., 2014)	World Health Organisation (WHO)	33	45	Open Access	Funded
Survivors’ experiences of dysphagia-related services following head and neck cancer: Implications for clinical practice. (Nund et al., 2014)	The National Institute for Health and Care Excellence, UK	30	26	Open Access	Not Funded
‘Wet Voice’ as a Predictor of Penetration and Aspiration in Oropharyngeal Dysphagia. (Warms & Richards, 2000)	ASHA Practice Policy	26	56	Pay-for-Access	Funded

The timeline shown in Figure 6 (generated by Altmetric Explorer), illustrates articles mentions from January 2014 to January 2019, grouped by specific major online attention sources. Twitter was the leading source of online mentions for dysphagia research throughout the four-year period.

Figure 6. Mentions received by the 100 dysphagia articles with the highest altmetric attention scores, from January 2014 to January 2019 grouped by online source of attention (see representative colours for each source below the graph). This figure highlights that Twitter contributes the greatest towards online mentions of dysphagia research within the selected study timeframe.

Most online attention originated from the USA and the UK. Table 6 highlights the main worldwide locations contributing to the attention received by the dysphagia articles published between January 2014 to January 2019, in news stories, tweets, Facebook posts and policy documents (information generated by Altmetric Explorer).

Table 6. The major geographic locations for online attention received by the top 100 articles (statistics generated by Altmetric Explorer).

Country	Total news stories	Total tweets	Total Facebook posts	Total policy documents
Country not specified	n/a	3817 (34.9%)	877(65.1%)	n/a
USA	172(66.9%)	2,183 (19.9%)	228(16.9%)	51(51.5%)
United Kingdom	20(7.8%)	1,976 (18%)	20(1.5%)	44(44.4%)
Australia	23(8.9%)	603 (5.5%)	10(0.7%)	0
Ireland	0	434 (4%)	0	0
Spain	1(0.4%)	391(3.6%)	5(0.4%)	0
Japan	0	353(3.2%)	0	0
Canada	1(0.4%)	203(1.9%)	91(6.8%)	0
Germany	4(1.6%)	85(0.8%)	1(0.1%)	1(1%)
New Zealand	7(2.7%)	63(0.6%)	13(1%)	1(1%)
Singapore	14(5.4%)	0	43(3.2%)	0
Finland	8(3.1%)	9(0.1%)	4(0.3%)	0

Altmetrics; associations with traditional impact metrics

A correlation analysis was conducted to investigate the relationship between AASs and traditional research impact metrics. No statistically significant correlations were identified between article citation counts (median = 7; range = 0–133) and their altmetric scores (r = .058, p = 0.19). Similarly, there was no correlation between JIF (median = 3.034; range = 0.1–70.67) and altmetric scores (r = .063, p = .536). Finally, no association was observed between number of article downloads (median = 1285; range = 529–45,000) and altmetric scores (r = .108, p = .105). Altmetric information and traditional metric characteristics are provided for all 100 articles in supplementary Table 1.

Open access and funding

57% of the 100 articles were funded (median AAS = 30; range = 19–317) and 43% were not funded (median AAS = 27; range = 20–186). No significant difference was found in AASs between funded or non-funded research, (p > 0.05). Regarding article access type, 62% were pay-for-access (median AAS = 29; range = 19–317), and 38% were open access (median AAS = 33; range = 19–186). A significant difference was identified in AASs between articles that were open access or pay-for-access, (U = 1030.50, p = 0.048).

Discussion

This is the first study to report on altmetrics as an impact measure in the context of dysphagia literature and to provide an evaluation of the dysphagia articles which garnered the greatest online attention within the selected timeframe.

Regarding the online platforms monitored by Altmetric Explorer, Twitter was clearly the social media outlet on which articles were most likely to be mentioned, operating as the strongest contributor in 77% of the articles’ overall altmetric scores. In fact, 99% of the articles had at least one mention on Twitter. This finding is unsurprising considering the rising use of Twitter as a method of research promotion for authors. Twitter was also found to contribute the most to AASs in oral cancer, nursing, and orthodontic studies (Dardas et al., 2018; Hassona et al. 2019; Livas & Delli, 2018). Research by Costas et al. (2015), showed that academic work in general receives greater attention on Twitter than other media platforms. Piwowar (2013), also states that approximately 1 in 40 scholars have an active Twitter account. This finding gives dysphagia researchers an indication of the applicability and benefits of Twitter as a research tool for promoting and increasing engagement in their work. Similarly, this finding may encourage clinicians to use Twitter as a method of discovering the most up-to-date dysphagia research that is being discussed online.

Thirty-four articles had at least one mention on news outlets (see Table 4). These results can be interpreted as having a greater significance due to news outlets’ ability to reach individuals who may not be as social media orientated and may not have personal accounts on social media platforms. News outlet, therefore, hold an advantage in diversifying and expanding the audience exposed to dysphagia research.

Three articles were mentioned in policy documents (see Table 5). This finding is important, as policies possess the ability to incite changes and improvements, particularly from the perspective of essential services and standards within dysphagia specific healthcare provision. The fact that only three of one hundred articles are included in policy within the 4-year study timeframe, is very low. Furthermore, considering many of these research articles are funded and are shared worldwide, it is interesting that so many are falling short of inclusion in policy. Efforts should be made to increase the amount of evidence-based research within policy to promote meaningful changes to service delivery and patient care.

The USA was the most common country of origin of first authors followed by Australia, and the UK. Also, dysphagia research was featured in news stories and policy documents in high income countries only (i.e., USA, UK, Australia, and Canada). This is in line with the demographic dissemination of research found in nursing, dentistry, and oral cancer studies (Dardas et al., 2018; Delli et al., 2017; Hassona et al., 2019). This information provides an insight into the current geographic locations generating the most online interest in dysphagia research. Also, it appears that there is a potential discrepancy between lower versus higher income countries and their accessing of online research. Considering the worldwide nature of scientific discovery today and the need for up-to-date clinical evidence in all sectors of the world, it is revealing that the online attention received by dysphagia research and the authors of this research, for the most part, appears to be confined to areas of greater economic and educational advantage. However, consideration must be given to the barriers to research access faced in low-income countries. Factors contributing to limited online engagement in low-income countries include lower levels of education, limited internet and technology access. Furthermore, research, especially from high income countries, involves costly instrumentation or discusses topics which may not be relevant to their everyday cultural or socioeconomic problems, (Acharya & Pathak, 2019). However, discrepancies in research access between low and high-income countries based on our results, must be interpreted cautiously. As highlighted previously in Table 6, a significant percentage of tweets and Facebook post were from unspecified countries, which are potentially low-income countries. This potentially highlights Facebook and Twitter as important tools for research engagement in low-income countries.

Articles focusing on neurological populations were the most common (30%) followed closely by mixed clinical populations (26%). This information gives an indication of the most prevalent dysphagia topics sparking public and clinician interest. It is unsurprising that neurological populations receive the most social attention given the worldwide awareness and prevalence of neurological conditions such as stroke. Neurological conditions are well recognized to cause dysphagia and have they have been the subject of dysphagia research for several decades. As a result, there may be more clinicians working in these services. In contrast, there may be fewer clinicians working in emerging research areas such as critical care, post-extubation dysphagia or with rarer conditions who are seeking access to the evidence base. Articles focusing on mixed clinical populations may receive high levels of attention online as they often describe or review dysphagia interventions applicable across many clinical populations e.g., articles on naso-gastric tube feeding or thickening fluids.

Although the journal Dysphagia published the greatest number of articles, 51 journals from various health sectors and topics were identified within the top 100 articles. This highlights the wider awareness of dysphagia and the health complications it poses as well as the use of dysphagia research across many healthcare disciplines. As clinicians, we seek to increase awareness among fellow members of the multidisciplinary team regarding dysphagia and its effect on the health and well-being of our clients. Increasing awareness among other healthcare disciplines has the potential to transfer reliable dysphagia information to a wider client group and the public with potential positive clinical and public education implications as a result.

Systematic reviews followed by randomized control trials were the predominant study designs used in the top 100 dysphagia articles receiving the highest online attention. These results contrasted with previous altmetric studies in nursing, oral cancer and dentistry (Dardas et al., 2018; Delli et al., 2017; Hassona et al., 2019), which found narrative (unsystematic) reviews generated the greatest online attention. These studies hypothesized that the potential applicability of research in everyday life was more important to the public than the scholarly importance or the low scientific evidence offered by the study design itself. Study designs act as a measure of research quality, and it is important to identify whether it is high quality dysphagia research being shared online or unreliable, low quality information. It is a significant finding, therefore, from both a research and clinical practice perspective, that of the top 100 dysphagia articles receiving the greatest online attention, most use a study design of high scientific quality, reliability, and validity. As per altmetric research in other healthcare disciplines, study quality was evaluated based on research design. However, researchers suggest that validated tools such as the ROBINS-2 and Cochrane Risk of Bias Tool should be used to evaluate methodological quality of studies in future altmetric research.

This study found no statistically significant correlation between AASs and the traditional research metrics (citations, JIF, or the number of downloads) received by the articles analysed. This is in keeping with the findings of altmetric studies in oral cancer, dentistry, and radiology research (Delli et al., 2017; Hasonna et al., 2019; Rosenkrantz et al., 2017). Contrary to these findings, the AASs of nursing and emergency medicine articles were significantly associated with citation counts (Barbic et al., 2016; Dardas et al., 2018). On interpreting these results, it is essential to consider factors such as sample size, altmetric provider, year of publication, and web source used in the studies which can cause differences in AAS associations with traditional metrics. Also, AAS differences have previously been highlighted among different disciplines and between speciality and non-specialty journals (Barbic et al., 2016), and should be considered when assessing altmetrics and associations with traditional metrics. It is important to remember that altmetrics is in its infancy, and research published before the explosion of social media is more likely to be under-represented in terms of AASs but may have much higher citation counts. Thus, it is necessary to interpret correlation results with caution and modesty at this early stage. Furthermore, it is important to acknowledge that each metric plays a unique role in measuring research impact. Altmetrics can be viewed as having a ‘disseminative impact’ while more traditional metrics have an ‘academic impact’ (Trueger et al., 2015). However, neither can describe the complete picture of scholarly use alone (Bornmann, 2014). As a result, it is important to approach this study with the view that AASs enhance rather than substitute conventional metrics in assessing the impact of dysphagia research from the moment of publication.

Study results revealed a statistically significant difference between article access type and the AAS assigned to articles. These results are in line with a large body of recent literature which has indicated that OA articles gather greater online attention, to the extent that OA Altmetric Advantage has become a common term (Holmberg et al., 2019; Hua et al., 2016). A study by Maggio et al., (2018) found that OA articles in health care journals received greater online attention than pay-for-access articles. Our findings are important in terms of the potential advantage of OA articles to future dysphagia research. OA dysphagia articles are available to a wider audience, beyond academia and clinicians, potentially generating greater societal impact which may then be reflected in a higher altmetric score (Holmberg et al, 2019). From both a research and clinical perspective, SLTs are continually seeking to interact directly with their clients and the wider public to boost awareness and self-management of dysphagia related health issues. As a result, researchers should strive to publish open access dysphagia articles where possible as they have the potential to generate greater awareness, education and make a larger impact in the wider community on important issues around swallowing difficulties. Based on study findings, the authors support and encourage the move by researchers, journals, and universities to publish open access articles to increase engagement among a greater and more varied audience who may previously have been restricted by pay walls.

The findings of this research have applications for the public, clinicians, researchers, and academic institutions. The increasing availability of dysphagia research shared on social media and through open access articles allows clinicians, patients, and their families to access and share reliable information that they may previously not have had access to. This is particularly relevant in dysphagia research, where clinician engagement is needed to evaluate research efficacy in real life practice, and education of the public is required to increase awareness and quality of life for those with dysphagia. The altmetric findings of this study provides a means of capturing the nature and trends in research engagement by those outside of academic communities. From an academic perspective, the significance of altmetrics is evident its acceptance as an impact measurement by publishers, academic institutions, and funders for hiring, promotions, appraising academic performance, establishing editorial policies, and processing grant applications, (Powell et al., 2018). For universities, altmetrics are increasingly being used in the ranking of third level institutions, which is an added incentive to incorporate their use into everyday university practice. As a result, the altmetric findings for dysphagia research in this study can be used to inform such decisions made by publishers, universities and funders working in the field of dysphagia. For researchers, altmetrics can provide immediate and accurate feedback on the dissemination and impact of their studies. Piwowar and Priem (2013) state that scholars are now including altmetric scores, as well as citation counts, in their CVs to strengthen their case for promotion or a grant. This study, therefore, provides an insight to dysphagia researchers into the advantage altmetrics holds for their work. Researchers can use the results from this study to inform their decisions on effective methods and platforms for research promotion. Furthermore, the study’s findings regarding open access articles highlight to researchers the benefits of publishing articles without pay wall restrictions where possible, to the increase the online attention of their work.

Limitations

‘Dysphagia’ was the sole search term used for the study since the search terms ‘swallowing’ and ‘deglutition’ within Altmetric Explorer yielded many inappropriate articles with no clinical relevance or reference to a swallowing difficulty. However, it must be acknowledged that by using only ‘dysphagia’ for the search criteria and without an extensive examination of all articles generated under keywords such as ‘swallowing’ or ‘deglutition’, potentially relevant research may have been omitted.

A limitation can also be identified in the constantly changing nature of social media which causes AASs assigned to articles to fluctuate overtime, (for example, if a tweet that mentions an article is deleted), (Livas & Delli, 2018). It is imperative to acknowledge the effect of time on altmetrics combined with the dynamic nature of social media when comparing alternative and traditional metrics. Correlations between the two metrics can quickly be eliminated or reversed at various times, even within the same year (Thelwall et al., 2013).

Directions for future research

Based on the findings of this study, there are two issues which warrant further research.

Firstly, research is required to investigate what social media platforms the public are most inclined to use to access evidence-based information. This will provide direction for researchers and publishers regarding target platforms for disseminating up-to-date evidence-based research. This will provide the potential to reach a wider target audience outside of academia.

Further research is also required to establish ways to increase the level of dysphagia research gaining news attention and influencing policy, which, based on our findings, is low. Considering the wide and varied audience available to news media, and the importance of policy for change and improvement of services, it is imperative that we increase the level of evidence-based dysphagia research gaining attention on news outlets and informing policies.

Conclusion

Altmetrics are set to become a widely used measure of social research engagement and will inevitably grow alongside the exponentially increasing popularity of online media. As professionals within the field of dysphagia strive to have their research reflected in clinical practice, policy, scholarship, funding, promotion and an overall improvement in patient quality of life, it is essential to remain up to date with the most effective methods of disseminating their work. This study provides dysphagia researchers and clinicians with an insight into the relevance of alternative metrics within the field of dysphagia and a means of benchmarking their progress as they begin to adopt this new impact measure.

Disclosure statement

No potential conflict of interest was reported by the author(s).