Scientific Evidence of Acupuncture for Post-Stroke Cognitive Impairment: An Overview of Systematic Reviews and Meta-Analyses

Abstract

Background

Acupuncture may be effective to treat post-stroke cognitive impairment (PSCI). We aimed to evaluate the reliability of the systematic reviews/meta-analyses (SRs/MAs) evidence regarding acupuncture treatment of PSCI.

Methods

The methodological quality was appraised with Methodological Quality of Systematic Reviews 2 (AMSTAR-2). We evaluated reporting quality with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and evidence quality with Grade of Recommendation, Assessment, Development and Evaluation (GRADE) system.

Results

The inclusion criteria were met by fifteen reviews. All studies assessed by AMSTAR-2 had critically poor methodological quality due to the limitations on the lack of the provision of the list of excluded trails, screening of duplicate study, and protocol registration. For reporting quality, response rate of “yes” was less than 50% in Q5 (protocol and registration topic), Q8 (Search) and Q23 (Additional analysis). Evidence quality of outcome measures was low or worse with GRADE because when the data was qualitatively synthesized, low quality and small sample size trials provided the data.

Conclusion

Acupuncture may be beneficial for PSCI. Because of limitations and inconsistent conclusions, further research is needed to provide higher evidence for acupuncture on PSCI.

Keywords:

• acupuncture
• treatment
• cognitive impairment
• stroke
• evidence

Introduction

Stroke is a complex systemic disease that is the world’s third leading cause of disability.¹ As a frequent complication of stroke, post-stroke cognitive impairment (PSCI) affects approximately 80% of stroke survivors and is characterized by impaired learning and memory and deficits in executive function.² PSCI tends to be associated with adverse outcomes, including poor functional outcome,³ poor quality of life,⁴ and high mortality.⁵ Therefore, effective management of PSCI is essential to promote the recovery of survivors and reduce the burden on society. Stroke rehabilitation methods have developed rapidly, in which cognitive function training (CFT) is used as the basic treatment for PSCI.⁶ However, no single rehabilitation method was determined to be significantly beneficial for recovery.⁶ As a result, recent research on the use of complementary therapies for stroke rehabilitation has been conducted.⁷

Acupuncture is an increasingly widely used alternative treatment worldwide.^8,9 The World Health Organization believes that acupuncture is a good complementary treatment option for stroke rehabilitation.¹⁰ The positive effects of acupuncture on stroke rehabilitation are mediated by a variety of mechanisms, including the stimulation of neuroprotective factors, regulation of neurochemicals, inhibition of inflammatory damage, neurogenesis stimulation and cell division in the central nervous system, as well as control of cerebral blood flow in the ischemic region.¹⁰ Comprehensive evidence suggests that acupuncture is beneficial for recovery from stroke complications, including post-stroke aphasia, post-stroke dysphagia, and post-stroke insomnia.^7,11,12 Furthermore, acupuncture has received increasing attention for improvement of cognitive function in stroke patients.¹³ However, its certainty of evidence has not been systematically assessed.

Systematic reviews (SRs) and meta-analyses (MAs) evidence, which make up the top of the evidence pyramid, is typically thought to provide a trustworthy foundation for judgment in medicine.¹⁴ However, the availability of such evidence to provide decision guidance to users of the evidence needs to be further evaluated because of various possible sources of bias in the gathering of evidence.¹⁵ To summarize the data from the various SRs/MAs that have been published for overlapping themes in a short length of time, an overview is particularly required.¹⁶ An overview has several advantages over traditional SRs/MAs, including the ability to thoroughly assess the same evidence that is currently repeated and not systematically assessed, supplying evidence users with more targeted, high-quality evidence, and spotting significant flaws in the evidence formation process.¹⁷ Literature searches have yielded a greater percentage of SRs/MAs evaluating the efficacy on PSCI with acupuncture, but their quality is heterogeneous and no uniform conclusions have been formed. We therefore conducted this study to synthesize the available evidence.

Methods

The Cochrane Handbook¹⁸ was followed in conducting this study. On the PROSPERO, the protocol was prospectively registered (CRD42022301171). Ethical approval and consent statement were not required for this meta-analayses study.

Criteria for Inclusion and Exclusion

Following were the inclusion criteria: (a) randomized controlled trials (RCTs) were the only studies that the SRs/MAs included; (b) stroke survivors with PSCI; (c) the application of acupuncture or in combination with CFT as experimental intervention, while CFT as the control intervention; (c) effective rate, or Montreal Cognitive Assessment (MoCA) were applied as outcome measurements. SRs/MAs comparing different types of acupuncture, SRs/MAs comparing the effects of acupuncture with medication, and SRs/MAs for which data were not available were excluded.

Search Strategy

The keywords of acupuncture, stroke, and cognition disorder were applied to systematically search Web of Science, Cochrane Library, PubMed, Embase, SinoMed, Wanfang, CNKI, and Chongqing VIP on October 2022. The detailed search strategy applied to each database is presented as Additional File 1.

Eligibility Assessment and Data Extraction

Publications were read by two reviewers independently. A third party intervened to resolve any discrepancies. General study characteristics, subject characteristics, intervention characteristics, outcome measurements, and primary results were among the data that were extracted.

Review Quality Evaluation

The methodological quality was appraised with Methodological Quality of Systematic Reviews 2 (AMSTAR-2).¹⁹ We evaluated reporting quality with the list of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and evidence quality with Grade of Recommendation, Assessment, Development and Evaluation (GRADE) system.²⁰ By bringing in a third reviewer for judgment, any disagreements were resolved. Additional File 2 offers comprehensive items for AMSTAR-2 and PRISMA.

Results

Eligible Studies

A total of 177 citations were identified. After removal of duplicates, 130 citations were screened by two reviewers independently. By further screening, 115 citations were excluded, and 15 articles^21–34 were included finally. The procedure for locating the qualified articles is shown in Figure 1.

Figure 1 Literature selection procedure.

Studies Characteristics

Fifteen studies that were conducted in China and published between 2010 and 2021 were included. The samples varied widely among the included studies (RCTs ranged from 8 to 37, subjects ranged from 504 to 2869). CFT alone served as the control intervention, while acupuncture was used as an experimental intervention. All studies conducted a meta-analysis. More details are outlined in Table 1.

Table 1 Studies Characteristics

Studies	Trials (Subjects)	Treatment Intervention	Control Intervention	Quality Assessment	Meta-Analysis	Results
Kuang 2021^Citation21	28 (2144)	AT + CFT	CFT	Cochrane criteria	Yes	Acupuncture is effective and safe in the treatment of PSCI. However, inclusion trials with high risk of bias warrant careful interpretation of the results.
Xie 2021^Citation22	19 (1327)	AT + CFT	CFT	Cochrane criteria	Yes	Better clinical effect is seen in acupuncture than CFT when treating PSCI.
Xu 2020^Citation23	15 (1301)	AT + CFT	CFT	Cochrane criteria	Yes	Existing clinical evidence suggests that acupuncture reduces the degree of neurological deficits after stroke and can improve patients’ self-living ability.
Hu 2020^Citation24	11 (805)	AT + CFT	CFT	Cochrane criteria	Yes	Acupuncture clearly improved the cognitive function of PSCI patients. However, the included studies were of low quality rather than high-quality, large- scale and multiple-center RCTs.
Zhou 2020^Citation21	37 (2869)	AT + CFT	CFT	Cochrane criteria	Yes	Acupuncture clearly improves PSCI, but subsequent randomized clinical trials should be more rigorously designed and larger.
Liu 2020^Citation25	22 (1856)	AT + CFT	CFT	Cochrane criteria	Yes	This study positively supports that acupuncture may effectively improve cognitive function and daily living ability in patients with PSCI.
Liu 2018^Citation26	22 (1637)	AT + CFT	CFT	Cochrane criteria	Yes	Acupuncture clearly improved PSCI patients’ cognitive function.
Zhan 2017^Citation27	14 (896)	AT + CFT	CFT	Cochrane criteria	Yes	Acupuncture clearly improved cognitive and motor function of PSCI and is safe. However, the included studies were of low quality and subsequent RCTs should be better-designed and multicenter.
Lin 2016^Citation28	19 (1131)	AT + CFT	CFT	Cochrane criteria	Yes	Acupuncture shows better efficacy than control treatment in improving cognitive function in PSCI patients, and this result suggests that acupuncture has the potential to treat cognitive dysfunction.
Chen 2016^Citation29	8 (504)	AT + CFT	CFT	Cochrane criteria	Yes	The limited evidence showed that scalp acupuncture is more effective than CFT alone in treating PSCI.
Xiong 2016^Citation30	13 (1113)	AT + CFT	CFT	Cochrane criteria	Yes	The therapeutic efficacy of acupuncture on cognitive dysfunction PSCI patients cannot be determined. Therefore, acupuncture still needs to be used with CFT when treating PSCI in clinical practice.
Zhang 2015^Citation31	11 (789)	AT + CFT	CFT	Cochrane criteria	Yes	Meta analysis results showed better therapeutic effect in acupuncture with CFT than CFT alone.
Liu 2015^Citation32	9 (579)	AT + CFT	CFT	Jada	Yes	The limited evidence showed that acupuncture is more effective than CFT alone in treating patients with PSCI.
Liu 2014^Citation33	21 (1421)	AT + CFT	CFT	Cochrane criteria	Yes	The present study showed a positive effect of acupuncture on PSCI, but additional studies are needed to draw further definitive conclusions.
Yuan 2010^Citation34	9 (620)	AT + CFT	CFT	Cochrane criteria	Yes	Existing clinical evidence suggests that acupuncture improved PSCI patients’ cognitive function, but no definitive conclusions can be drawn based on the current evidence.

Abbreviations: AT, acupuncture; CFT, cognitive function training; PSCI, post-stroke cognitive impairment.

Methodological Quality

Figures 2 and 3 show the results of methodological quality. The major methodological flaws were as follows: (a) item 2 (almost all studies did not state the protocol); item 7 (a list of excluded studies was missing in all reviews, which failed to justify exclusion). Thus, all reviews were rated low or very low in methodological quality.

Figure 2 Methodological quality summary.

Note: Red represents no; yellow represents partially yes; green represents yes.

Figure 3 Methodological quality graph.

Note: Red represents no; yellow represents partially yes; green represents yes.

Reporting Quality

Sections of “Title”, “Abstract”, “Introduction”, “Discussion”, and “Funding” were all well-reported (100%) according to the PRISMA statement. However, some items had reporting deficiencies. In “Methods”, response rate of “yes” was less than 50% in Q5 (topic of protocol and registration) and Q8 (Search); in “Results”, response rate of “yes” was less than 50% in Q23 (Additional analysis). Details of the reporting quality are outlined in Table 2.

Table 2 Result of the PRISMA Assessments

Topic	Items	Kuang 2021^Citation21	Xie 2021^Citation22	Xu 2020^Citation23	Hu 2020^Citation24	Zhou 2020^Citation21	Liu 2020^Citation25	Liu 2018^Citation26	Zhan 2017^Citation27	Lin 2016^Citation28	Chen 2016^Citation29	Xiong 2016^Citation30	Zhang 2015^Citation31	Liu 2015^Citation32	Liu 2014^Citation33	Yuan 2010^Citation34	Compliance (%)
Title	Q1	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
Abstract	Q2	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
Introduction	Q3	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q4	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
Methods	Q5	Y	N	N	Y	N	N	N	N	N	N	N	N	N	N	N	13.3%
	Q6	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q7	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q8	Y	Y	PY	PY	Y	Y	Y	PY	PY	PY	Y	Y	PY	PY	PY	46.7%
	Q9	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q10	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q11	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q12	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q13	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q14	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q15	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q16	Y	N	Y	Y	Y	Y	Y	N	N	N	N	Y	N	Y	N	53.3%
Results	Q17	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q18	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q19	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q20	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q21	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q22	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	88.9%
	Q23	Y	N	N	N	Y	Y	N	N	N	N	N	Y	N	N	N	26.7%
Discussion	Q24	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q25	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
	Q26	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%
Funding	Q27	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	100%

Abbreviations: Y, yes; N, no; PY, partially yes.

Evidence Quality

Using the GRADE system, 30 outcome measurements of interests were assessed. The evidence quality for these outcome measurements was very low for 8 (26.67%), low for 16 (53.33%), moderate for 6 (20%), and high for 0 (0%). Details of the evidence quality are outlined in Table 3.

Table 3 Evidence Quality Results

Review	Outcomes	Certainty Assessment							Relative Effect (95% CI)	Evidence Quality
		№ of Trials	Design	Limitations	Inconsistency	Indirectness	Imprecision	Publication Bias
Zhou 2021^Citation21	MoCA	14	Rct	Serious^a	Serious^b	No	No	No	MD 2.26 (2.18, 3.13)	Low
Xie, 2021^Citation22	ER	7	Rct	Serious^a	No	No	No	No	OR 6.30 (3.58, 11.10)	Moderate
	MoCA	10	Rct	Serious^a	Serious^b	No	No	Serious^d	MD 3.46 (3.09, 3.84)	Very low
	LOTCA	3	Rct	Serious^a	Serious^b	No	Serious^c	No	MD 9.29 (4.25, 14.33)	Low
Xu 2020^Citation23	MoCA	9	Rct	Serious^a	No	No	No	No	MD 2.81 (2.42, 3.20)	Moderate
Hu 2020^Citation24	ER	5	Rct	Serious^a	No	No	No	No	OR 3.15 (1.81, 5.46)	Moderate
	MoCA	4	Rct	Serious^a	Serious^b	No	Serious^c	No	WMD 1.84 (0.81, 2.88)	Very low
Zhou 2020^Citation21	MoCA	8	Rct	Serious^a	Serious^b	No	No	No	WMD 2.66 (1.95, 3.37)	Low
Liu 2020^Citation25	MoCA	9	Rct	Serious^a	Serious^b	No	No	No	MD 1.73 (1.39, 2.06)	Low
Zhan 2017^Citation27	ER	3	Rct	Serious^a	Serious^b	No	Serious^c	No	OR 1.37 (0.98, 1.31)	Very low
	MoCA	6	Rct	Serious^a	Serious^b	No	No	No	MD 1.92 (0.96, 2.88)	Low
Lin 2016^Citation28	MoCA	3	Rct	Serious^a	No	No	Serious^c	No	WMD 1.74 (0.84, 2.63)	Low
Chen 2016^Citation29	ER	2	Rct	Serious^a	No	No	Serious^c	No	OR 14.63 (2.61, 82.16)	Low
Hang 2015^Citation31	ER	3	Rct	Serious^a	Serious^b	No	Serious^c	No	RR 1.58 (1.10, 2.26)	Very low
Wang 2015^Citation32	ER	9	Rct	Serious^a	No	No	No	No	OR 2.64 (1.40, 5.00)	Moderate
Yuan 2010^Citation34	ER	4	Rct	Serious^a	No	No	Serious^c	No	OR 8.59 (4.76, 15.51)	Low

Notes: ^aExperimental designs were wildly biased or blind to random, distributed results; ^bless confidence interval overlap, small heterogeneity test P and larger I² could be seen; ^cthe Confidence interval was not narrow enough, or too small for simple dimensions; ^dasymmetric funnel plots or fewer included studies suggested that there may be a large publication bias.

Abbreviations: MMSE, Mini-Mental State Examination; MoCA, Montreal Cognitive Assessment; ER, effective rate.

Efficacy Evaluation

The combined effects of acupuncture and CFT were significantly superior to CFT alone, according to the pooled results of eight SRs/MAs.^{21–25,27,28} The pooled results of six reviews^{22,24,28,31,32,34} revealed that CFT plus acupuncture was significantly superior to CFT alone. Nevertheless, the other review²⁷ reported no significant difference in effective rate. More details of relative effects for all outcome measurements are outlined in Table 3.

Discussion

A poor-quality SR/MA may mislead decision-making.³⁵ The assessment of recent evidence from SRs and MAs on numerous related topics forms the basis of an overview and verifies the validity of the evidence.³⁶ Current evidence from SRs/MAs regarding acupuncture for PSCI is inconclusive. Hence, we performed this overview.

Summary of Main Findings

First, the evidence for using acupuncture to treat PSCI currently available has a great deal of room for improvement. In this study, we rigorously assessed methodological and evidence quality of the included SRs/MAs, and it was discovered that the current evidence deficiencies were frequent and challenging to satisfy the evidence users. With methodological quality, all SRs/MAs were found to have one or more critical items major flaws, and it was precisely because of such flaws that all SRs/MAs were deemed with low or worse methodological quality. Almost no studies showed that the review technique was developed before the SAs/MAs were conducted, hence they were unable to support any substantial programming changes. Additionally, all studies did not provide excluded trials list and therefore failed to justify the exclusion. For evidence quality, all outcome measurements were graded between moderate and very low quality. The most frequent cause of evidence degradation came from the risk of bias resulting from RCTs.

Second, based on the available evidence from SRs/MAs, there are no conclusive findings regarding the effects of acupuncture on PSCI. It was well-known that high-quality SRs contribute to providing valuable evidence and conversely might mislead clinical decisions.^37–40 The findings of the methodology quality assessment and the quality of evidence assessment were unsatisfactory, suggesting that the findings of these studies might not be fully consistent with the actual application, and the reliability of the evidence has to be further examined. In addition, although all included studies suggested that acupuncture may have a positive effect on PSCI, most authors were reluctant to draw definitive conclusions because of the high risk of bias or small sample size of the trails. Therefore, acupuncture for PSCI rehabilitation should only be suggested with caution.

Practice and Research Implications

The findings of this overview revealed recurring fields for improvement that could help guide the conduct of future high-quality SRs/MAs. The rigor of an SR/MA must be ensured firstly, and any potential risk of bias must be avoided, by designing and registering study protocols in advance. Second, to ensure the study can be replicated, a thorough search strategy and list of excluded studies should be given. Third, when conducting data analysis, due consideration should be given to the scientific character of the analysis strategies. For example, performing subgroup analysis or sensitivity analysis may be considered when the heterogeneity of included studies is significant. In addition, any conflicts of interest and the source of funding must be adequately disclosed, as studies that are paid for by industry may produce findings that are more geared in their favor. In summary, the currently published SRs/MAs were identified with critical flaws, and for the purpose of providing high-quality evidence, future researchers should conduct SRs/MAs strictly follow with the criteria.

Limitations

Limitations should be acknowledged. Firstly, although the evaluation process establishes objective criterion tools, there may still be inevitable subjective factors in the assessment process. Furthermore, all of the studies that were included were carried out in China, though this finding is not unexpected given that acupuncture originated in China and has been most frequently used there. Acupuncture has been gaining international recognition in recent years, but patients are generally unfamiliar with this treatment, so it is still not universally used internationally.⁴¹ All of these factors pose challenges to the internationalization of acupuncture and therefore limit the development of international acupuncture research. Given that the included reviews were performed in China, which inevitably poses a potential risk of bias, it remains necessary to encourage further international studies of acupuncture for PSCI in other countries or regions outside of China.

Conclusion

Acupuncture may be beneficial for PSCI. Because of limitations and inconsistent conclusions, further research is needed to provide higher evidence for acupuncture on PSCI.

Disclosure

Qiongyang Zhou and Yue Ji are the co-first authors. The authors report no conflicts of interest in this work.