Acupuncture for insomnia? An overview of systematic reviews

Abstract

Background: Several systematic reviews of acupuncture as a treatment of insomnia have recently emerged. Their results are far from uniform. Aim: To summarize and critically evaluate these reviews with a view of defining the reasons for their discrepant conclusions and providing an overall verdict about the therapeutic value of acupuncture for insomnia. Methods: Thirteen electronic databases (Medline, Embase, Amed, CINHAL, Health Technology Assessments, DARE, Cochrane, six Korean/Chinese databases) were searched for relevant articles and data from the included reviews were extracted according to pre-defined criteria. Their methodological quality was assessed using the ‘Overview Quality Assessment Questionnaire’. Results: Ten systematic reviews of acupuncture for insomnia were published between 2003 and 2010. They differed in numerous respects. Several reviews draw strongly positive conclusions. Owing to these several caveats, the best evidence is, however, not clearly positive.

Conclusion: The evidence for acupuncture as a treatment of insomnia is plagued by important limitations, e.g. the poor quality of most primary studies and some systematic reviews. Those that are sensitive to such limitations, fail to arrive at a positive verdict about the effectiveness of acupuncture.

Key words::

• Acupuncture
• insomnia
• systematic review
• evidence
• bias

Introduction

Insomnia is a frequent complaint and common reason for people to try complementary or alternative treatments (1,2). A systematic review of the evidence disclosed significant methodological flaws (3). Acupuncture is one of the complementary therapies, which is frequently recommended as a treatment of insomnia. Several hypotheses about its mode of action exist (4), many patients seem to prefer it to drug therapy (5), and mainstream healthcare has adopted acupuncture to some extent (6,7). Yet, the findings from clinical trials are often less than convincing, and their results are full of contradictions. In this situation, a systematic review should be able to clarify the issue. In recent years, several such assessments have become available. Confusingly, however, their conclusions are not entirely uniform either. The above-mentioned review noted ‘inconsistent results and heterogeneous methodology’ in relation to such systematic reviews (3).

The aim of this overview is to summarize all systematic reviews of acupuncture as a treatment of insomnia with a view of arriving at a more definite conclusion.

Methods

Electronic literature searches were carried out in Medline, Embase, Amed, CINHAL, Health Technology Assessments, DARE, the Cochrane Library, six Korean Medical Databases, and Chinese Databases without restrictions of time or language (up to 6 December 2010). The search terms were acupuncture, insomnia, systematic review, and meta-analysis. The details are in Table I. In addition, our departmental files were hand-searched. Abstracts of reviews thus located were inspected by two authors and those appearing to meet the inclusion criteria were retrieved and read full by all three authors. Reviews were defined as systematic if they included an explicit and repeatable methods section describing the search strategy and explicit inclusion/exclusion criteria.

Table I. Search strategy.

1.	‘Acupuncture Therapy’ [Mesh]
2.	acupunc Field: Title/Abstract
3.	acupress Field: Title/Abstract
4.	electroacupunc Field: Title/Abstract
5.	meridian Field: Title/Abstract
6.	acupoint Field: Title/Abstract
7.	#1 OR #2 OR #3 OR #4 OR #5 OR #6 OR #7
8.	‘Sleep’ [Mesh]
9.	sleep Field: Title/Abstract
10.	insomnia Field: Title/Abstract
11.	wakeful Field: Title/Abstract
12.	#8 OR #9 OR #10 OR #11
13.	systematic review Field: Title/Abstract
14.	review Field: Title/Abstract
15.	meta-analysis Field: Title/Abstract
16.	No. 13 OR no. 14 OR no. 15
17.	No. 7 AND no. 12 AND no. 16

To be included, a systematic review had to be concerned specifically with the effectiveness of acupuncture for insomnia and had to include evidence from at least two controlled clinical trials. Systematic reviews were considered regardless of the acupuncture types, e.g. body, ear or electric acupuncture, etc. Systematic reviews of complex packages of interventions, which happened to include acupuncture, were excluded. Reviews, which depended entirely upon previous systematic reviews for their primary data, were also excluded.

The Overview Quality Assessment Questionnaire (OQAQ) was used to evaluate the methodological quality of all included SRs (8,9). Its score ranges from 1 to 7; a score of three or less was considered as indicative of extensive or major flaws and a score of 5 or more as suggesting minor or minimal flaws. Two authors assess the OQAQ independently and discrepancies were settled by discussion.

Data were extracted independently by two authors (EE & MSL) using pre-defined criteria (Table II). For the Chinese literature, one author (TYC) did the data extraction. Disagreements were resolved by discussion between the authors.

Table II. Summary of key data in 10 systematic reviews on acupuncture for insomnia.

First author (ref) year	Country	Number of primary studies (Chinese)	Total sample size	Quality of primary studies	Overall conclusion (quote)	Meta-analysis	Quality of review (OQAQ)	Comment
Cao (10) 2009	China	46 (45)	3811	Mostly poor	Acupuncture appears to be effective in treatment of insomnia	WMD: −2.94 (95% CI: −5.77 to −0.11) compared with sham acupuncture	4	3 Chinese databases were searched, only 3 studies were sham controlled
Huang (11) 2009	US	12 (10)	1305	Variable	The currently available studies have demonstrated the safety and potential beneficial effects of acupuncture in treating most forms of insomnia	None	3	5 studies were sham-controlled of which 4 were positive
Liu (12) 2009	China	10 (10)	1387	Variable	…acupuncture is proven to be an effective treatment of insomnia	OR: 2.64 (95% CI: 2.07–3.37) compared with control treatments	1	5 RCTs and 5 CCTs were included
Yeung (13) 2009	China	20 (19)	1299	Mostly poor	…the data, while promising, do not allow a clear conclusion	None	6	Funnel plot revealed publication bias
Lee (14) 2008	South Korea, UK	10 (8)	642	Poor	…the evidence for the effectiveness of auricular acupuncture …is limited	WMD: 7.5 (95% CI: 2.2–12.9) two sham-controlled trials	7	Review focused on auricular only acupuncture
Cheuk (15) 2007	China	7 (6)	590	Poor	…the current evidence is not sufficiently extensive or rigorous to support the use of any form of acupuncture for the treatment of insomnia	SMD: −1.08 (95% CI: −1.86 to −0.31) compared to placebo	7	Several variants of acupuncture, e.g. acupressure, were included
Chen (16) 2007	China	6 (6)	673	Poor	auricular acupuncture appears to be effective for treating insomnia	RR: 1.93 (95%CI: 1.40–2.66) compared with control	4	Review was of auricular acupuncture only; high heterogeneity in meta-analysis
Kalavapalli (17) 2007	US	19 (16)	1586	Mostly poor	…acupuncture may be useful in the treatment of insomnia…	None	1	Only 6 studies were RCTs, case series were also included
Li (18) 2005	China	6 (6)	585	Poor	Acupuncture is possibly effective for insomnia	OR: 3.55 (95% CI: 2.50–5.05) compared with drug group	3	Non-English articles were excluded
Sok (19) 2003	US, South Korea	11 (9)	990	Poor	…acupuncture may be an effective intervention for the relief of insomnia	None	1	Non-English articles were excluded. All studies reported positive effects

Overview quality assessment questionnaire (OQAQ). The overall score is from 1 to 7.

OQAQ ≤ 3: having extensive or major flaws; 5 ≤ OQAQ: having minor or minimal flaws.

WMD, weighted mean difference; OR, odds ratio; RR, risk ratio; SMD, standardized mean difference.

Results

The searches identified 12 unique articles. Two reviews were excluded; one because it had no explicit inclusion/exclusion criteria (20) and one because it was not systematic(21). Ten systematic reviews were included (10–19). Their key data are summarized in Table II. The details of quality the assessment of the primary studies, inclusion and exclusion criteria of each systematic review were listed in Table III, and Table IV (Table IV is available on The European Journal of General Practice website).

Table III. Overview quality assessment questionnaire (OQAQ) for the included systematic reviews.

First author (Ref) Year	1Were the search methods reported?	2Was the search comprehensive?	3Were the inclusion criteria reported?	4Was selection bias avoided?	5Were the validity criteria reported?	6Was validity assessed appropriately?	7Were the methods used to combine studies reported?	8Were the findings combined appropriately	9Were the conclusions supported by the reported? data?	10Overall score
Cao (10) 2009	2	2	2	2	2	2	1	0	1	4
Huang (11) 2009	2	2	2	1	2	2	1	1	1	3
Liu (12) 2009	2	0	1	1	2	2	1	0	0	1
Yeung (13) 2009	2	2	2	1	2	2	2	2	2	6
Lee (14) 2008	2	2	2	2	2	2	2	2	2	7
Cheuk (15) 2007	2	2	2	2	2	2	2	2	2	7
Chen (16) 2007	2	2	2	2	2	2	1	0	0	4
Kalav-apalli (17) (2007)	2	1	1	0	0	0	1	1	0	1
Li (18) 2005	2	2	2	1	2	2	1	1	0	3
Sok (19) 2003	2	2	1	0	0	1	0	0	1	1

Table IV. Quality assessment and inclusion and exclusion criteria of each included systematic review.

First author (Ref) year	Quality assessment	Inclusion criteria	Exclusion criteria
Cao (10) 2009	Agency for Healthcare Research and Quality (AHRQ) quality assessment criteria	P: patients with insomniaI: acupunctureC: no treatment, placebo, basic medical therapiesO: total sleep duration, increased sleep duration,Pittsburgh Sleep Quality Index (PSQI) scores,Index of Severity of Insomnia, Sleep Quality, other validated score, adverse effectsS: RCT, acupuncture + other treatment versus other treatmentNo limitation on language and publication type	Multiple publication reporting the same group of participants
Huang (11) 2009	Cochrane criteria	P: n.r.I: n.r.C: n.r.O: n.r.S: original case series or clinical trials published in peer-reviewed journals, human studies, article written in English or Chinese language	1. Although acupuncture mentioned, the main treatment modality was complementary alternative medicine2. Treatments focused on other sleep disorders such as sleep apnoea, night terrors, or somnambulism3. Multiple publication reporting the same group of participants or their subsets
Liu (12) 2009	Jadad score	P: n.r.I: n.r.C: n.r.O: n.r.S: RCT and CCT	1. Trials which comparing the point specificity and type of acupuncture2. Trials which are testing the effects of moxibustion, electric acupuncture, Confucianism point injection, sticking points of Chinese medicine, acupressure, etc.3. Acupuncture plus other treatment4. Duplicated publication
Yeung (13) 2009	Jadad score	P: participants with chief complaint of insomniaI: acupunctureC: placebo, Western medication, no treatmentO: not set any specification for outcome measuresS: RCT and CCTNo specification for duration or study quality	n.r.
Lee (14) 2008	Jadad score	P: patients with insomniaI: treatments with semi-permanent acupuncture needle or device (bead, seed, or magnetic pellet)C: placebo auricular acupuncture, active interventions without acupuncture, no treatmentO: n.r.S: RCTNo language restrictions were imposed	1. Studies comparing two different forms of auricular acupuncture2. Trials testing auricular acupuncture with another form of acupuncture or with other treatments which have not been proved effective for treating insomnia
Cheuk (15) 2007	Cochrane criteria	P: people of any age and gender with insomnia explicitly documented by standardized measures or by reports/diaries or patients with insomnia diagnosed by standard diagnostic criteriaI: traditional acupuncture in classical meridian points, or contemporary acupuncture in non meridian or trigger points (regardless of the source or methods of stimulation – hand, laser, seed, magnetic or electrical stimulation)C: placebo or sham acupuncture, no treatmentO: subjective report of sleep quality, sleep parameters, standard scales related to sleep (Pittsburgh Sleep Quality Index), Daytime functioning (Stanford Sleepiness Scale, Epworth Sleepiness Scale), quality of life, frequency of adverse effectsS: RCT, parallel, cross-over, acupuncture versus no treatment, acupuncture vs. placebo or sham treatment, acupuncture plus baseline medication or treatment vs. baseline medication or treatment alone, acupuncture plus baseline medication or treatment vs. placebo or sham treatment plus baseline medication or treatmentNo language restrictions were imposed	n.r.
Chen (16) 2007	Cochrane criteria	P: people of any age and gender with insomnia explicitly documented by standardized measures or by reports/diaries or patients with insomnia diagnosed by standard diagnostic criteriaI: auricular acupuncture is used as sole treatment, no special restrictions on the use of needles and other TCM manipulationsC: no treatment, sham auricular acupuncture, placebo, Western medicineO: subjective report of sleep quality, sleep parameters, standard scales related to sleep (Pittsburgh Sleep Quality Index), Daytime functioning (Stanford Sleepiness Scale, Epworth Sleepiness Scale), quality of life, frequency of adverse effectsS: RCTs which are written in the Chinese or EnglishNo language restrictions were imposed	1. Quasi-randomized trials and case studies2. Auricular acupuncture is used as co-intervention3. Auricular acupuncture4. Studies comparing different groups of auricular acupuncture points, different times for receiving treatments and different ways for stimulating the auricular acupuncture points
Kalavapalli (17) 2007	n.r	P: patients with insomniaI: acupunctureC: n.r.O: n.r.S: n.r.Published between 1985 and 2005Written either in English or English translation	n.r.
Li (18) 2005	Jadad score	P: patients with insomnia (according to clinical criteria)I: acupuncture, acupuncture combined basic drugC: drug therapy (two group use of same drugs)O: n.r.S: RCTWritten either in Chinese or English	1. Trials with no clear diagnostic criteria of insomnia2. Other underlying diseases associated with insomnia3. Animal experiments4. Studies comparing different acupuncture points, massage therapy as control group
Sok (19) 2003	n.r.	P: patients with insomniaI: acupuncture therapyC: n.r.O: n.r.S: n.r.Published between 1975 and 2002Written in English	n.r.

n.r., not reported.

The reviews were of variable quality but 3 seemed of sufficient rigour, i.e. scoring 6 or more points on the OQAQ (13–15) (Table III). The first authors originated from China (n = 6) (10,12,13,15,16,18), US (n = 3) (11,17,19) and South Korea (n = 1) (14). All first authors were affiliated to academic institutions. Six reviews (10,12,14–16,18) incorporated a meta-analytic approach. Mostly due to different inclusion/exclusion criteria (Table II), the reviews were based on 6 to 46 clinical studies.

All systematic reviews agreed that most of the primary studies were of poor methodological quality. Three reviews concluded that the current data were insufficient for firm conclusions (13–15) and 5 reviews concluded that acupuncture might be effective (10,16–19). Two of the most recent reviews concluded that acupuncture is beneficial (11,12). The three best quality reviews (13–15), concluded that the evidence is insufficient.

Discussion

Our analysis shows that, between 2003 and 2010, a remarkable multitude of systematic reviews of acupuncture for insomnia has emerged. A similar overview covering the period between 1989 and 2000 included not a single systematic review of acupuncture for insomnia (22). This indicates that the interest in this subject has increased substantially. A systematic review of all types of complementary or alternative treatments (3) mentioned 4 of the 10 systematic reviews included in the present overview. The conclusions of the reviews range from clearly positive (11,12) to sceptical (13,15) about the value of acupuncture for this indication. This level of discrepancy demands an explanation.

The most obvious reason might be that, as time passed, more positive primary studies emerged, and thus newer reviews generated more clearly positive overall results. Yet this does not seem to be the case. There is no clear trend to suggest that the addition of more recent studies has rendered the reviews more positive.

Table II shows that all 10 systematic reviews heavily rely on primary studies from China. Several groups have demonstrated that nearly 100% of all acupuncture studies from China generate positive results (23,24). This phenomenon raises considerable doubts about the reliability of these data. Table II also shows that the quality of the primary studies is often poor. Trials of poor quality tend to produce false positive results.

A possible explanation for the discrepant conclusions of the 10 systematic reviews, therefore, is that some of the authors failed to account adequately for the poor quality of the primary data. Systematic reviews make little sense, if they do not critically evaluate the quality and the reliability of the primary studies. If this happens, the reader is likely to be misled to believe that there is solid evidence where, in fact, the evidence is flawed and of debatable validity. Even a review with a relatively large amount of primary studies, like Cao et al. (10) could be unreliable, if the primary data are not reliable.

The plethora of recent systematic reviews discussed here thus provides us with a poignant reminder about the purpose of systematic reviews. Such articles should not merely summarize the existing primary studies but they have to assess critically their quality. If the latter aspect is missing, the overall conclusion is at risk of being false-positive. The authors of the 10 systematic reviews discussed here did, of course, note the mostly poor quality of the primary studies. Yet it seems that many failed to draw the right conclusion from this observation; most systematic reviews were not sensitive to these limitations.

Seen in this light, the evidence regarding the therapeutic value of acupuncture for the management of insomnia seems to be over-estimated by most of the recently published systematic reviews. The conclusion cannot be that acupuncture has been proven but that the limitations of most of the primary studies prevent firm conclusions.

Acknowledgements

MSL and TYC were funded by KIOM.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the writing and content of the paper.