In this issue you will see that we have published a systematic review examining the evidence for the effectiveness of oral health instruction in the orthodontic patient. Although systematic reviews are established as the method of choice for summarizing the literature in medicine, they have a bad name in orthodontics. I was at a meeting once when someone described them as a form of professional masochism. Reviews of orthodontic research always seem to reach the conclusion that there is insufficient evidence to reach a conclusion.
So what’s the point?
Before considering whether we should continue to publish systematic reviews that are no help to the practising clinician, it is worth remembering how they developed. The organization whose name is synonymous with the systematic review is the Cochrane Collaboration. This is an international organization of mostly volunteers, who not only give their time to write reviews, but also referee and edit them, provide training to others in writing reviews and analysing data, handsearch the literature to ensure that all relevant material is available, translate articles into English and many other tasks. The collaboration particularly encourages the public to comment and contribute to reviews.
The name of the collaboration is derived from Archie Cochrane, a British doctor and epidemiologist who worked for many years in South Wales. Cochrane spent four years, during the Second World War, as a doctor in a prisoner of war camp. He tackled chronic malnutrition, diarrhoea and infections such as diphtheria, typhoid and tuberculosis, with few resources. He later wrote ‘I had considerable freedom of clinical choice of therapy: my trouble was that I did not know which to use and when. I would gladly have sacrificed my freedom for a little knowledge….I was afraid that I shortened the lives of some of my friends by unnecessary intervention.’1
The above quote is from the book ‘Effectiveness and efficiency: Random reflections on health services, which Cochran wrote in 1972. In the book Cochran argues that because resources for health would always be limited then only properly evaluated interventions that have been shown to be effective should be used. The first Cochrane centre was opened at Oxford in 1992, four years after Archie Cochrane’s death. The Cochrane Collaboration was founded in 1993 and a year later the Oral Health Group was established, initially in the USA, and then moved to its present base in Manchester, UK in 1996.
The Cochrane Collaboration is not the only organization involved in producing systematic reviews. The Centre for Reviews and Dissemination at the University of York, part of the UK’s National Institute for Health Research has produced influential reviews on water fluoridation and the management of impacted third molars, but it is the Cochrane Collaboration that many people will associate with systematic reviews of health-care interventions.
Reproduced with Permission from the Cochrane Collaboration
Why do I believe that reviews, produced in an open, structured and unbiased manner should still be published even if they currently provide no guidance to clinicians about the most effective treatments? One indication is in the Cochrane logo. The logo shows a forest plot; a graph frequently used in Cochrane reviews as a diagrammatic representation of the review results. The plot is from a review examining the effects of giving corticosteroids to women who are about to give birth prematurely. The outcome of interest in the graph is the odds of babies dying from the complications of immaturity. The seven horizontal lines represent the results from seven RCTs. The diamond at the bottom shows the overall result when all the trials are combined. The vertical line represents the position of the results if the babies had an equal chance of dying or surviving after the mother was given the steroids. Horizontal lines to the left of the vertical line indicate trials which found the babies had a better chance of surviving after the mother was given steroids. Horizontal lines to the right of the vertical line indicate trials which found babies have a higher chance of dying after the mother was given steroids. If the horizontal line crosses the vertical line then there is no evidence either way.
The top horizontal line was the first RCT reported in 1972. This showed that steroids had a positive effect, however it was just one trial and therefore the evidence was not considered to be very strong. Over the next 10 years, several more trials were carried out and the diagram shows what reviewers would have found if they had carried out a systematic review in 1982. The combined results showed that steroids given to the mother just before birth reduced the chances of a premature baby dying of immaturity by 30 to 50%. Unfortunately no review was undertaken until 1989 and in the meantime many premature babies probably suffered and died unnecessarily, because clinicians were unaware of the effectiveness of this simple treatment.
There are now several examples like this in medicine, where the evidence from combining the results of several trials in a systematic review has changed clinical practice. Unfortunately, we in dentistry tend to lag behind medicine with regard to available sound evidence of effectiveness. It takes time to build up such evidence and it is important for clinicians to be patient and for researchers to be aware of the work that needs to be done.
It is a frequent finding when undertaking a systematic review, that there are relevant data out there, but those data cannot be used in the review due to poor reporting. This is why it is so important that authors follow published guidelines for reporting research, such as the CONSORT guidelines for clinical trials. In addition, if authors are contacted by researchers undertaking systematic reviews seeking further information I urge them to make every effort to provide the information and data requested.
We must continue to support the work of organizations such as the Cochrane Collaboration, who seek to provide evidence of effectiveness for the things we do as clinicians. In the future, I am sure that up-to-date systematic reviews will be a valuable aid to busy clinicians to inform them of the current state of knowledge. They are now extremely important to researchers in identifying the relevant questions that clinicians need answering and the (extensive) gaps in our knowledge. I believe that we have to keep publishing these reviews, however frustrating it is for the reader. Eventually the effort will pay off.
Bibliography
Cochrane AL. Effectiveness and Efficiency. Random Reflections on Health Services. London: Nuffield Provincial Hospitals Trust, 1972. (Reprinted in 1989 in association with the BMJ, Reprinted in 1999 for Nuffield Trust by the Royal Society of Medicine Press, London (ISBN 1-85315-394-X).