Abstract
Objective: As part of an ongoing project to identify all the randomised controlled trials (RCTs) in the critical care literature, in association with the Cochrane Collaboration, this study was undertaken to compare hand-searching (direct manual searching) with Medline searching (employing a comprehensive search strategy) in a number of critical care journals and to assess the quality of reporting of RCTs. Methods: Recruited volunteers hand-searched each issue of each journal for RCTs. A comprehensive Medline search was conducted for each journal indexed by Medline. The yield of RCTs was compared for the two searching methods. Data relating to study design were extracted from a sample of RCTs to assess the quality of reporting. Results: Hand-searching in 15 critical care journals was completed by 32 volunteers. In the seven journals indexed by Medline, 1300 RCTs were identified. A comparison of hand-searching with Medline searching showed that for five out of the seven journals hand-searching identified significantly more RCTs than Medline searching. However, pooled results were heavily influenced by the results of the journal Chest , which showed that Medline identified more RCTs than hand-searching (85.2% compared to 65.1%, respectively). Quality of reporting was found to be poor making assessment of the quality of the trial difficult in many studies. Conclusion: The results showed variation between the quality of hand-searching across the journals, suggesting that hand-searching was not a "gold standard" method in this study and that the dual approach, promoted by the Cochrane Collaboration, may be the optimal approach for the identification of RCTs. Improvement in the quality of reporting studies will help the identification and ranking of studies incorporated into systematic reviews of the evidence.