Titles versus titles and abstracts for initial screening of articles for systematic reviews

Figures & data

Figure 1 Flow diagram of article selection and review using a titles first approach (A) and titles abstract screening process (B).

Table 1 Reviewer provides the labels based on titles and abstracts review

	Reviewer reported “yes” for inclusion
	Yes	No
Ultimately included in systematic review?
Yes	tp^TA	0
No	fp^TA	tn^TA

Abbreviations: ta, titles and abstracts together; tp, true positive; fp, false positive; tn, true negative.

Table 2 Reviewer provides the labels based on titles-only review

	Reviewer reported “yes” for inclusion
	Yes	No
Ultimately included in systematic review?
Yes	tp^T	fn^T
No	fp^T	tn^T

Table 3 Interreviewer agreement on the titles-first screening step versus the title and abstract screening step, by reviewer pair

Reviewer pair	1 Neurologist A and pulmonologist	2 Pulmonologist and neurologist B	3 Neurologist B and gynecologist/obstetrician	4 Gynecologist/obstetrician and general surgeon	5 General surgeon and neurologist A	Overall
Citations reviewed (n)	586	584	614	588	593	2965
% agreement on titles-alone classification (95% CI)	90.6 (88.2–93.0)	87.8 (85.2–90.5)	93.8 (91.9–95.7)	93.5 (91.5–95.5)	93.7 (91.8–95.7)	91.9 (91.0–92.9)
Kappa statistic on titles-alone classification (95% CI)	0.51 (0.40–0.62)	0.40 (0.30–0.51)	0.63 (0.52–0.74)	0.67 (0.57–0.77)	0.53 (0.39–0.66)	0.54 (0.49–0.59)
% agreement on titles and abstracts classification (95% CI)	96.2 (94.7–97.8)	95.3 (93.7–97.1)	95.2 (94.4–97.5)	97.4 (96.2–98.7)	96.6 (95.2–98.1)	96.3 (95.6–97.0)
Kappa statistic on titles and abstracts classification (95% CI)	0.56 (0.39–0.72)	0.43 (0.25–0.60)	0.56 (0.40–0.71)	0.72 (0.59–0.86)	0.49 (0.29–0.68)	0.56 (0.49–0.63)

Note: Percent agreement = (number of citations with a consensus of a “yes” “yes” or “no” “no” response in a reviewer pair)/total number citations screened.

Abbreviation: CI, confidence interval.

Table 4 Titles-first classified as relevant

	n	n	n
Yes	13	0	13
No	394	2558	2952
Total	407	2558	2965

Notes: Precision = 3.2%; recall = 100%; F-measure = 0.0619; n = number in sample.

Table 5 Titles and abstracts classified as relevant

	n	n	n
Yes	13	0	13
No	170	2782	2952
Total	183	2782	2965

Notes: Precision = 7.1%; recall = 100%; F-measure = 0.1327; n = number in sample.

Table 6 Overall data for both titles-only and titles and abstracts classified as relevant

	n	n	n
Yes	26	0	26
No	564	5340	5904
Total	590	5340	5930

Note: n = number in sample.