Tuberculosis diagnostics trials: do they lack methodological rigor?

Figures & data

Table 1. The QUADAS tool, a validated quality assessment tool for diagnostic accuracy studies.

No.	QUADAS item	Potential bias if the criterion is not met
1	Was the spectrum of patients representative of the patients who will receive the test in practice?	Spectrum bias (commonly seen in case–control studies)
2	Were selection criteria clearly described?	Selection bias
3	Is the reference standard likely to correctly classify the target condition?	Misclassification of disease status due to an imperfect reference standard
4	Is the time period between reference standard and index test short enough to be reasonably sure that the target condition did not change between the two tests?	Disease progression bias
5	Did the whole sample or a random selection of the sample, receive verification using a reference standard of diagnosis?	Partial verification (work-up) bias
6	Did patients receive the same reference standard regardless of the index test result?	Differential verification (work-up) bias
7	Was the reference standard independent of the index test (i.e., the index test did not form part of the reference standard)?	Incorporation bias
8	Was the execution of the index test described in sufficient detail to permit replication of the test?	Source of variation, and likely to affect generalizability
9	Was the execution of the reference standard described in sufficient detail to permit its replication?	Source of variation, and likely to affect generalizability
10	Were the index test results interpreted without knowledge of the results of the reference standard?	Review bias due to lack of blinding
11	Were the reference standard results interpreted without knowledge of the results of the index test?	Review bias due to lack of blinding
12	Were the same clinical data available when test results were interpreted as would be available when the test is used in practice? For example, if clinical data (e.g., results of physical examination) will be available when the test is interpreted in clinical practice, then this should also be available when the test is evaluated.	Source of variation, and likely to affect generalizability
13	Were uninterpretable and/or intermediate test results reported?	Bias due to exclusion of uninterpretable and intermediate test results from the analyses
14	Were withdrawals from the study explained?	Bias due to differential loss to follow-up

Adapted with permission from [9].

QUADAS: Quality assessment of diagnostic accuracy studies.

Table 2. Methodological quality of studies on tuberculosis diagnostics in recently published meta-analyses.

Meta-analysis	No. of studies	Diagnostic test	Average size of each study	Prospective data collection (%)	Consecutive or random sampling of subjects (%)	Cross-sectional design (%)	Blinded interpretation of test results* (%)	Complete verification of index test results^‡ (%)	Ref.
Sarmiento et al. (2003)	16	PCR on respiratory specimens for smear-negative pulmonary TB	NR	50	NR	NR	63	100	[12]
Goto et al. (2003)	40	ADA for TB pleural effusion	137	NR	NR	NR	0	NR	[13]
Pai et al. (2003)	49	NAT for TB meningitis	42	61	49	61	59	94	[14]
Greco et al. (2003)	44	ADA and IFN-γ tests for TB pleural effusion	135	NR	NR	NR	9	NR	[15]
Pai et al. (2004)	40	NAT for TB pleural effusion	60	63	53	70	55	100	[16]
Flores et al. (2005)	84	In-house PCR for pulmonary TB	149	NR	NR	71	34	NR	[17]
Kalantri et al. (2005)	13	Phage amplification tests for pulmonary TB	448	NR	NR	85	23	100	[18]
Pai et al. (2005)	21	Phage-based tests for rifampin resistance	85	NR	38	NR	57	100	[19]
Morgan et al. (2005)	15	Line probe assay for rifampin resistance	91	NR	0	NR	13	100	[20]
Greco et al. (2006)	63	Commercial NAT for pulmonary TB	410	16	32	NR	16	NR	[21]
Steingart et al. (2006)	45	Fluorescence versus conventional sputum smear microscopy for pulmonary TB	493	100	36	NR	49	NR	[22]
Steingart et al. (2006)	83	Direct versus concentrated sputum smear microscopy for pulmonary TB	256	100	21	NR	31	NR	[23]

*At least single blind.

^‡By reference standard.

ADA: Adenosine deaminase; IFN: Interferon; NAT: Nucleic acid amplification test; NR: Not reported; TB: Tuberculosis.

Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med. Res. Methodol.3, 25 (2003).

 PubMedGoogle Scholar

Sarmiento OL, Weigle KA, Alexander J, Weber DJ, Miller WC. Assessment by meta-analysis of PCR for diagnosis of smear-negative pulmonary tuberculosis. J. Clin. Microbiol.41(7), 3233–3240 (2003).

 PubMed Web of Science ®Google Scholar

Goto M, Noguchi Y, Koyama H et al. Diagnostic value of adenosine deaminase in tuberculous pleural effusion: a meta-analysis. Ann. Clin. Biochem.40(Pt 4), 374–381 (2003).

 PubMed Web of Science ®Google Scholar

Pai M, Flores LL, Pai N et al. Diagnostic accuracy of nucleic acid amplification tests for tuberculous meningitis: a systematic review and meta-analysis. Lancet Infect. Dis.3(10), 633–643 (2003).

 PubMed Web of Science ®Google Scholar

Greco S, Girardi E, Masciangelo R, Capoccetta GB, Saltini C. Adenosine deaminase and interferon γ measurements for the diagnosis of tuberculous pleurisy: a meta-analysis. Int. J. Tuberc. Lung Dis.7(8), 777–786 (2003).

 PubMed Web of Science ®Google Scholar

Pai M, Flores LL, Hubbard A, Riley LW, Colford JM Jr. Nucleic acid amplification tests in the diagnosis of tuberculous pleuritis: a systematic review and meta-analysis. BMC Infect. Dis.4(1), 6 (2004).

 PubMed Web of Science ®Google Scholar

Flores LL, Pai M, Colford JM Jr, Riley LW. In-house nucleic acid amplification tests for the detection of Mycobacterium tuberculosis in sputum specimens: meta-analysis and meta-regression. BMC Microbiol.5, 55 (2005).

 PubMed Web of Science ®Google Scholar

Kalantri S, Pai M, Pascopella L, Riley L, Reingold A. Bacteriophage-based tests for the detection of Mycobacterium tuberculosis in clinical specimens: a systematic review and meta-analysis. BMC Infect. Dis.5(1), 59 (2005).

 PubMed Web of Science ®Google Scholar

Pai M, Kalantri S, Pascopella L, Riley LW, Reingold AL. Bacteriophage-based assays for the rapid detection of rifampicin resistance in Mycobacterium tuberculosis: a meta-analysis. J. Infect.51(3), 175–187 (2005).

 PubMed Web of Science ®Google Scholar

Morgan M, Kalantri S, Flores L, Pai M. A commercial line probe assay for the rapid detection of rifampicin resistance in Mycobacterium tuberculosis: a systematic review and meta-analysis. BMC Infect. Dis.5, 62 (2005).

 PubMed Web of Science ®Google Scholar

Greco S, Girardi E, Navarra S, Saltini C. The current evidence on diagnostic accuracy of commercial based nucleic acid amplification tests for the diagnosis of pulmonary tuberculosis. Thorax (2006) (In Press).

 Google Scholar

Steingart K, Henry M, Ng V et al. Fluorescence versus conventional sputum smear microscopy for tuberculosis: a systematic review. Presented at: 10th Annual Conference of the International Union Against Tuberculosis and Lung Disease (North American Region). IL, USA, March 2–4 (2006).

 Google Scholar

Steingart KR, Ng V, Henry MC et al. Sputum processing methods to improve the sensitivity of smear microscopy for tuberculosis: a systematic review. Presented at: 10th Annual Conference of the International Union Against Tuberculosis and Lung Disease (North American Region). IL, USA, March 2–4 (2006).

 Google Scholar