Independent Validation Test of the Vote-Counting Strategy Used to Rank Biomarkers From Published Studies

Abstract

Aim

Vote counting is frequently used in meta-analyses to rank biomarker candidates, but to our knowledge, there have been no independent assessments of its validity. Here, we used predictions from a recent meta-analysis to determine how well number of supporting studies, combined sample size and mean fold change performed as vote-counting strategy criteria.

Materials & methods

Fifty miRNAs previously ranked for their ability to distinguish lung cancer tissue from normal were assayed by RT-qPCR using 45 paired tumor-normal samples.

Results

Number of supporting studies predicted biomarker performance (p = 0.0006; r = 0.44), but sample size and fold change did not (p > 0.2).

Conclusion

Despite limitations, counting the number supporting studies appears to be an effective criterion for ranking biomarkers. Predictions based on sample size and fold change provided little added value. External validation studies should be conducted to establish the performance characteristics of strategies used to rank biomarkers.

Keywords::

• biomarkers
• lung cancer
• meta-analysis
• microRNAs
• miRNAs
• statistical methods

Acknowledgements

We are grateful to Adrie van Bokhoven and Heather Malinowski of the University of Colorado’s Tissue Bank and Biorepository for providing us with RNA from patient tissue samples.

Financial & competing interests disclosure

This study was funded by an NIH Special Program of Research Excellence (SPORE) grant awarded to the University of Colorado Cancer Center (P50 CA058187). The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilized in the production of this manuscript.

Ethical conduct

The authors state that they have obtained appropriate institutional review board approval or have followed the principles outlined in the Declaration of Helsinki for all human or animal experimental investigations. In addition, for investigations involving human subjects, informed consent has been obtained from the participants involved.

Additional information

Funding

This study was funded by an NIH Special Program of Research Excellence (SPORE) grant awarded to the University of Colorado Cancer Center (P50 CA058187). The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.