What is a biomarker? Research investments and lack of clinical integration necessitate a review of biomarker terminology and validation schema

Abstract

A continual trend of annual growth can be seen within research devoted to the discovery and validation of disease biomarkers within both the natural and clinical sciences. This expansion of intellectual endeavours was quantified through database searches of (a) research grant awards provided by the various branches of the National Institutes of Health (NIH) and (b) academic publications. A search of awards presented between 1986 and 2009 revealed a total of 28,856 grants awarded by the NIH containing the term “biomarker”. The total funds for these awards in 2008 and 2009 alone were over $2.5 billion. During the same respective timeframes, searches of “biomarker” and either “discovery”, “genomics”, “proteomics” or “metabolomics” yielded a total of 4,928 NIH grants whose combined funding exceeded $1.2 billion. The derived trend in NIH awards paralleled the annual expansion in “biomarker” literature. A PubMed search for the term, between 1990 and 2009, revealed a total of 441,510 published articles, with 38,457 published in 2008. These enormous investments and academic outputs however have not translated into the expected integration of new biomarkers for patient care. For example no proteomics derived biomarkers are currently being utilized in routine clinical management. This translational chasm necessitates a review of the previously proposed biomarker definitions and evaluation schema. A subsequent discussion of both the analytical and pre-analytical considerations for such research is also presented within. This required knowledge should aid scientists in their pursuit and validation of new biological markers of disease.

Introduction

There is currently an overwhelming interest in biological marker (biomarker) research within both the natural and clinical sciences. Evidence of this effort is clear, as the volume of literature devoted to biomarker discovery, characterization and validation continues to trend with an annual expansion. The underlying motivations for such efforts are obvious. A newly discovered, and properly validated, biomarker would offer high value as a potential prognostic or diagnostic indicator for disease manifestation, progression, or both. For instance, such a marker could be used as an unbiased differential indicator of illness onset, aid in the classification of a diseased or non-diseased state, provide the ability to stage disease progression and/or offer insight into its relative severity. An individual's risk of developing a disorder may also be gleaned from biomarker research; as such a prognostic vessel could be employed for risk stratification of the general population. In addition to identifying illnesses, the efficacy of a respective disorders clinical or therapeutic intervention may also be surmised from such an indicator. Although, the summation of these potential utilities may rightly justify the present interest in biomarker research, it should be noted that upon discovery, a novel putative marker must traverse the exhaustive and rigorous path of clinical validation. When considered in concert with the biochemical and analytical challenges associated with identifying new markers [1], this tortuous pipeline of scientific investigation has yielded relatively few new indicators worthy of clinical integration [2]. Despite this reality, researchers within both academic and for-profit institutes are likely to continue their pursuit of these potential markers.

Much of the literature through which these intellectual pursuits are based upon, namely; biomarker definitions and terminology [3], the validity of integrating surrogate end-points into therapy [4–10], required biomarker evaluation schema [9–12] and the correct statistical analysis of biomarker data [10,13,14] have been previously discussed. Included within this literature are detailed workshop and policy statements regarding the major aspects of biomarker research [15–17]. In addition, an excellent and comprehensive paper presented by Ransohoff in 2007 offers a thorough review of the issues surrounding biomarker research, evaluation and validation [18]. Although this manuscript was published 3 years ago, many of the issues raised by Ransohoff remain relevant in 2010. Thus scientists conducting associated biomarker research are encouraged to digest and consider the presented text [18] during their related scientific endeavours.

The goal of this current, brief, review is not to replicate the efforts of these previous, expansive, publications. Instead it aims to provide the necessary context for future biomarker research by summarizing the field's current state, specifically in terms of both the monetary investments and academic outputs of past and current biomarker studies. It may be considered “common knowledge” that all aspects of current biomarker research trend along a trajectory of annual expansion, with growth being seen in, but not limited to; initial biomarker discovery work, the evaluation of a promising targets and the invocation of such markers within therapeutic intervention studies. What is not clear, however, is exactly how much these pursuits have expanded from their humble beginnings. In addition a representative quantitative assessment of the current monetary investments in such pursuits, and their relationship to both the academic outputs and clinical utility, in terms of publications and validated biomarkers respectively, is somewhat unknown.

To this end, research awards provided by the NIH and journal articles devoted to “biomarker” research were both respectively inspected to identify trends within funding and publication levels. This summation suggested that a historical review of published biomarker research, along with a re-examination of formal definitions and terms routinely employed within the field, should be undertaken to enhance the probability of future successes. Resources related to the previously proposed stages of biomarker validation are also included within this discussion, as researchers should employ such templates for analogous studies.

Collectively, this manuscript provides an objective overview of the current state of biomarker investments and reinforces the previously proposed tenets through which the community should respectively view the accepted definitions of a biomarker and their clinical utility assessment protocols.

Investments and outcome of current biomarker research

To provide a quantitative assessment of the current investment in biomarker research, grant awards provided by the NIH for such studies were chosen as a representative case study. The American Agencies/Institutes or Centers (IC) within the NIH represent a major source for biochemical and health research funding. Collectively they also provide a global assessment of both the current and past attraction of the NIH for funding biomarker and its related initiatives.

To retrospectively probe the relative interest amongst the various NIH branches, the NIH RePORT database (http://projectreporter.nih.gov/reporter.cfm) was mined in January 2010 for grants containing a series of terms related to biomarker research. This search considered grants dispersed by all of the Agencies/Institutes/Centers, funding mechanisms, award types, activity codes and study sections of the NIH. All queries were done considering the respective awards administering IC, rather than the funding IC. The resultant information was subsequently downloaded and categorized in terms of the year the grant was awarded, the specific administering IC and the respective funding levels associated with this research. Noted was that the NIH RePORT system only permitted observation of the actual funding amounts for grants respectively awarded in 2008 and 2009. In addition, several such awards received additional monies from a funding IC different to that of the administering IC. To account for this, all annual grant totals quoted in this text refer to the awards respective administering IC. All funding levels refer to all of the funds awarded by the various IC's of the NIH, as derived from the database queries outlined above. In addition, it should be stated that an exhaustive effort was not undertaken to examine each of the derived grants for their specific hypotheses, content or results. Instead, faith was placed in the search engine of the NIH RePORT website to produce all appropriate grants containing the queried terms, with an understanding that the database is dynamic and its contents are routinely updated.

Given that all awards summarized within contained the term “biomarker”, their respective intent may be viewed, but is likely not limited to; an investigation of a biomarkers relative value as a potential prognostic, diagnostic or predictive indicator of diseases. Examples of such study intents include the deployment of biomarker(s) within patient selection and their respective therapeutic risk stratification, the determination of effective treatment targets, an assessment of therapeutic efficacy, identification of a drug mechanism, and/or quantification of a diseased state and its progression [19].

Initially an annual search of grants, from 1986 to 2009, was conducted for awards containing the term “biomarker”. The analysis of these queries identified a total of 28,856 grants being awarded over this time frame. An annual breakdown of the total number of NIH “biomarker” grants awarded per year is provided in Figure 1. Interestingly, the first grants awarded in 1986 were by the National Cancer Institute (NCI) for the study of breast cancer [20] and the National Institute on Aging (NIA) for the two studies examining biomarkers of aging [21,22]. From these initial studies, Figure 1 reveals that the evolution of time is accompanied by a relative expansion in biomarker research. Amongst the trend of annual grant award growth, the most striking result is the dramatic increase in awarded grants for 2009. The previous year saw a total of 933 grants being dispersed with cumulative funding of $397,136,492. 2009 saw 5,009 grants being awarded with research funding totalling $2,109,799,544. In terms of the relative increase in the number of awards and their respective funding, the difference from 2008 to 2009 represents an increase of ≥ 500%. Even considering that 2008 was anomaly in terms of trending annual increases (Figure 1), this deliberate capital investment in related scientific endeavours represents a significant infusion of monies for investigators. The observed difference is likely a product of the increased research funding appropriated to the NIH under the American Recovery and Reinvestment Act of 2009.

Figure 1. Number of NIH grants from 1986 to 2009 which containing the term(s) “biomarker” or “biomarker and discovery”, as derived from the NIH RePORT database.

To gain insight into potential future ramifications or outcomes of this funding boost, the NIH RePORT database was subsequently mined for grants containing the terms, “biomarker” and “discovery”. The same 23 year span of awards was considered with this query in-order to analogously elucidate and assess funding trends pertaining to the identification of new biomarkers and gain insight into future outcomes of the 2009 funding level. However, before continuing this discussion, the obvious flaw in this NIH RePORT database search methodology should be noted. Although a derived grant may have contained both the terms “biomarker” and “discovery”, the nature of the work may not have been specifically directed at the discovery of new biomarkers. Similarly, grants returned for the singular query of “biomarker” may have been discovery oriented, although this intent may not have been explicitly mentioned in the terms of the submitted proposal. Indeed, the history of science is peppered with such unintended discoveries and advances through which the final outcome was not the preliminary hypotheses focus. However, for the purpose of this review, these NIH RePORT searches are meant to provide context to the relative types of biomarker grants being awarded and thus provide the reader insight into the current, historical and potential future funding interests within the NIH granting agencies. Under this pretext, it is not surprising that the path of annual funding expansion for “biomarker” and “discovery” research follows an analogous trajectory to that of singular “biomarker” grants (Figure 1). Of interest though is that, in spite of the acknowledged methodological search limitations, grants returned with both terms constitute a small fraction of overall biomarker funding. From 1986 to 2009 a total of 1,574 grants were awarded containing both terms, which represented ∼ 6% of the total biomarker awards funded during that same timeframe. Although this relatively reduced share more than doubled in 2009, with 631 grants being funded for an ∼ 13% share of the total biomarker research awards, the pursuit of new biomarkers remains but a singular component of current and related research. In addition, the derived respective levels highlight the heterogeneity of the work currently being undertaken within the NIH, but do not offer great insight into further potential biomarker undertakings.

To gain a better understanding of these broad funding trends and concepts, grants returned from the NIH RePORT search of the term “biomarker” were subsequently further analyzed with respect to the relative amount of monies provided by the respective funding IC. The results of this analysis are contained within Figure 2 and illustrate the dominance of funding provided by the NCI, which alone dispersed grant funding totalling $774,486,073 for 2008 and 2009 combined. The National Heart, Lung and Blood Institute (NHLBI) and NIA respectively represent the second and third highest funding IC's. In terms of the total funding provided to researchers during the two year time span (i.e., $2,506,936,046) these top three Institutes accounted for $1,273,568,608 of this level, with the remaining 33 indentified IC comprising the remaining ∼ 49% of administered funds (Figure 2). The dominance of NCI, NHLBI and NIA in terms of “biomarker” and “discovery” awards was also observed within a parallel analysis of the corresponding NIH RePORT query results. This study revealed that NCI, NHLBI and NIA collectively provided ∼ 53% of the total administered funding of $369,380,146 for such studies. In part, these trends are directly related to the relative size of these Institutes and their mandated study prerogatives. Indeed, disorders related to aging, cancer and the various heart, lung and blood diseases respectively constitute a significant portion of societal clinical ills and thus necessitate an expanded investigation of the potential role(s) biomarkers may play within the respective disease aetiologies, pathobiologies, evolutions and therapies. Such fundamental research is essential for the design and enhancement of current and future patient clinical management, as well as aiding in the on-going quests to elucidate the cure of these complex diseases. Although this statement rings true for all disorders, the relative prevalence of illnesses examined by these three Institutes aids in understanding the derived funding dominance and trends. Award levels however are not proportional to scientific discovery and translational success. Yet regardless of this potential disconnect, the undertaken IC specific analysis does provide insight into the popular disease pathologies currently being targeted by scientist and clinicians and signals areas of research worthy of monitoring for future biomarker advances.

Figure 2. Summary of the combined 2008 and 2009 funding (in millions of US $) for grants containing the term “biomarker” as provided by the various Institutes and Centers within NIH, as derived from the NIH RePORT database.

To further expand the mosaic of NIH biomarker funding the NIH RePORT database was again surveyed to gage the relative enthusiasm for academic pursuits employing the major “–Omics” initiatives (i.e., genomics, proteomics and metabolomics). Thus, the results from the combined term searches of “biomarker” and each of these individual fields was respectively conducted, downloaded, and analyzed for NIH grants administered between 1986 and 2009. The first awards containing the terms “biomarker” and either “genomics”, “proteomics” or “metabolomics” appeared within the database results of 1999, 1998 and 2003 respectfully. Since this initiation, a total of 1,125, 1,948 and 281 awards have been administered by the various branches of the NIH for biomarker initiatives within each of these fields, respectively. As observed with the relative contributions related to biomarker and “discovery”, these “–Omics” studies constitute only a small portion of the total number of NIH biomarker grants over the search timeframe. Given that such initiatives only emerged within the last twelve years, it may be expected that of the total number of NIH biomarker studies awarded (i.e., 28,856 grants from 1986 to 2009) the genomic (∼ 4%), proteomic (∼ 7%) and metabolomic (∼ 1%) contributions account for only a minor fraction. However, in terms of recent awards and funding levels, the collective fragment of awards devoted to these three fields was $854,188,047 in 2008 and 2009 (Figure 4). These initiatives account for roughly a third of all biomarker funding dispersed by the NIH in the past two years. Interestingly, within this “-Omics” funding level, genomics, proteomics and metabolomics biomarker research respectfully accounted for ∼ 44,47 and 9% of this total funding sum. Yet, in terms of the total number of grants awarded these relative shares shift to ∼ 41,49 and 9% respectively. This indicates an enhanced funding interest in recent genomic biomarker projects relative to parallel proteomic studies. Specific IC interests in such endeavours, as gleaned through an examination of the parcelled data, revealed that dominant funder of these “–Omics” initiatives was the NCI whom contributed $124,262,714 (330 grants), $130,191,275 (354 grants) and $13,363,869 (20 grants) for genomic, proteomic and metabolomic research, respectively. In addition, the NHLBI, National Institute on Environmental Health Sciences (NIEHS) and National Center for Research Resources (NCRR) ranked in the top five funders for all three fields. Interestingly though the National Institute of Mental Health (NIMH) was the second highest source of funding for genomic biomarker projects (38 grants and $33,994,440 in funding for 2008 and 2009). During this timeframe the relative enthusiasm of the NIMH for analogous proteomic ($2,682,348 in funding) and metabolomic ($1,676,567 in funding) was considerably less. Also of note was the broad interest in biomarker grant by the National Institute of Diabetes and Kidney Disease (NIDDK), which was respectively the fourth ($26,986,008 for 44 grants), seventh ($18,917,354 for 64 grants) and third ($9,388,806 for 26 grants) highest funder of genomic, proteomics and metabolomic awards in 2008 and 2009. When considered in concert, these relative levels of NIH branch funding demonstrate an intense interest in biomarker research employing these recent “–Omics” initiatives. However, if the field of proteomics is used as a case study, the large capital investments summarized have yet to produce many successful, protein-based biomarkers of disease [2].

Figure 3. Summary of the number (a) and funding (b) of NIH funded grants from 2008-2009 for awards containing the term(s) “biomarker”, “biomarker and proteomics”, “biomarker and genomics” or “biomarker and metabolomics”, as derived from the NIH RePORT database.

Figure 4. Comparison of the number NIH grants and scholarly publications containing the term “biomarker” from 1990-2009. Levels were derived from NIH RePORT and PubMed term searches respectively.

Despite this documented difficulty in translational biomarker research [1], intellectual and monetary investments in these initiatives have produced considerable academic outputs in the form of journal publications. A search of the PubMed database (http://www.ncbi.nlm.nih.gov/pubmed/) from 1990 to 2009 for publications containing the keyword “biomarker” in December 2009 yielded a total of 441,510 publications. An annual survey of these relative levels is provided in Figure 4. The overlay of this publication trajectory with the total number of “biomarker” NIH grants previously discussed reveals an identical overall trend of annual expansion. The publication volume outlier in this path lies in the totals for 2009 as the 29,697 publications represents a ∼ 23% drop over the previous year's total. This observed relative reduction in academic productivity may in part be related to the observed decrease in NIH branch funding for “biomarker” research in 2008 (Figure 4), although given the relative complexity of biomarker initiatives such an explanation likely represents an oversimplification of the trend. Regardless of this decrease in recent publication volume, the search revealed that thousands of articles devoted to the study of biomarkers have been published since 1990. Despite this overwhelming productivity, there have been relatively few biomarker successes from these initiatives. Again, considering the case of proteomics, Anderson [2] recently investigated the number of protein biomarkers currently being measured employed in clinical practice. The author subsequently summarized these biopolymers in terms of their annual introduction to reveal that from 1993 to 2008 only a total of 22 protein biomarkers were integrated into patient management. None of these were products of proteomic-based investigations [2]. This observation perhaps lends support to earlier reservations over the degree funding being appropriated to proteomics-based biomarker initiatives [23,24].

Given that the large sums of money (i.e., $2,506, 936,046 in 2008-2009 alone) and academic production (i.e., 441,510 publications from 1990–2009) has not translated into the desired boon in clinically useful biological markers, it would serve the community to review what a biomarker represents and the stages through which they should be evaluated beyond their initial discovery. These initiatives will serve as the focus for the remainder of this review article and attempt to provide context to current disconnect between intellectual biomarker initiatives and the integration of new markers into the clinical therapy.

Definition of a biomarker and a historical overview of its evolution

A recent, extensive, review by Lassere provided a historical overview of the term “biomarker” in the literature [10]. As biomarker research expanded, so did the level of naming and terminology ambiguity within the biomarker literature. In part, this uncertainty over the specific definition of a biomarker, and all related terminology, prompted the NIH to form the Biomarker Definition Working Group, whom was tasked to alleviate this growing confusion and provide clarity to scientists within the field. In 2001 [3] this consortium of individuals proposed specific definitions for a biomarker, clinical end-point and surrogate end-point. According to this group [3] a biomarker is a characteristic that is objectively measured and evaluated as an indicator of normal biological processes, pathogenic processes, or pharmacological response to a therapeutic intervention. A clinical end-point represents a characteristic or variable that reflects how a patient feels, functions or survives. Also, the Biomarker Definition Working Group specifically stated that a surrogate end-point represents a biomarker that is intended to substitute for a clinical endpoint [3]. This point is of particular importance as it is implicate that the term “surrogate” means “to substitute for” and thus for a biomarker to act as a valid alternative end-point it must possess the required specificity for the clinical end-point(s) they are replacing, the class of therapeutic deployed for the clinical intervention, and the population and disease state characteristics for which the surrogacy is being employed [3]. In addition, a surrogate end-point is also expected to predict the clinical benefit (or harm or lack of benefit or harm) based on epidemiological, therapeutic, pathophysiologic, or other scientific evidence [3]. It should be also noted that although biomarkers can be considered surrogate end-points, it is rare for them to attain such status [3], as based on epidemiologic, therapeutic, pathophysiologic or other evidence they must be able to reasonably predict clinical benefit [25]. From the provided biomarker definition though, it is clear that a biomarker may represent a variety of agents which serve as prognostic or diagnostic indicators of disease or a sensitive and specific tool for risk assessment. Such markers may be biological, physical or molecular in nature. Examples of the relative diversity within biomarker classes can be seen in a recent review manuscript [15]. In the years since this important publication by the Biomarker Definition Working Group, these definitions appear to have been accepted by the community as valid respective representations and criterion for the terms, although confusion in translational research remains within the context of biomarker discovery, validation and utility [26–28].

Evaluation schema for putative biomarker candidates

It is imperative that all proposed biomarkers, regardless of intent or nature, demonstrate their utility by both providing a strong biological rationale for investigation and undergoing a rigorous demonstration of the sensitive, selective, accurate and precise assessment of the outcome for which they are proposed to measure. This proof is imperative for further potential applications of putative markers as alternative agents for assessing disease manifestation, risk, and status or therapy efficacy. Several researchers and groups have proposed phases or guidelines for the biomarker evaluation [11,12,16,29–33], including prognostic vs. diagnostic models [13]. Last year the American Heart Association (AHA) released a scientific statement describing the criterion for which novel markers of cardiovascular risk should be evaluated [17]. This schema represents a singular contribution to this broad initiative, with the merits and limitations of select contributions being previously reviewed [15,18]. Such schema represent a step-wise, linear, path for moving a tentative biomarker from its initial discovery phase to its use in clinical management.

Summation and future biomarker outlooks

Going forward, a fundamental understanding of biomarker terminology definitions and appropriate rigorous validation scheme should continue to be implemented, as the challenges facing biomarker research are well known. For example, the review from Ransohoff detailed how the strong claims from both genomic and proteomic biomarker initiatives suffer from poor reproducibility and applicability [18]. It should also not be lost in this discussion that clinical diagnoses and management, based on patient biomarker concentrations, mandate that the marker possess high accuracy, precision, sensitivity and specificity for their intended application. In addition, the modern clinical laboratory places a further substantial burden upon biomarker studies as issues regarding assay platform standardization and analytical performance must be adequately addressed in published studies. Lijmer et al. recently summarized initiatives and proposals for phased evaluations of medical diagnostic tests [34] which should aid in improving assay uniformity within clinical laboratories. Of equal importance are studies investigating the effects of pre-analytical variables, such as sample collection, handling, storage and processing, on the derived biomarker concentration levels. In-vitro concentration perturbations must not be confused or integrated into the interpretation of true, physiologically derived, changes in marker levels. Thus, such studies should be an integral component of all future biomarker discovery and validation undertakings. New biological markers of disease may yet offer further prognostic and diagnostic utility for patient care and it is hoped that the required knowledge summarized within this article will aid scientists in their warranted pursuit of these yet widely elusive indicators.

Table

Summary of NIH Acronyms
AHRQ	Agency for Healthcare Research and Quality
CID	Center for Infectious Diseases
CIT	Center for Information Technology
CLC	NIH Clinical Center
COOPRD	Cooperative Research and Development Agreement
FDA	Food and Drug Administration
FIC	John E. Fogarty International Center for Advanced Study in the Health Sciences
GIFTFND	Gift Fund
NCBDDD	National Center for Birth Defects and Development Disabilities
NCCAM	National Center for Complementary and Alternative Medicine
NCCDPHP	National Center for Chronic Disease Prevention and Health Promotion
NCI	National Cancer Institute
NCMHD	National Center for Minority Health and Health Disparities
NCRR	National Center for Research Resources
NEI	National Eye Institute
NHGRI	National Human Genome Research Institute
NHLBI	National Heart, Lung and Blood Institute
NIA	National Institute on Aging
NIAAA	National Institute on Alcohol Abuse and Alcoholism
NIAID	National Institute of Allergy and Infectious Diseases
NIAMS	National Institute of Arthritis and Musculoskeletal and Skin Diseases
NIBIB	National Institute of Biomedical Imaging and Bioengineering
NICHD	Eunice Kennedy Shriver National Institute of Child Health and Human Development
NIDA	National Institute on Drug Abuse
NIDCD	National Institute on Deafness and Other Communication Disorders
NIDCR	National Institute of Dental and Craniofacial Research
NIDDK	National Institute of Diabetes and Digestive and Kidney Diseases
NIEHS	National Institute on Environmental Health Sciences
NIGMS	National Institute of General Medical Sciences
NIMH	National Institute of Mental Health
NINDS	National Institute of Neurological Disorders and Stroke
NINR	National Institute of Nursing Research
NIOSH	National Institute for Occupational Safety and Health
NLM	National Library of Medicine
OD	Office of the Director
ODNCI	Office of the Director, National Cancer Institute

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.