An appraisal of pivotal evaluation designs in validating noninvasive biomarkers for head and neck cancer detection

Introduction

Obtaining consensual molecular markers for contemporary clinical practice to diagnose head and neck cancer subtypes remains an all-important cause in diagnostic oncology. Biofluid specimens obtained noninvasively have seemed the more likely of the lot for clinical application with major advantages that centers on repeatable collection and representativeness of the diverse phenotypic profile of head and neck tumors [1,2]. With many promising biofluid markers being proffered in scientific reports [3–5], concerns exist regarding the scientific rigor utilized in these biomarker validation endeavors as conventional case–control and retrospective methods are laden with systematic biases that reports’ validity. In 2008, the Early Detection Research Network (EDRN) proposed the prospective-specimen-collection retrospective-blinded evaluation (PRoBE) method for pivotal evaluation of biomarker classification accuracy [6,7]. This modality, which may well represent the most-thorough biomarker validation design available (that mimics real-world application), incorporates a nested case–control approach that proposes biofluid sampling before undertaking confirmatory diagnostic tests. Furthermore, the design requires the evaluation of disease-specific or preferential indicators without knowledge of cancer status or otherwise [7]. In essence, evaluating the current practice of the PRoBE method will be tantamount to assessing the state of bedside readiness for the many biofluid markers suggested for head and neck cancer diagnosis. Hence, this brief report aims at mapping the utilization of the PRoBE protocol for research endeavors involving head and neck cancer biomarkers in noninvasive samples.

Material and methods

Protocol and registration

This mapping review was conducted according to the protocol registered with the International prospective register of systematic reviews (PROSPERO; CRD42020161831) and the Preferred Reporting Items for Systematic reviews and Meta-analysis extension for scoping reviews (PRISMA-ScR) [8].

Search strategy and eligibility criteria

To determine the extent and nature of the application of the PRoBE design to diagnostic head and neck cancer biomarkers and appraise their study methodology, a rapid search of electronic databases including PubMed, EMBASE, Web of Science, Cochrane Library, and LILACS from 1 January 2009 to 30 March 2020 was performed. Search keywords were centered on nonspecific terms including ‘molecular diagnosis, molecular screening, diagnostic biomarkers, tumour OR tumor markers, and head and neck cancer’. Syntaxes were also tailored to the respective databases searched.

Studies were included if they were diagnostic head and neck cancer biomarker studies conducted among adult patients. Head and neck cancers in this study included epithelial malignant neoplasms of the oral cavity, oropharynx, nasopharynx, larynx, hypopharynx, and sinonasal complex. Specifically, eligible studies were based on the collection of noninvasive specimens (external breath, serum or plasma, saliva, urine, and cytologic samples) before tissue biopsy and histopathology, and performance of laboratory analysis while blinded to cancer and non-cancer status. Only articles published in English language were considered. Reviews, news reports, in-vitro and animal experiments, retrospective case–control studies, and single-arm studies were excluded.

Study selection and appraisal

Eligible studies were selected in a two-stage process that includes the screening of titles and abstracts of retrieved citations for relevance to the scope and full-text evaluation of studies successfully screened based on the eligibility criteria. The selection process was independently conducted by two authors and consensual agreement between them formed the basis for final selection.

Risk of bias assessment of eligible studies was conducted using the updated Quality Assessment of diagnostic accuracy studies (QUADAS-2) tool. Evaluation process and scoring assessments were done independently by two authors and disagreements were resolved via consensus following a series of discussions.

Data abstraction and data items

Data extraction for selected articles was also conducted independently and in duplicate using electronically prepared spreadsheets. Items charted included authors, publication year study location, biomarker type, type of specimen, reference standard, and main study conclusions.

Results and discussion

Following deduplication, screening, and eligibility check by two review authors, only 0.5% (n = 3) of 612 articles were found to have utilized both components of the PRoBE validation design. Flow chart indicative of search outputs, selection process, and exclusion basis are shown in Supplementary Figure 1. Interestingly, only one of the selected articles referred to the approach for providing the framework to the validatory investigations underwent. The other two studies were pilot studies to provide pre-validation to the tumor discriminatory analytes [9] and the scientific methods were utilized with no clear classification accuracy estimates reported in these studies [10,11]. The characteristics of the studies included are highlighted in Table 1.

Table 1. Characteristics of head and neck diagnostic biomarker studies utilizing the PRoBE design.

Authors	Publication year	Study location	Head and neck cancer subtype	Noninvasive specimen	Biomarker type	Biomarker(s)	Form of application	Reference standard	Main study conclusion(s)	Reference
Dyckhoff et al.	2011	Germany	Parotid carcinomas	Gland-specific saliva	Proteomic	CA-19-9	Single	Histopathology	Preoperative marker assay may be helpful in discriminating malignant and benign parotid lesions.	[10]
Martin et al.	2015	USA	Oral carcinoma	Whole saliva	Transcriptomic	DUSP1, SAT, OAZ1	Combined	Histopathology	Salivary mRNA markers can differentiate suspicious lesions into high-risk and low-risk classes.	[9]
Szanto et al.	2012	Hungary	Oral carcinoma	Whole saliva	Proteomic	Annexin-1, Peroxiredoxin-2	Combined	Histopathology	Targeted proteomic analysis of saliva may become the method of choice applicable for early OSCC detection.	[11]

All PRoBE studies considered saliva as the biofluid of choice – two of which were conducted using whole saliva samples [9,11] while one favored parotid saliva for detection of salivary gland malignancies [10]. This infers a dearth in study availability for cytologic, blood-, urine-based head and neck cancer biomarkers that utilized both prospective sampling and blinded operator evaluation methods. Further, all studies included were specific for either the diagnosis of oral or salivary gland carcinomas. No study was found for the other head and neck cancer subtypes. Based on the molecular structure of the biomarkers, those verified in the PRoBE studies were proteomic, peptidomic, or transcriptomic (coding) in nature. No studies were found available for the many epigenomic, genomic, non-coding RNA, metabolomic, and microbiome molecular analytes that have been suggested thus far.

Appraising the methodological quality of these studies using the risk of bias domain of the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool showed that two studies had a low risk of bias in three of four domains [9,10]. Unclear risk of bias in patient selection precluded them from being evaluated as low-bias studies solely due to the non-reporting of their participant sampling methods (Table 2). Hence, it may be that utilizing the PRoBE method increases the likelihood of obtaining high-quality efficacy reports for pooled analysis.

Table 2. Risk of Bias domain of the QUADAS-2 tool for methodological assessment of PRoBE studies.

Study authors (year)	Risk of bias
	Patient selection	Index test	Reference standard	Flow and timing
Dyckhoff et al. (2011)	?	☺	☺	☺
Martin et al. (2015)	?	☺	☺	☺
Szanto et al. (2012)	☹	☹	?	?

☺ low risk; ☹ high risk; ? unclear risk.

Overall, the very low proportion of biomarker validation studies based on the PRoBE protocol discloses that a systematic review and meta-analysis on the evaluation of classification accuracy estimates in studies using the design may not be a feasible endeavor at this present time. Moreover, this reveals that the many promising markers for head and neck cancer have not been evaluated with the most-rigorous methods available. Therefore, to provide a clear progression to obtaining valid putative biofluid markers, we suggest more emphasis on validation methods based on the PRoBE designs to follow the series of pre-validatory investigations conducted in the last decade. Notably, these investigations are now more warranted to validate the use of molecular markers in blood, urine, and external breath specimens, and biomarkers proposed for the consensual diagnosis of head and neck malignancies. Likewise, rigorous validations are required for biomarkers specific for head and neck cancer subtypes other than oral cavity and oropharyngeal carcinomas.

Author contributions

JA conceptualized, performed the scoping search and wrote the initial draft of this manuscript. CCJW performed the scoping search and edited the original draft of the manuscript. PT was responsible for conceptualization, supervision, oversight, and review of the manuscript. All authors read and approved the final manuscript.

Disclosure statement

The authors declare that they have no competing interest.