Abstract
In absence of a gold standard for chronic obstructive pulmonary disease (COPD) it remains difficult to compare the true diagnostic characteristics of the forced expiratory volume in 1 second to the forced vital capacity (FEV1/FVC) <0.70 and < lower limit of normal (LLN). COPD is a clinical diagnosis, based on symptoms signs and lung function results combined, and an expert panel assessment would be an adequate reference standard. We compared the diagnostic properties of FEV1/FVC <LLN and <0.70 against this panel diagnosis: 342 participants, aged >50, consulting for persistent cough, but without physician-diagnosed COPD, were prospectively enrolled. All underwent extensive history taking, physical examination, spirometry and diffusion testing. An expert panel, including a board certified respiratory physician, assessed all diagnostic information to determine the presence or absence of COPD and served as reference standard. Then, 104 participants were diagnosed with COPD by the panel. The reproducibility of the panel diagnosis was high (kappa of 0.94). Sensitivity estimates of <0.70 were significantly higher than that of <LLN (0.73 and 0.47, respectively, p < 0.001). The fixed approach was less specific than the LLN (0.95 and 0.99, respectively, p < 0.001). There was no significant difference in diagnostic property when using pre- or post-bronchodilator FEV1/FVC (p = 0.615). In a symptomatic primary care population, the FEV1/FVC <0.70 was more accurate to detect COPD.
Introduction
A diagnosis of COPD is usually confirmed by spirometric airflow obstruction defined as a lowered ratio of the forced expiratory volume in one second to the forced vital capacity (FEV1/FVC ratio). A debate exists on the appropriate threshold value: the lower limit of normal (LLN), which represents the lowest 5th percentile according to age and gender, or the fixed 0.70 value (Citation1, 2). The ATS and ERS guidelines advocate using the LLN (Citation3), whereas the Global Initiative for Chronic Obstructive Lung Disease (GOLD) the 0.70 value (Citation4).
Many studies compared the two thresholds (Citation5), but unfortunately, did not compare them with an independent gold standard for COPD, but rather selected one of the proposed thresholds as the reference test. The reference test unavoidably is superior in such analyses, rendering the outcome of these comparative studies predictable: nominating e.g., the <0.70 threshold as a reference leads unavoidably to an outcome of an inferior LLN and vice versa (Citation5).
A preferable design would be to compare both thresholds with an independent reference standard for COPD (Citation6). COPD is a clinical diagnosis and is based on symptoms, signs and lung function results combined. A panel diagnosis, taking into account other relevant clinical factors / parameters, is the best way forward (Citation7). To the best of our knowledge, no studies have been published comparing the LLN and <0.70 approaches against a panel diagnosis of COPD.
The objective of the current study was to compare the two mentioned thresholds of FEV1/FVC against a panel diagnosis of COPD as the reference standard. Secondly, we investigated whether the diagnostic characteristics were influenced by using pre- and post-bronchodilator values.
Methods
Design and study population
A diagnostic study was performed between 2006 and 2009 in the Netherlands, the FRESCO study (From Respiratory Symptoms to COPD). The study protocol is described in detail elsewhere (Citation8). Inclusion criteria for patients were: an age over 50 years and consulting their general practitioner (GP) for cough lasting ≥14 days. Exclusion criteria were physician diagnosed COPD, suspected pneumonia, and terminal illness. All participants gave written informed consent and the ethics committee of the University Medical Center Utrecht approved the study.
Diagnostic work-up
On the day a patient presented with cough, a standardized history was taken (including smoking habits and respiratory complaints) and a full physical examination was performed. The use of medication and comorbidities were registered from the GP's medical file. Subsequently, all patients underwent further extensive diagnostic work-up, including full lung function testing (spirometry, body plethysmography and diffusing capacity of the lung for carbon monoxide by the single breath method) 90 days after the initial visit to their GP to ensure stable phase measurements.
Lung function results were obtained before and after bronchodilation with 400 microgram of salbutamol, and expressed as percentage of predicted for age, gender and height (Citation9). Lower limits of normal (LLN) were calculated by using the reference equations from the European Coal and Steel Community (ECSC) (Citation9). Finally, clinical follow-up information, like new diagnoses or hospital admissions, was provided by the participants’ GP.
Panel diagnosis of COPD
We used an outcome panel of two physicians, one GP with expertise in COPD and a board certified respiratory physician to decide by consensus and using (inter)national guidelines whether COPD or asthma was present or not (Citation4, Citation10). The decisions were determined during a meeting in which all available patient information was presented. All lung function test results were available to the panel.
For a diagnosis of COPD, recurrent complaints of cough, sputum or breathlessness were obligatory, as well as a post bronchodilator obstruction. A history of smoking was supportive but not obligatory for COPD. Obstruction was defined as a lowered FEV1/FVC ratio and a concave dip in the second part of the curve. However, no strict threshold of FEV1/FVC was applied for the presence of obstruction; rather, this was assessed per individual. As an illustration, a 52-year-old woman with recurrent complaints of cough and dyspnoea who had smoked 25 pack-years, with a post-bronchodilator FEV1/FVC ratio of 0.71 and an FEV1 of 80% of predicted, was diagnosed with COPD because both a lowered diffusion capacity (transfer coefficient for carbon monoxide (Kco) = 70% of predicted) and an increased residual volume (160% of predicted) supporting this diagnosis. When no consensus was reached, a third physician (BB) could be consulted.
Reproducibility of the expert panel
The reproducibility of the panel was estimated by repeating the consensus diagnosis procedure after more than a year, of a random sample of 41 patients, without information on the original diagnosis, resulting in Cohen‘s kappa of 0.94 (good). The panel included an independent respiratory physician not involved in the initial panel diagnosis.
Statistical analysis
Mean and standard deviation (SD) were calculated for normally distributed continuous variables and proportions or frequencies for categorical variables. Differences between groups were evaluated using (un)paired T-tests or analysis of variance where appropriate; for categorical variables the χ2 or the McNemar test was used (the latter for within-subject comparisons). 95% confidence intervals (CI95%) were calculated where appropriate (Citation11).
The post-dilator FEV1/FVC was dichotomized based on a] the GOLD 0.70 threshold and b] the appropriate European Coal and Steel Community (ECSC) lower limit of normal (LLN) (Citation9). Using the panel diagnosis of COPD as a reference, the sensitivity and specificity for these categorical variables were estimated. The differences between ‘LLN’ and ‘GOLD’ COPD sensitivity and specificity were evaluated using the McNemar test for paired proportions (Citation12). CI95% of the differences was calculated (Citation11). Diagnostic accuracy was defined by the sum of the number of true positives and true negatives divided by the total number of subjects.
Again using the panel diagnosis of absence / presence of COPD as reference, the area under the receiver operating characteristic curve (ROC area) of the continuous pre/post-dilator FEV1/FVC values was estimated. Differences with CI95% between the two resulting ROC areas were evaluated with the method of Metz (Citation13), which adjusts for the fact that variables are strongly correlated. CI95% of the differences was calculated. P-values <0.05 were considered statistically significant. All statistical analyses were performed using SPSS 18 for Windows (SPSS, Chicago, Illinois, USA).
Results
Baseline demographics
Of the 400 participants initially included, 58 were excluded (40 due to missing / incomplete results from lung function tests and 18 based on other reasons (death, poor health, moving, etc.), see . Of the remaining 342 participants, 53.5% were females and the mean (SD) age was 63.0 (8.3) years. Detailed demographics and spirometry, stratified by COPD-status is provided in .
Figure 1. Flow chart of inclusion criteria and number of participants included.
Table 1. Participants demographics and results from the pulmonary function tests stratified the panel diagnosis
Diagnosis of COPD: FEV1/FVC <0.70 versus <LLN
COPD was present in 104 participants (30.5%). Those diagnosed with COPD were more often male of higher age or current smoker (). The sensitivity of FEV1/FVC <0.70 for diagnosing COPD was significantly higher than that of FEV1/FVC <LLN 0.73 and 0.47, respectively (p < 0.001; CI95% 0.18 –0.34). The specificity of FEV1/FVC <LLN was slightly, but significantly higher compared to FEV1/FVC <0.70: 0.99 and 0.95, respectively (p < 0.001; CI95% 0.01 –0.06). The positive and negative predictive values of <0.70 and <LLN are provided in .
Table 2. Positive and negative predictive values of FEV1/FVC <0.70 and <LLN for COPD, (post-bronchodilator values)
A larger number of participants had COPD according to FEV1/FVC <0.70 compared to <LLN, 88(25.7%) and 52(15.2%), respectively. The LLN-approach ‘missed’ 55 participants with COPD, the GOLD approach 28, when compared to the expert panel diagnosis. The false-positives and false-negative rates according to either the LLN and <0.70 are presented in . The diagnostic accuracy of the <0.70 was higher than that of <LLN, 91.2% and 83.0%, respectively.
Table 3. Number (percentage) of correct and wrong diagnoses of COPD according to FEV1/FVC <0.70 and <LLN (post-bronchodilator values)
Diffusion tests and bodyplethysmography
The Kco% predicted in participants with FEV1/FVC <0.70 was significantly lower than those with FEV1/FVC >0.70, 87.6% (19.5) and 97.6% (17.8), p < 0.001, respectively. Comparable results were found using the LLN as threshold, 86.4% (20.4) and 96.4% (18.1), p = 0.005, respectively.
The RV/TLC% ratio, which represents the degree of air trapping, was significantly higher in participants with FEV1/FVC <0.70 than those with FEV1/FVC >0.70, 43.2% (7.2) and 38.2% (8.0), p < 0.001, respectively. Similar results were seen in participants with FEV1/FVC <LLN and with >LLN, 44.9% (7.4) and 38.6% (7.4), p < 0.001, respectively.
Pre- versus post-bronchodilator values
The difference between the ROC area of the pre-and post-bronchodilator FEV1/FVC values was not significant, 0.929 and 0.933, respectively (δ = 0.004, p = 0.615; CI95% -0.02 –0.01) (). For the FEV1/FVC <0.70 12.3% of the participants had a change in their COPD status, while this was 6.5% for the FEV1/FVC <LLN (). Also for the FEV1 as percent of predicted, there were no significant differences (δ = 0.013; p 0.336; CI95% -0.026 –0.009) in the ROC area between pre- and post-bronchodilator values, 0.862 and 0.849, respectively ().
Figure 2. Receiver operating curve (ROC) for pre- and post-bronchodilator FEV1/FVC values in diagnosing COPD.
Figure 3. Receiver operating curve (ROC) for pre- and post-bronchodilator FEV1 [L] values in diagnosing COPD.
![Figure 3. Receiver operating curve (ROC) for pre- and post-bronchodilator FEV1 [L] values in diagnosing COPD.](/cms/asset/31b9656f-3873-4dcb-98f5-540d0adcbb60/icop_a_667851_f0003_b.gif)
Table 4. Number of participants changing from below the thresholds to above the thresholds or vice versa after the administration of a bronchodilator according to FEV1/FVC <0.70 and <LLN
Participants with COPD had significantly greater degree of reversibility, i.e., increase in FEV1 as percentage of predicted according to age gender and height (). The discrimination (ROC area) of this reversibility for diagnosing COPD was poor (0.622).
Discussion
Main results
In the current study we compared FEV1/FVC <0.70 with <LLN for diagnosing early COPD, using a panel diagnosis of COPD as reference, in subjects presenting with persistent cough. We showed that the FEV1/FVC <0.70 sensitivity was higher compared to the LLN, while specificity estimates were comparable, resulting in more ‘missed’ cases using the LLN. Secondly, we showed that using post-bronchodilator values for the FEV1/FVC or FEV1 did not increase discrimination between presence or absence of COPD compared to pre-bronchodilator values.
Comparison with other studies
A large number of studies showed that COPD prevalence rates were higher when applying the FEV1/FVC <0.70 compared to <LLN and interpreted this as “overdiagnosis” of COPD resulting from using a fixed value of <0.70. Our results show the same pattern; however, we stress that the results of any head to head comparison of two thresholds or tests is predictable and without a proper reference test, or gold standard, it is impossible to label either of them as superior. Furthermore, most previous studies used open populations including asymptomatic subjects in which the risk of overdiagnosis is greater than in our study population (Citation5).
To our knowledge no previous studies reported the comparison of FEV1/FVC <0.70 and <LLN to a panel diagnosis of COPD. The arguments in the past that the using <0.70 as threshold leads to overdiagnosis is not substantiated by our data: the specificity of the two thresholds do not differ much and the negative predictive value of the <0.70 was higher than of the <LLN. In addition, the results from the diffusion tests, reflecting the functional capacity of the lungs, did not substantially differ between participants with an FEV1/FVC <0.70 and <LLN
One previous study compared both thresholds against prognostic outcomes and found that participants labelled healthy according to the <LLN approach, but diseased according to the <0.70 had a higher mortality risk and suffered from more COPD-related hospital admissions compared to subjects with a normal lung function (Citation14). Another, cross-sectional, study reported a worse quality of life in subjects with comparable lung function results (<0.70 but >LLN) (Citation15). Mannino et al. furthermore recently showed that patients, considered ‘being non-diseased’ using the LLN approach, show an increased mortality risk, based on data from the Third National Health and Nutrition Examination Survey (NHANES III) (Citation16). These studies suggest that the LLN may miss affected subjects.
Implications for practice
Even the <0.70 threshold missed a considerable number of subjects with COPD. This illustrates that COPD is a clinical diagnosis in which physicians must consider more characteristics than just the FEV1/FVC ratio. Subjects with still normal FEV1/FVC ratios, either according to 0.70 or LLN, may still be labelled as having COPD, based on abnormal diffusion capacity tests and/or bodyplethysmography results.
Our results suggest that there is no need to obtain also post-bronchodilator values for the diagnosis of COPD in a symptomatic population with persistent cough. A number of participants was reclassified from below to above the thresholds or vice versa, but this did not result in significant changes in the diagnostic characteristics of pre- and post-bronchodilator FEV1/FVC values. This is in contrast with the recommendation of the GOLD committee to use post-bronchodilator values of FEV1/FVC. Our finding is of interest because only performing pre-bronchodilator spirometry will shorten the time needed to perform spirometry. Especially in a primary care setting this could enhance the use of spirometry and early detection of COPD. However, it is uncertain whether these results are generalisable to other populations than our study patients.
The poor discrimination (ROC area) of the of the FEV1-reversibility as sole diagnostic in diagnosing COPD in our study underscores the recent amendment by the GOLD committee that the degree of reversibility is not longer a prerequisite in the diagnosis of COPD (Citation17).
Strong and weak points of the study
We included participants in whom the diagnostic dilemma is most urgent: middle aged and elderly subjects without physician diagnosed COPD who seek medical help because of persistent respiratory complaints (cough). Other studies mainly analysed open population subjects (Citation5).
A second strength is that we validated the panel diagnosis of COPD by repeating the complete diagnostic process in a random sample of subjects. The validity of the ‘reference standard’ used in the current study, i.e., the panel diagnosis, is of key importance when interpreting the outcomes of this study. The panel diagnosis provided the ‘best’ available diagnosis of COPD, which was a combined assessment of all symptoms, signs, spirometry tests outcomes (including the shape of the flow-volume curve) and diffusion tests outcomes. The reproducibility was high (kappa = 0.94). Moreover, the reproducibility of the panel diagnosis was tested by an independent physician not being involved in the first panel diagnosis (inter-observer agreement).
We acknowledge that the panel diagnosis still may not be the ‘perfect’ diagnosis. The panel had knowledge of the FEV1/FVC value, which could introduce bias. This possible ‘imperfect standard bias’ however probably did not bias the ranking of the thresholds in terms of sensitivity / specificity. Because FEV1/FVC <0.70 and <LLN were both compared against this panel diagnosis both suffered to the same extent from its possible diagnostic inaccuracy. In other words, although the panel diagnosis cannot be 100% correct, the ranking of the diagnostic capability of FEV1/FVC <0.70 and <LLN remains valid (Citation12).
Conclusions
In our study population the fixed criterion showed better sensitivity rates when compared to the LLN, while specificity rates were comparable. Second, there were no differences in diagnostic accuracy between pre- and post-bronchodilator values.
Declaration of Interest
The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper. This work was funded by the Netherlands Organisation for Health Research and Development (ZonMw), Grant No. 945-04-015.
Acknowledgments
FMH, PZ, AS, BB contributed in conception, design and writing of the first version of the manuscript and are guarantor of the paper. Data collection was performed by BB, AS and. PZ, FMH and BB were responsible for the data analyses. TV and JWL contributed to writing. All authors approved the final version of the manuscript.
References
- Enright P, Brusasco V. Counterpoint: should we abandon FEV/FVC < 0.70 to detect airway obstruction? Yes. Chest 2010; 138:1040–1042.
- Celli BR, Halbert RJ. Point: should we abandon FEV/FVC <0.70 to detect airway obstruction? No. Chest 2010; 138:1037–1040.
- Brusasco V, Crapo R, Viegi G. Coming together: the ATS/ERS consensus on clinical pulmonary function testing. Eur Respir J 2005; 26:1–2.
- Rabe KF, Hurd S, Anzueto A, Barnes PJ, Buist SA, Calverley P, Fukuchi Y, Jenkins C, Rodriguez-Roisin R, Van WC, Zielinski J. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease: GOLD executive summary. Am J Respir Crit Care Med 2007; 176:532–555.
- Mohamed Hoesein FA, Zanen P, Lammers JW. 2011. Lower limit of normal or FEV(1)/FVC <0.70 in diagnosing COPD: An evidence-based review. Respir Med 2011; 105(6):907–15.
- Reitsma JB, Rutjes AW, Khan KS, Coomarasamy A, Bossuyt PM. A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. J Clin Epidemiol 2009; 62:797–806.
- Weller SC, Mann NC. Assessing rater performance without a “gold standard” using consensus theory. Med Decis Mak 1997; 17:71–79.
- Broekhuizen BD, Sachs AP, Hoes AW, Moons KG, van den Berg JW, Dalinghaus WV, Lammers E, Verheij TJ. Undetected chronic obstructive pulmonary disease and asthma in people over 50 years with persistent cough. Br J Gen Pract 2010; 60:489–494.
- Quanjer PH, Tammeling GJ, Cotes JE, Pedersen OF, Peslin R, Yernault JC. Lung volumes and forced ventilatory flows. Report Working Party Standardization of Lung Function Tests, European Community for Steel and Coal. Official Statement of the European Respiratory Society. Eur Respir J Suppl 1993; 16:5–40.
- Levy ML, Quanjer PH, Booker R, Cooper BG, Holmes S, Small I. Diagnostic spirometry in primary care: Proposed standards for general practice compliant with American Thoracic Society and European Respiratory Society recommendations. Prim Care Respir J 2009; 18:130–147.
- Altman DG, Machin D, Bryant TN, Gardner MJ, editors. Statistics with confidence, 2nd ed. BMJ Books, London; 2000.
- Zhou XH, Obuchowski NA, McClish DK, editors. Statistical Methods in Diagnostic Medicine, 1st ed. John Wiley & Sons Inc, New York; 2002.
- Metz CE, Herman BA, Roe CA. Statistical comparison of two ROC-curve estimates obtained from partially-paired datasets. Med Decis Mak 1998; 18:110–121.
- Mannino DM, Sonia BA, Vollmer WM. Chronic obstructive pulmonary disease in the older adult: what defines abnormal lung function? Thorax 2007; 62:237–241.
- Garcia-Rio F, Soriano JB, Miravitlles M, Munoz L, Duran-Tauleria E, Sanchez G, Sobradillo V, Ancochea J. Subjects “over-diagnosed” as COPD by the 0.7 fixed ratio have a poor health-related quality of life. Chest 2011; 139(5):1072–80.
- Mannino DM, Diaz-Guzman E. Interpreting lung function data using 80% predicted and fixed thresholds identifies patients at increased risk of mortality. Chest 2012; 141(1):73–80.
- GOLD. From the Global Strategy for the Diagnosis, Management and Prevention of COPD, Global Initiative for Chronic Obstructive Lung Disease (GOLD) 2011. Available from: http://www.goldcopd.org/.