ABSTRACT
Human immunodeficiency virus (HIV) is associated with increased risk for chronic obstructive pulmonary disease (COPD); yet substantial under-recognition of COPD exists. We administered a patient-completed, physician-reviewed COPD screening tool in an outpatient HIV clinic to determine whether screening is feasible or possible. Patients attending nonacute, routine HIV care visits were provided a brief COPD screening tool, which included three questions focused on age, respiratory symptoms, and smoking history. Providers were given completed forms for review and ordered spirometry at their discretion. Forms and medical records were subsequently reviewed to determine completion and results of spirometry testing. Of the 1,510 patients screened during the study period, 968 (64%) forms were completed. After excluding 79 incomplete forms, 889 (92%) unique patient forms were included in this analysis. Among these, 204 (23%) met criteria for spirometry referral, among whom physicians ordered spirometry in 64 (31%). At 6 months following study completion, 19 (30%) of the patients referred for spirometry had the test completed, with 5 (26%) demonstrating airflow obstruction. Nearly one out of four HIV patients met indication for screening spirometry and roughly one out of four undergoing spirometry had COPD. Critical drop-offs in the screening and diagnostic process occurred at questionnaire completion and spirometry ordering. Interventions tailored to these critical steps could improve the yield from COPD screening and help to optimize the identification of COPD in high-risk HIV-infected populations. COPD screening in a clinic focused on longitudinal HIV care can effectively identify COPD among those completing the screening continuum.
KEYWORDS:
Introduction
Human immunodeficiency virus (HIV) infection is increasingly recognized as a chronic illness. With HIV-infected patients living longer, there has been a shift in morbidity and mortality to noninfectious conditions (Citation1, Citation2). Despite increased longevity among patients living with HIV Citation(3), accelerated senescence Citation(4) and resultant premature co-morbid diseases Citation(5) have become of elevated concern both to HIV patients and their care providers.
Guidelines for the care of HIV-infected patients have evolved to include recommendations regarding screening and managing co-morbid conditions such as dyslipidaemia, insulin resistance, coronary heart disease, renal disease, and malignancies Citation(6). HIV has been shown to be an independent risk factor for chronic obstructive pulmonary disease (COPD), yet screening guidelines for COPD are lacking for this population (Citation7, Citation8). In contrast, well-defined, evidence-based guidelines for COPD screening for the general population of smokers are available as part of the Global Initiative for Chronic Obstructive Lung Disease (GOLD) strategy for COPD diagnosis Citation(9). In prior studies, we documented that a majority of HIV-infected participants with obstructive lung disease has not been previously diagnosed by a provider Citation(8).
Targeted screening among symptomatic smokers is recommended in order to diagnose and treat COPD and thereby reduce morbidity (Citation4, Citation10). In order to determine the impact and feasibility of screening for COPD in an HIV population, we implemented a COPD screening tool in an urban clinic providing primary care for patients with HIV.
Methods
Study population
Participants were drawn from the Johns Hopkins HIV clinic in Baltimore, MD, which provides longitudinal, primary care for approximately 3,000 patients with HIV. Staffed by more than 25 providers, the clinic integrates specialty services including subspecialty care, pharmacy, nutrition, substance abuse, and adherence support. Multidisciplinary care also incorporates counseling, case management, and social work. All patients attending nonurgent, routine HIV care visits between 1 April and 30 June 2013 at the Johns Hopkins HIV clinic were considered eligible for this study. Repeat visits were excluded. Conduct of this study was in accordance with the ethical standards of the Helsinki Declaration of 1975, as revised in 2000, and was approved by the Johns Hopkins Institutional Review Board.
COPD screener tool design
A COPD screening questionnaire for the HIV population has not been developed; therefore, our COPD screener tool was designed for use specifically within this study based upon review of published COPD screening tools for the general population (Citation4, Citation11–15). Key overlapping content domains among screening instruments included demographics, respiratory symptoms, and smoking history. The COPD screener tool was a single-page questionnaire that included three questions: “Are you 40 years or older?”, “Have you smoked more than 100 cigarettes (5 packs) in your entire lifetime?”, and “Do you have any of the following on a regular basis: cough, phlegm or mucous, wheezing, shortness of breath when walking up a slight hill or hurrying on flat ground?” Participants could record answers to each question indicated via Yes/No check boxes (Supplementary ).
Figure 1. Flow diagram of COPD screener tool, from administration to COPD diagnosis. Each box includes the number of patients completing the specified step in the continuum of COPD screening and diagnosis. Percentages represent the proportion of patients completing the preceding step. COPD=chronic obstructive pulmonary disease; MD=medical doctor; PFTs=pulmonary function tests.
COPD screener administration
Providers and staff were educated regarding appropriate questionnaire completion, flow through the screening process, and anticipated provider involvement during their weekly clinical case conference and through direct interaction. Patients were provided the questionnaire in paper form at the time of clinic registration and counseled regarding the purpose of the questionnaire by the registration staff. Participants completed the questionnaire independently and returned the questionnaire to their provider during the clinic visit. Those with impaired vision or reduced health literacy could request assistance with form completion.
The COPD screener tool advised providers that patients answering “Yes” to all three questions warranted consideration for spirometry testing to screen for COPD consistent with current guidelines (4, 10). Providers were provided with five choices to document their decision: (1) spirometry ordered, (2) spirometry not indicated (i.e., symptoms from another cause),(3) COPD diagnosis already established, (4) patient testing deferred (with text field to indicate deferral reason), and (5) patient does not meet all three screening criteria listed. Referrals or orders of any necessary testing were at the discretion of the provider. No additional intervention occurred by study staff. At the end of each clinic session, all forms were reviewed by study staff to determine appropriate completion, with data abstracted and entered into a database. Six months following study completion, medical records were accessed to extract documentation of spirometry ordering, scheduling, and measurement results. COPD was defined as a prebronchodilator [forced expiratory volume in 1 second/forced vital capacity (FEV1/FVC)] ratio < 0.70.
Statistical analyses
Our primary outcome is the identification of the critical step-offs in the continuum of COPD screening, which we define as the screening steps required to lead to a diagnosis of COPD. Descriptive characteristics of the study population at stages in the continuum of COPD screening and referral are presented as frequencies, means (standard deviation [SD]) when normally distributed, or medians (interquartile range [IQR]) when the distribution was skewed. Characteristics were compared using the t test, Wilcoxon rank-sum test, or Pearson χ2, as appropriate. A two-sided p-value ≤ 0.05 was used to define statistical significance. Statistical analyses were conducted using Stata version 12.0 (Stata Corporation, College Station, TX, USA).
Results
Implementation of the COPD screener tool
Study flow from COPD screener tool administration to COPD diagnosis is illustrated in . Between 1 April and 30 June 2013, 1,510 patients were seen in the Johns Hopkins HIV clinic for nonurgent, routine HIV care. Forms were returned by 968 patients (64% of the 1,510 seen in clinic). After excluding 79 incomplete forms, a total of 889 initial screening visits were conducted (92% of the 968 administered forms). Of these, 889 initial screening visits, 204 patients (23%) met criteria for spirometry to screen for COPD based on their positive responses to all three questions on the COPD screener tool. Spirometry was ordered for 64 (31%) of the 204 patients recommended to undergo testing; 19 (30%) of those patients in whom it was ordered completed spirometry testing. Of the remaining 140 patients who met indications for spirometry yet spirometry was not ordered, the most common reason for spirometry deferral was left blank (n = 63, 45%). Among those for whom a deferral reason was listed (n = 77), the most common deferral reasons included a known diagnosis of COPD (n = 29, 38%), presence of an acute medical issue (n = 13, 17%), inappropriate testing as per the visit provider (n = 11, 14%), or prior Pulmonary Function Test (PFT) order or performance (n = 11, 14%). Ultimately, COPD was diagnosed in 5 out of the 19 patients (26%) undergoing spirometry testing.
Figure 2. Critical step-offs in the COPD screening process. Each bar represents the percent of participants completing the specified step in the continuum of COPD screening out of the total number of patients screen. COPD=chronic obstructive pulmonary disease.
Patient characteristics
The mean age of the 889 patients completing initial screening visits was 50 years (SD 11) (). The majority were male (65%) and black (80%). Among patients completing spirometry, the mean age was 54.1 years (SD 6.0); 63% were male and 95% were black. These characteristics were not statistically different from those patients who did not complete spirometry. Patients who were diagnosed with COPD on spirometry had similar mean age (54.8 years versus 53.8 years; p = 0.76), proportion of males (60% versus 64%; p = 0.87), and proportion of blacks (100% versus 92%; p = 0.54) compared to patients who did not have COPD evident on spirometry.
Table 1. Demographic and spirometric characteristics of participants at stages of COPD screening.
Spirometric measurements among screened patients
Among the 19 patients completing spirometry, the mean FEV1/FVC ratio was 0.75 (SD 0.08), mean FEV1 was 2.53 L (SD 0.8), and mean FEV1% predicted was 83.7 (SD 17, ); five patients (26%) met spirometric criteria for COPD. The mean FEV1/FVC ratio was 0.64 (SD 0.04) and mean FEV1 was 2.08 L (SD 0.5) among the five patients diagnosed with COPD through the COPD screener tool. For participants meeting COPD criteria, the degree of impairment was mild (FEV1% predicted ≥ 80%) in one and moderate (50–80% predicted) in four. For those undergoing spirometry who were not found to have COPD, the mean FEV1/FVC ratio was 0.79 (SD 0.05) and mean FEV1 was 2.69 L (SD 0.8).
Comparison of spirometric measurements between screened and routine care patients
Through review of the Johns Hopkins Hospital Pulmonary Function Laboratory database, we identified 47 HIV-infected individuals receiving care in the HIV clinic that had undergone spirometry testing in the 12 months prior to the COPD screening study. Among these 47 individuals, the majority were male (57%) and black (89%) with a mean age of 52.2 years (SD 8.9). The mean FEV1/FVC ratio was 0.71 (SD 0.10), mean FEV1 was 2.37 L (SD 0.8), and mean FEV1% predicted was 80.1 (SD 22); 11 patients (23%) met spirometric criteria for COPD. For those 11 patients meeting spirometric criteria for COPD, the mean FEV1/FVC ratio was 0.60 (SD 0.08), mean FEV1 was 1.62 L (SD 0.6), and mean FEV1% predicted was 53.7 (SD 18). Among these 11 patients, the degree of airflow obstruction was mild in one (FEV1% predicted ≥ 80%), moderate in five (50–79% predicted), severe in four (30–49% predicted), and very severe in one (<30% predicted). For those undergoing spirometry who were not found to have COPD, the mean FEV1/FVC ratio was 0.79 (SD 0.05), mean FEV1 was 2.59 L (SD 0.7), and mean FEV1% predicted was 88.1 (SD 16). Compared to the 19 individuals completing spirometry prompted by screening, these 47 individuals were of similar age, race, and gender with similar mean spirometric values (p > 0.05 for all comparisons). Defining airflow obstruction using a fixed ratio versus lower limit of normal definition Citation(16) resulted in a similar classification of 95% of participants undergoing spirometry.
Continuum of COPD screening and diagnosis
highlights critical step-offs in the COPD screening process by displaying the percentages of the 1,510 nonacute, routine HIV clinic patients passing through each step of the COPD screener tool continuum to COPD diagnosis. The first step-off in the continuum of screening to diagnosis occurred with form completion, which resulted in a loss of 36% of eligible patients. Another 8% of patients were removed from the continuum of COPD screening due to incomplete COPD screener tool completion. Although a step-off is noted between screening questionnaire completed and meeting indication for spirometry, this is reflective of the patient's response to COPD screening tool questions, not a deviation from the screening process. The third decrement in flow through the screening to diagnosis process occurred between meeting an indication for COPD screening spirometry and having spirometry ordered by the provider. After excluding the 40 individuals with a known COPD diagnosis or with spirometry previously performed or ordered, 39% of participants meeting indication for spirometry had spirometry ordered by the provider during the visit.
Discussion
In this study of 1,510 HIV-infected individuals attending nonurgent primary care visits, we demonstrated that delivery of a simple, patient-completed COPD screening tool can be effectively deployed in an HIV specialty clinic. While there were substantial barriers in the steps of the COPD screening to diagnosis continuum, the yield of COPD diagnoses (26%) among persons completing the screening process was high. We observed that approximately one out of every four HIV-infected individuals completing the COPD screener tool met indication for spirometry screening. Among individuals completing screening spirometry, approximately one out of every four met spirometric criteria for a diagnosis of COPD. Based on flow through our study, we identified several stages in the continuum of COPD screening and diagnosis where effectiveness could be improved, namely by targeting at the level of patients for questionnaire completion and providers for spirometry ordering.
In recent years, screening for a number of treatable noncommunicable diseases for which HIV patients are at increased risk is recommended, but does not yet include chronic lung disease Citation(17). Current recommended chronic disease screening tests for HIV patients range from simple blood tests to more involved patient questionnaires Citation(18). Our patient-administered COPD screening tool provides a simple, brief, three-question instrument that can be completed by patients in the waiting room in just a couple of minutes. The tool prompts providers to consider a diagnosis of COPD in a high-risk population for whom diagnosis would result in timely initiation of COPD therapy, as well as appropriate risk stratification for COPD-related complications, particularly exacerbation. Based on the different approaches for managing bacterial pneumonia and acute exacerbations of COPD, identification of underlying COPD among HIV-infected patients may also improve the management of acute respiratory symptoms Citation(19).
A validated COPD screening tool within an outpatient population of HIV-infected patients does not exist. Tools validated in the general population, such as the COPD-Population Screener (PS) Questionnaire screening questionnaire Citation(13), have low sensitivity (20%) and specificity (79%) when applied to an HIV population Citation(20), underscoring the need for an HIV-specific COPD screening tool. Our tool includes factors common to the validated tools available for COPD screening in the general population and was streamlined to improve efficiency in a busy clinic providing longitudinal HIV care. Therefore, our findings identifying the critical step-offs in the COPD screening process represent the utility of our tool and cannot be applied to other existing tools. Notably, in an observational cohort of HIV-infected participants, 15% of those with respiratory symptoms had obstructed spirometry Citation(21) as compared to our finding that 21% of those with symptoms as identified through our COPD screener tool had obstructed spirometry. This suggests that in an HIV-infected population, respiratory symptoms may not represent the best screening criteria for COPD. It is important to note that because our study was designed to determine the critical step-offs in the COPD screening continuum, we excluded those with a known diagnosis of COPD or met other provider determined criteria to defer spirometry testing, thereby reducing the prevalence of disease in our population and the positive predictive value of the COPD screener tool.
We provided printed questionnaires to patients at the time of check-in to the clinic. However, with the advancement and widespread uptake of electronic medical record (EMR) systems and the addition of EMR-based reminder systems, an EMR-based COPD screener tool could easily be integrated into routine medical care. EMR-based reminder systems have been shown to increase provider adherence to disease-specific guideline recommendations Citation(22), as well as ambulatory preventative care guidelines Citation(23). Integration of the COPD screener tool into an EMR-based reminder system with efficient linkage to provider order sets could improve timely and appropriate ordering of screening spirometry. The critical drop-offs in the flow through our screening process occurring at form completion and spirometry ordering steps could potentially be remedied to some degree through EMR integration.
Within general U.S. populations, airflow obstruction has been shown to affect approximately 7–14% of patients (Citation24–26). We observed a higher prevalence of airflow obstruction within our HIV-infected population (26% of those completing spirometry). We have previously reported that HIV-infected individuals receiving antiretroviral therapy (ART) are susceptible to unrecognized obstructive lung disease Citation(8). Implementation of a standardized screening process could enhance recognition of COPD, which carries ramifications beyond COPD management such as risk stratification for COPD-related complications, including COPD exacerbation.
Among the patients for whom spirometry was indicated based on the COPD screening tool, spirometry was ordered for 31%. The reason for deferring spirometry was most commonly left blank (45%); however, among forms noting a reason for deferral, a known diagnosis of COPD was the most common reason. Because our study was designed to determine the feasibility of screening for COPD within a busy HIV clinic, we did not selectively administer the COPD screening tool as this would require additional steps and greater complexity. Testing was also appropriately deferred for patients with a preexisting spirometry order or result (8%). Therefore, when excluding individuals with a known COPD diagnosis or with spirometry previously performed or ordered, 39% of participants meeting indication for spirometry had spirometry ordered by the provider during the visit. The second most common reason for deferring spirometry was the presence of an acute medical issue (9%), which highlights the challenges of providing primary and longitudinal care for an urban HIV population. Causes such as a lack of time (3%), a lack of insurance (2%), or deferred by patient (2%), though much less common, are important factors that may have greater importance in selected populations. The reasons for not ordering spirometry in patients who met indication were often blank or nonspecific. Although our form included several possible reasons for deferral, this represents an area for future study.
Substantial drop-off in screening flow occurred in the administration of the COPD screener tool with 36% of eligible patients not completing the questionnaire. This completion rate is similar to other studies administering questionnaires to HIV patients (Citation27, Citation28). The proportion of completed questionnaires may have been higher if we provided study staff to facilitate explanation and completion of the questionnaire; however, this level of involvement would not have been representative of the real-world screening process we intended to emulate.
This study had several limitations. Since this study was designed to assess the feasibility of implementing the COPD screener tool, we did not assess outcomes such as medication changes following spirometric diagnosis of COPD, a comparison of spirometry findings among participants who did not meet indications for screening spirometry, or the detection of known diagnoses. COPD was defined based upon prebronchodilator spirometry and therefore cannot exclude a diagnosis of asthma. However, our questionnaire was designed to assess for COPD risk factors including age and smoking history and therefore bronchodilator reversibility would suggest asthma–COPD overall syndrome, rather than pure asthma. We lack full sociodemographic and risk behavior characterization of our individual participants to allow determination of factors that may have been associated with nonadherence to the screening process. Generalizability of results from single-center studies is often limited; however, our HIV clinic provides care for approximately 3,000 HIV patients with a mix of socioeconomic and risk behavior groups including 26% men who have sex with men (MSM) and 41% injection drug users (IDU). Our eligibility requirement for inclusion in the study was based only upon attending a nonurgent clinic visit during the study time period. Therefore, our findings are likely to accurately represent our clinic population, and should be generalizable to other urban, academic HIV clinics.
Conclusions
In conclusion, implementation of a simple, COPD screener tool in an urban, HIV clinic provided high yield of COPD diagnoses among persons completing the screening process. However, barriers to large-scale and more effective implementation occurred throughout the screening continuum. Efforts designed to enhance completion of the screening tool by patients and ordering of spirometry by providers should be evaluated to optimize the screening process. This study highlights the importance of screening for COPD among symptomatic, HIV-infected smokers as the first step in improving recognition and management of COPD in this population. Ultimately, early identification of COPD among persons aging with HIV infection may potentially improve clinical outcomes and reduce hospitalization and related costs.
ICOP_A_1161016_SupplementaryFigure1.pdf
Download PDF (140.4 KB)Acknowledgments
We would like to thank the patients and staff of the Johns Hopkins HIV clinic.
Funding
This work was supported by the National Institutes of Health (NIH grants R01-HL-90483, U01-HL-121814, U01-DA-036935, P30 AI-094189 and R34-HL-117349). Dr. Lambert is supported by NIH grants (T32 HL007534 32 and KL2 TR001077 03).
Authors’ contributions
AAL contributed to study design, analysis plan, interpretation, and writing and editing of the report. MBD contributed to study design, analysis plan, interpretation, and writing and editing of the report. AK contributed to study design, interpretation, and editing of the report. JM contributed to study design, interpretation, and editing of the report. JK contributed to study design, interpretation, and editing of the report. RDM contributed to study design, interpretation, and editing of the report. RAW contributed to study design, analysis plan, interpretation, and writing and editing of the report. GDK contributed to study design, analysis plan, interpretation, and writing and editing of the report.
Related Research Data
References
- Deeks SG, Lewin SR, Havlir DV. The end of AIDS: HIV infection as a chronic disease. Lancet. 2013; 382(9903):1525–1533.
- Palella FJ, Jr., Baker RK, Moorman AC, Chmiel JS, Wood KC, Brooks JT, et al. Mortality in the highly active antiretroviral therapy era: changing causes of death and disease in the HIV outpatient study. J Acquir Immune Defic Syndr 2006; 43(1):27–34.
- van Sighem AI, Gras LA, Reiss P, Brinkman K, de Wolf F, study Anoc. Life expectancy of recently diagnosed asymptomatic HIV-infected patients approaches that of uninfected individuals. AIDS 2010; 24(10):1527–1535.
- Deeks SG. HIV infection, inflammation, immunosenescence, and aging. Annu Rev Med 2011; 62:141–155.
- Guaraldi G, Orlando G, Zona S, Menozzi M, Carli F, Garlassi E, et al. Premature age-related comorbidities among HIV-infected persons compared with the general population. Clin Infect Dis 2011; 53(11):1120–1126.
- U.S. Department of Health and Human Services HRaSA. Guide for HIV/AIDS Clinical Care. 2014.
- Crothers K, Butt AA, Gibert CL, Rodriguez-Barradas MC, Crystal S, Justice AC, et al. Increased COPD among HIV-positive compared to HIV-negative veterans. Chest 2006; 130(5):1326–1333.
- Drummond MB, Kirk GD, Astemborski J, McCormack MC, Marshall MM, Mehta SH, et al. Prevalence and risk factors for unrecognized obstructive lung disease among urban drug users. Int J Chron Obstruct Pulmon Dis 2011; 6:89–95.
- From the Global Strategy for the Diagnosis, Management and Prevention of COPD, Global Initiative for Chronic Obstructive Lung Disease (GOLD) 2013 [cited 2013 March 21]. Available from: http://www.goldcopd.org/.
- Qaseem A, Wilt TJ, Weinberger SE, Hanania NA, Criner G, van der Molen T, et al. Diagnosis and management of stable chronic obstructive pulmonary disease: a clinical practice guideline update from the American College of Physicians, American College of Chest Physicians, American Thoracic Society, and European Respiratory Society. Ann Intern Med 2011; 155(3):179–191.
- Yawn BP, Mapel DW, Mannino DM, Martinez FJ, Donohue JF, Hanania NA, et al. Development of the lung function questionnaire (LFQ) to identify airflow obstruction. Int J Chron Obstruct Pulmon Dis 2010; 5:1–10.
- Price DB, Tinkelman DG, Halbert RJ, Nordyke RJ, Isonaka S, Nonikov D, et al. Symptom-based questionnaire for identifying COPD in smokers. Respiration 2006; 73(3):285–295.
- Martinez FJ, Raczek AE, Seifer FD, Conoscenti CS, Curtice TG, D'Eletto T, et al. Development and initial validation of a self-scored COPD population screener questionnaire (COPD-PS). COPD 2008; 5(2):85–95.
- Mullerova H, Wedzicha J, Soriano JB, Vestbo J. Validation of a chronic obstructive pulmonary disease screening questionnaire for population surveys. Respir Med 2004; 98(1):78–83.
- Freeman D, Nordyke RJ, Isonaka S, Nonikov DV, Maroni JM, Price D, et al. Questions for COPD diagnostic screening in a primary care setting. Respir Med 2005; 99(10):1311–1318.
- Pellegrino R, Viegi G, Brusasco V, Crapo RO, Burgos F, Casaburi R, et al. Interpretative strategies for lung function tests. Eur Respir J 2005; 26(5):948–968.
- Aberg JA, Gallant JE, Ghanem KG, Emmanuel P, Zingman BS, Horberg MA, et al. Primary care guidelines for the management of persons infected with HIV: 2013 update by the HIV Medicine Association of the Infectious Diseases Society of America. Clin Infect Dis 2014; 58(1):1–10.
- Centers for Disease C, Prevention, Health R, Services A, National Institutes of H, America HIVMAotIDSo. Incorporating HIV prevention into the medical care of persons living with HIV. Recommendations of CDC, the Health Resources and Services Administration, the National Institutes of Health, and the HIV Medicine Association of the Infectious Diseases Society of America. MMWR Recomm Rep 2003; 52(RR-12):1–24.
- Lambert AA, Kirk GD, Astemborski J, Mehta SH, Wise RA, Drummond MB. HIV infection is associated with increased risk for acute exacerbation of COPD. J Acquir Immune Defic Syndr 2015; 69(1):68–74.
- Shirley DK, Kaner RJ, Glesby MJ. Screening for Chronic Obstructive Pulmonary Disease (COPD) in an Urban HIV Clinic: A Pilot Study. AIDS Patient Care STDS 2015; 29(5):232–239.
- Drummond MB, Huang L, Diaz PT, Kirk GD, Kleerup EC, Morris A, et al. Factors associated with abnormal spirometry among HIV-infected individuals. AIDS 2015; 29(13):1691–1700.
- Feldstein A, Elmer PJ, Smith DH, Herson M, Orwoll E, Chen C, et al. Electronic medical record reminder improves osteoporosis management after a fracture: a randomized, controlled trial. J Am Geriatr Soc 2006; 54(3):450–457.
- Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc 1996; 3(6):399–409.
- Mannino DM, Gagnon RC, Petty TL, Lydick E. Obstructive lung disease and low lung function in adults in the United States: data from the National Health and Nutrition Examination Survey, 1988–1994. Arch Intern Med 2000; 160(11):1683–1689.
- Mannino DM, Ford ES, Redd SC. Obstructive and restrictive lung disease and markers of inflammation: data from the Third National Health and Nutrition Examination. Am J Med 2003; 114(9):758–762.
- Hankinson JL, Bang KM. Acceptability and reproducibility criteria of the American Thoracic Society as observed in a sample of the general population. Am Rev Respir Dis 1991; 143(3):516–521.
- Woodcock A, Bradley C. Validation of the HIV treatment satisfaction questionnaire (HIVTSQ). Qual Life Res 2001; 10(6):517–531.
- Badia X, Podzamczer D, Casado A, Lopez-Lavid C, Garcia M. Evaluating changes in health status in HIV-infected patients: Medical outcomes study-HIV and multidimensional quality of life-HIV quality of life questionnaires. Spanish MOS-HIV and MQOL-HIV Validation Group. AIDS 2000; 14(10):1439–1447.