ABSTRACT
Purpose: This study was conducted to characterize the vaccination practices and challenges of family medicine physicians in Los Angeles County, California.
Methods: The Los Angeles Academy of Family Physicians (LA AFP) sent out electronic surveys to all of their active members (N = 1121) between December 2017 and January 2018, and asked them to answer questions about themselves, their practice, their patient population, and their immunization practices and challenges. We then analyzed the results through basic statistical calculations and Pearson’s chi-squared tests.
Results: Seventy-four people (6.6%) responded to the survey, and 75% of responders stated that they administer all Advisory Committee on Immunization Practices (ACIP) recommended vaccines. The lowest vaccine administration rates were for the high-dose influenza vaccine, which 66.2% (n = 49) of respondents reported to administer, followed by the meningococcal B vaccine (68.9%; n = 51). The respondents who belonged to practices with more than 11 providers, were part of a large hospital or healthcare system, had electronic medical records (EMRs), and used the California Immunization Registry (CAIR) were more likely to report to vaccinate. The number one responding physician-reported challenge to vaccination was limited time and resources to address patient resistance followed by vaccine cost and lack of infrastructure to store vaccines.
Conclusions: In this pilot study, structural and logistical challenges appeared to make the biggest impact on adult vaccination for the responding family medicine physicians. Solutions addressing these challenges will help improve the adult immunization rates.
Introduction
Vaccines have been recognized as one of the most impactful public health measures, preventing 2–3 million deaths worldwide every year.Citation1 The Advisory Committee on Immunization Practices (ACIP) at the Centers for Disease Control and Prevention (CDC) publishes vaccine recommendations in the United States by age group; recommended adult vaccines include the influenza, Tdap/Td, human papillomavirus (HPV), measles, mumps, rubella (MMR) pneumococcal, meningococcal, hepatitis A, hepatitis B, herpes zoster, and Haemophilus influenzae type b vaccines.Citation2 Based on the patient’s medical history and risk factors vaccine recommendations and scheduling may change (see ref 3), which may make it difficult for providers to identify the recommended vaccines for the individual patient. Approximately half of the adult patients in the USA have one chronic condition (see ref 4) that makes them susceptible to the complications of vaccine-preventable infections.Citation5
Despite strong CDC recommendations, adult immunization rates are significantly below the US Department of Health and Human Service’s Healthy People 2020 goals.Citation6 Every year approximately 42,000 adults in the United States die from vaccine-preventable diseases.Citation7
In California, the vaccination rates for adults are far below the federal Healthy People 2020 recommendations.Citation8 For example, in 2015–16, the adult flu vaccination rate in California was only 39% compared to the Healthy People 2020 goal of 70%.Citation8 In 2016 the pneumococcal immunization rate for people 18–64 years of age who are at high risk for pneumonia (i.e. smokers and people with asthma, chronic heart disease and COPD) was 34.1% compared to the recommended goal of 60%.Citation6,Citation9
A recent review by LJ Tan grouped the interventions to improve adult vaccination in three categories: those that improve the access of patients to vaccination, those that improve the patient and community demand for vaccines, and lastly provider- and healthcare system-directed interventions.Citation10 Although systems-wide interventions using immunization information systems, reminder-recall programs, standing orders and implementation of quality measures have been recommended (see ref 10), there is very little data on the role of logistical (such as time and patient flow in the clinic) and structural factors (such as vaccine storage and space) involved in adult immunizations. There is also a lack of data on the impact of interventions targeting those barriers to improve adult immunizations.
Family medicine physicians provide a significant portion of the primary care in the USA (see ref 11) and vaccinate patients with the majority of the adult vaccines.Citation12 In this study we examined the immunization practices and challenges, including those that are structural and logistical, of family medicine physicians in Los Angeles County.
Methods
The Immunization Coalition of Los Angeles County (ICLAC) and the Los Angeles Academy of Family Physicians (LA AFP) collaborated in November 2017 to create a 28-question multiple-choice and free-text survey on immunization practices and challenges in LA County. The study was approved by the Western Institutional Review Board (application number 1923530–43995433). The survey was pilot tested by eight LA AFP board members and revised based on their feedback. LA AFP emailed the electronic link of the final version of the survey to the active, practicing members. Since this study was conducted before recombinant zoster vaccine (ShingrixR) approval, the data presented on herpes zoster vaccination is only for the live attenuated zoster vaccine (ZostavaxR).
Information was collected on provider demographics, practice size (1–10, 11–30, or more than 30 providers), practice type (family medicine versus multispecialty), practice affiliation (academic, hospital/large healthcare, private, PPO/concierge) and patient type (obstetrics and gynecology, pediatric, adult or ≥ 65-years-old). In addition, the respondents were asked whether they use electronic medical records or the California Immunization Registry (CAIR).
The providers were asked to report whether they administer (inject) vaccines to appropriate patients with each ACIP recommended adult vaccine. If the providers answered “no” to any vaccine, they were asked to indicate, using a free text response, why they do not provide that vaccine in their office. They were asked whether and where they refer their patients to for vaccines they do not administer in their office.
The respondents were also asked to rank four challenges to vaccine administration (patient resistance, including lack of educational programs for providers, insufficient time to go over patient concerns during the clinical visit, insufficient resources to use when going over patient concerns during clinical visit, and insufficient adult immunization policies/mandates such as immunization requirements for nursing home or long-term care facility attendants). For both questions, we determined the average ranking of each answer choice; those with lower averages were selected by the respondents as more frequent challenge, and those with higher averages were chosen as a less frequent challenge; in other words, the most frequent challenge was recorded as “1” and the least frequent challenge was scored as “3”.
We used Pearson’s chi-squared test and two-tailed t-test to find the significance of the variable relationships using R statistical software. We computed p-values both from the asymptotic chi-squared distribution of the test statistic and by Monte Carlo simulation with 2000 replicates, due to the relatively small sample size (n = 74). Statistical significance was defined as p-values less than 0.05.
Results
Seventy-four out of 1121 practicing LA AFP providers (6.6%) responded to the survey. Equal numbers of providers described themselves as female and male (n = 28; 37.8% for both; n = 1; 1.4% chose “rather not say”). The largest proportion of the respondents belonged to a practice of more than 30 providers (n = 23; 31.1%), were in a multispecialty practice (n = 24; 32.4%), were affiliated with a hospital or large healthcare system (n = 28; 37.8%) and took care of adults aged 19 years and older (n = 41; 55.4%). Fifty-percent (n = 37) of the respondents reported that they take care of pediatric patients. No respondent took care of “only pediatric” or “only OB/GYN” patients, and only 2 out of 74 respondents (2.7%) took care of “only adult” patients.
Of the respondents, 75.7% (n = 56) reported that they administer all ACIP recommended vaccines, 20.3% (n = 15) reported that they administer only some of the ACIP recommended vaccines, and 4.1% (n = 3) reported that they do not administer vaccines at the clinic. High-dose flu, meningococcal B and herpes zoster vaccines were reported to be administered by 66.2% (n = 49), 68.9% (n = 51) and 71.6% (n = 53) of the respondents, respectively (). For those vaccines, which had the lowest rates of administration, the respondents stated that either their practice does not stock the vaccine, the vaccine was not needed for their patient population or the vaccine cost was too high.
Table 1. Respondents who answered that they administer the vaccine questioned.
Interestingly, 28.4% (n = 21) of the responding providers who did not administer the vaccines stated that they do not refer patients to other facilities for immunizations. Among responding providers who refer patients, a retail pharmacy of the patient’s choice was the preferred location (18.9%; n = 14) followed by another clinic or provider (13.5%; n = 10) and the health department (12.2%; n = 9). Since health departments should only be seeing patients without health insurance, this data indicates that more provider education is needed as it relates to patient vaccine referrals.
Half of the respondents indicated that their practice uses an electronic medical record (EMR) system (51.4%; n = 38). Responding providers who used an EMR system were more likely to administer vaccines compared to those who did not use an EMR system or did not answer the question (p < 0.0005 for all vaccines). Providers who reported that they use the California Immunization Registry (CAIR) were also more likely to respond “yes” to whether they administered any particular vaccine compared to those who did not use CAIR or did not answer the question (p < 0.0005 for all vaccines). Fifty percent (n = 37) of the responding providers reported that their practice uses CAIR to “pull” adult immunization information, 47.3% (n = 35) reported that the practice “pushes” patient immunization information into CAIR, 44.6% (n = 33) reported that their EMR system does not communicate with CAIR, and 27.0% (n = 20) reported that they did not know whether CAIR is used. Sixteen out of 37 (43.2%) responding providers who do not take care of pediatric patients reported to use CAIR.
Federally Qualified Health Centers (FQHC) are community-based health care clinics that provide primary care services to underserved populations.Citation13 They are funded by the Health Resources and Services Administration (HRSA), which is an agency of the U.S. Department of Health and Human Services with a primary goal to improve the health care of people from socio-economically challenging areas. The responding providers who belonged to a Federally Qualified Health Center (FQHC) were more likely to report using CAIR compared to providers from organizations with academic affiliations or organizations that are a part of a large hospital/healthcare system ().
Table 2. Results of chi-square tests of association between responding provider characteristics and CAIR use.
According to the Pearson’s chi-squared test results, the number of providers in a practice was significantly associated with vaccine administration for all vaccines (p < 0.05). Although it differed somewhat based on the vaccine, providers at practices with 11 or more providers were more likely to administer the vaccine compared to responding providers at practices with 1–10 providers or those who did not answer the question. These findings may be explained by the ACIP vaccine recommendations for OB/GYN patients; the HPV vaccine is recommended to prevent cervical cancer and Hepatitis B vaccine is recommended for the newborn, at the time of delivery, to prevent potential maternal transmission of the Hepatitis B virus.Citation14 Other studies have shown higher pneumococcal immunization rates among female patients.Citation15
The responding providers stated that their top vaccine delivery challenges were 1) patient resistance, 2) price of adult vaccines and 3) lack of infrastructure to store and manage adult vaccines (). When asked to rank the biggest barrier to overcoming patient resistance, the most popular answer was insufficient time to go over patient concerns during the clinical visit, followed by insufficient resources (such as brochures and flyers) to use when going over patient concerns during the clinical visit ().
Table 3. Adult immunization challenges of responding Los Angeles family medicine physicians.
Table 4. Most significant barriers of responding Los Angeles family medicine physicians to overcoming patient resistance.
Discussion
Our study found that logistical and structural factors such as lack of time, use of electronic tools like EMR and CAIR, belonging to a larger healthcare organization and the ability to finance the stocking of vaccines were major determinants of adult vaccination for family medicine physicians in Los Angeles.
Healthcare logistics are becoming more important in healthcare management and have been defined as “operational handlings for the delivery of care, including its supportive services, from origination to recipient”.Citation16 Due to the growing and aging population and increasing number of healthcare recommendations and treatments (see ref 17), efficient management of healthcare logistics, which may be patient-centric (for example, patient flow) or structural (for example, availability of EMR or vaccine storage), is essential.
High vaccine cost has been previously reported as a barrier for immunization.Citation18 While respondents in this study indicated that cost was an issue, patient resistance and not having enough time or effective materials to address patient resistance were the top challenges for the responding family medicine physicians. This finding corroborates other studies where primary care physicians have cited time as a barrier to providing patient counseling.Citation19 Despite the availability of strong evidence on the effectiveness of preventive services and published national guidelines, the rates of delivery of preventive health care services are below target levels.Citation20 Primary care providers report that lack of time is the most common barrier to providing preventive services during office visits. Citation21,Citation22 Yarnall, et al reported that 7.4 hours per day would be needed to deliver all of the recommended preventive services to a panel of patients.Citation23 Consequently, primary care providers have to prioritize between many recommended preventive measures.Citation24,Citation25
A lack of time to discuss vaccine safety may have a bigger impact on immunization rates of certain populations who may have higher levels of vaccine hesitancy, and this may contribute to the vaccine disparities. Singleton, et al. found that the influenza and pneumococcal polysaccharide vaccine coverage for blacks and Hispanics was at least 15 percent lower than that for whites, with one of the top reasons for not receiving the influenza vaccine being concern about side effects.Citation26 Strassberg, et al. demonstrated that vaccination acceptance rates were 70.7% for the influenza vaccine and 76.3% for the Tdap vaccine among pregnant women due to concerns over vaccine safety.Citation27
Lack of effective tools to address patient resistance may add to the time demand of the provider. Recently, Dempsey and colleagues demonstrated that 57% of the providers who responded to their survey spent 5–9 minutes with HPV-hesitant parents, and increasing numbers of communication tools used was associated with longer HPV vaccine discussion times.Citation28 The authors suggested that the currently available HPV vaccine-related tools may not be time efficient and “may be adding to the time spent discussing the vaccine.”Citation28 The literature on family physicians’ time spent addressing adult vaccine resistance is limited. Considering that family physicians have many other patient care priorities, lack of time and effective tools to address patient resistance is a very significant barrier to improving adult immunization rates.
Immunization registries are electronic databases and tools that have been used by physicians to report and track receipt of immunizations for pediatric and adult patients. The use of immunization registries has been shown to improve vaccination rates for children.Citation29,Citation30 However, adults are less likely than children to participate in immunization registries; in 2016, 94% of children younger than 6 years of age participated in an immunization information system compared to 44% of adults.Citation31 Nationally, adult immunization registry use has increased in recent years. The percent of reporting on adult vaccination increased from 24% to 44% between 2011 and 2016.Citation32,Citation33 In our study 50% of the responding providers reported to use CAIR for adult vaccines, and both EMR and CAIR use correlated with administration of more vaccines. Interestingly 44.6% of the respondents reported that their EMR system does not communicate with CAIR. A recent study from University of Columbia has shown that it is possible to automatically upload electronic daily immunization data into a citywide immunization registry in New York without additional workload for the staff (see ref 34) and bidirectional data exchange may improve immunization rates.Citation35
Interestingly, 27.0% of the responding providers in our study did not know whether their practice used CAIR. In addition to helping the providers quickly identify the vaccine needs of complex adult patients with comorbidities, vaccine reminder and informational programs may be implemented by CAIR to directly communicate with the patients. The California Department of Public Health plans to add email and phone/text to CAIR’s reminder-recall options so that vaccine safety information can be delivered to the patients directly (personal communication with Steve Nickell, Ph.D. Chief, Registry & Assessment Section, Immunization Branch, Division of Communicable Disease Control, Center for Infectious Diseases, CA Dept. of Public Health).
Our study found that reporting providers from larger practices (with 11–30 providers or more than 30 providers) and those affiliated with a hospital or large healthcare system were more likely to administer most vaccines. This may be because larger healthcare systems have more support personnel, such as community health workers/promotoras to communicate with the patients and help the patient flow and finances to purchase and stock the vaccines.Citation24 Small practices may experience additional logistical barriers to vaccine administration including maintaining vaccine temperature and affording rent for space to store vaccines.Citation24 A study by Campos-Outcalt and colleagues has shown that solo or small family medicine practices were less likely to administer recommended vaccines.Citation36
The National Committee for Quality Assurance (NCQA) has recently updated the adult immunization reporting requirements of the Healthcare Effectiveness Data and Information Set (HEDIS), to include Adult Immunization Status (AIS) and Prenatal Immunization Status (PRS).Citation37 The new HEDIS quality measures include reporting of the flu and Tdap vaccines for all adults, recombinant herpes zoster vaccine for those 50 years and older, pneumonia vaccine for those 65 years and older and Tdap and flu vaccines for pregnant women.Citation38 The new measures have the potential to give health plans and provider groups the ability to identify gaps in care, track outcomes and provide incentives for quality improvement. However, successful implementation of the adult HEDIS immunization measures requires resolution of the challenges experienced by the providers.
The limitations of our study are small sample size, low response rate, limited geographic reach (LA County) and lack of zip-code data which could help illustrate where in LA County these resource disparities exist. Despite these limitations, many findings related to the character of the clinic were significant, indicating that the barriers to vaccination are structural. Addressing logistical issues such as use of immunization registries and patient flow in the clinic may allow more time for physicians to communicate with their patients and increase the ease of vaccine recommendations for providers.
Abbreviations
ACIP | = | Advisory Committee on Immunization Practices |
CDC | = | Centers for Disease Control and Prevention |
ICLAC | = | Immunization Coalition of Los Angeles County |
LA AFP | = | Los Angeles Academy of Family Physicians |
EMR | = | Electronic Medical Record |
FQHC | = | Federally Qualified Health Center |
MCV4 | = | Meningococcal Vaccine |
HPV | = | Human Papillomavirus |
CAIR | = | California Immunization Registry |
PPSV23 | = | Pneumococcal Polysaccharide Vaccine |
Disclosure of potential conflicts of interest
The authors report no conflicts of interest.
Acknowledgments
The authors thank Matthew Emons, MD, ICLAC Steering Committee member, for his input on the 2019 HEDIS immunization measures.
Additional information
Funding
References
- World Health Organization. 10 facts on immunization. Geneva (Switzerland): United Nations; 2018 Mar [ accessed 2018 May 22]. http://www.who.int/features/factfiles/immunization/en/
- Kim DK, Riley LE, Hunter P, on behalf of the Advisory Committee on Immunization Practices. Recommended immunization schedule for adults Aged 19 Years or Older, United States, 2018. Ann Intern Med. 2018;168:210. doi:10.7326/M17-3439.
- Centers for Disease Control and Prevention. Recommended Immunization Schedule for Adults Aged 19 Years or Older, United States, 2018. Atlanta (GA): United States Department of Health and Human Services; 2018 Mar 5[ accessed 2018 May 22]. https://www.cdc.gov/vaccines/schedules/hcp/imz/adult.html#f5
- Centers for Disease Control and Prevention. Promoting Back to School: What’s in Their Backpacks? Atlanta (GA): United States Department of Health and Human Services; 2018 Feb 6 [ accessed 2018 May 22]. https://www.cdc.gov/chronicdisease/index.htm
- Centers for Disease Control and Prevention. Educating Adults with Chronic Health Conditions: Vaccination Resources. Atlanta (GA): United States Department of Health and Human Services; 2018 Jul 27 [ accessed 2018 May 22]. https://www.cdc.gov/vaccines/hcp/adults/for-patients/health-conditions.html
- Healthy People 2020. Immunization and Infectious Diseases - Objectives. Washington (DC): United States Department of Health and Human Services; 2018 Sept 9 [ accessed 2018 May 22]. https://www.healthypeople.gov/2020/topics-objectives/topic/immunization-and-infectious-diseases/objectives
- Healthy People 2020. Immunization and Infectious Diseases - Overview. Washington (DC): United States Department of Health and Human Services; 2018 May 22 [ accessed 2018 May 22]. https://www.healthypeople.gov/2020/topics-objectives/topic/immunization-and-infectious-diseases
- Centers for Disease Control and Prevention. 2015-16 Influenza Season Vaccination Coverage Dashboard. Atlanta (GA): United States Department of Health and Human Services; 2016 Sept 29 [ accessed 2018 Feb 9]. https://www.cdc.gov/flu/fluvaxview/reportshtml/reporti1516/reportii/index.html.
- Centers for Disease Control and Prevention. 2016 Adult Vaccination Coverage General Population Dashboard. Atlanta (GA): United States Department of Health and Human Services; 2018 Feb 8 [ accessed 2018 Mar 22]. https://www.cdc.gov/vaccines/imz-managers/coverage/adultvaxview/data-reports/general-population/dashboard/2016.html
- Tan LLJ. A review of the key factors to improve adult immunization coverage rates: what can the clinician do? Vaccine. 2018;36: 5373–5378. doi:10.1016/j.vaccine.2017.07.050.
- Association of American Medical Colleges. Number of People per Active Physician by Specialty, 2015. Washington (DC): AAMC; 2015 [ accessed 2018 Sept 25]. https://www.aamc.org/data/workforce/reports/458490/1-2-chart.html
- Lutz CS, Kim DK, Black CL, Ball SW, Devlin RG, Srivastav A, Fiebelkorn AP, Bridges CB. Clinicians’ and Pharmacists’ Reported implementation of vaccination practices for adults. Am J Prev Med. 2018;55: 308–318. doi:10.1016/j.amepre.2018.05.011.
- Health Resources & Services Administration. Federally Qualified Health Centers: Eligibility. Rockville(MD): U.S. Department of Human and Health Services; 2018 Jan [ accessed 2018 Sept 25]. https://www.hrsa.gov/opa/eligibility-and-registration/health-centers/fqhc/index.html
- O’Leary ST, Campbell JD, Kimberlin DW. Update From the Advisory Committee on Immunization Practices. J Pediatric Infect Dis Soc. 2018; doi: 10.1093/jpids/piy050.
- Fry CA, Silverman EP, Miller S. Addressing pneumococcal vaccine uptake disparities among African-American Adults in the United States. Public Health Nurs. 2016;33: 277–282. doi:10.1111/phn.12257.
- Zhang C, Grandits T, Härenstam KP, Hauge JB, Meijer S. A systematic literature review of simulation models for non-technical skill training in healthcare logistics. Adv Simul. 2018;3: 1–16. doi:10.1186/s41077-018-0072-7.
- Savage P. Economic and logistic challenges of modern oncology healthcare. Clin Oncol (R Coll Radiol). 2017;29: 547–549. doi:10.1016/j.clon.2017.03.009.
- Tan LJ. Adult vaccination: now Is the time to realize an unfulfilled potential. Hum Vacc Immunother. 2015;11: 2158–2166. doi:10.4161/21645515.2014.982998.
- Kolasa KM, Rickett K. Barriers to providing nutrition counseling cited by physicians: a Survey of primary care practitioners. Nutr Clin Pract. 2010;25: 502–509. doi:10.1177/0884533610380057.
- McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348: 2635–2645. doi:10.1065/NEJMsa022615.
- Korownyk C, McCormack J, Kolber MR, Garrison S, Allan GM. Competing demands and opportunities in primary care. Can Fam Physician. 63;2017:664–668.
- Østbye T, Yarnall KSH, Krause KM, Pollak KI, Gradison M, Michener L. Is there time for management of patients with chronic diseases in primary care? Ann Fam Med. 2005;3: 209–214. doi:10.1370/afm.310.
- Yarnall KS, Pollak KI, Østbye T, Krause KM, Michener JL. Primary care: is there enough time for prevention? Am J Public Health. 93;2003:635–641.
- O’Connor PJ, Sperl-Hillen JM, Margolis KL, Kottke TE. Strategies to prioritize clinical options in primary care. Ann Fam Med. 2017;15: 10–13. doi:10.1370/afm.2027.
- Maciosek MV, LaFrance AB, Dehmer SP, McGree DA, Flottemesch TJ, Xu Z, Solberg LI. Updated priorities among effective clinical preventive services. Ann Fam Med. 2017;15: 14–22. doi:10.1370/afm.2017.
- Singleton J, Santibanez T, Wortley P. Influenza and pneumococcal vaccination of adults Aged ≥65: racial/ethnic differences. Am J Prev Med. 2005;29: 412–420. doi:10.1016/j.amepre.2005.08.012.
- Strassberg ER, Power M, Schulkin J, Stark LM, Mackeen AD, Murtough KL, Paglia MJ. Patient attitudes toward influenza and tetanus, diphtheria and acellular pertussis vaccination in pregnancy. Vaccine. 2018;36: 4548–4554. doi:10.1016/j.vaccine.2018.05.121.
- Dempsey AF, Lockhart S, Campagna EJ, Pyrzanowski J, Barnard J, O’Leary ST. Providers’ time spent and tools used when discussing the HPV vaccine with parents of adolescents. Vaccine. 2016;34: 6217–6222. doi:10.1016/j.vaccine.2016.10.083.
- Wilcox SA, Koepke CP, Levenson R, Thalheimer JC. Registry-driven, community-based immunization outreach: a randomized controlled trial. Am J Public Health. 2001;91: 1507–1511. doi:10.2105/AJPH.91.9.1507.
- Kempe A, Steiner JF, Renfrew BL, Lowery E, Haas K, Berman S. How much does a regional immunization registry increase documented immunization rates at primary care sites in rural Colorado? Ambul Pediatr. 2001;1: 213–216. doi:10.1367/1539-4409(2001)001<0213:HMDARI>2.0.CO;2.
- Groom H, Hopkins D, Pabst L, Morgan J, Patel M, Calonge N, Coyle R, Dombkowski K, Groom A, Kurilo M. Immunization information systems to increase vaccination rates: A community guide systematic review. J Public Health Manag Pract. 2015;21:227–248. doi:10.1097/PHH.0000000000000069.
- Centers for Disease Control and Prevention. IISAR Data Participation Rates. Atlanta (GA): United States Department of Health and Human Services; 2018 May 16 [ accessed 2018 May 22]. https://www.cdc.gov/vaccines/programs/iis/annual-report-iisar/2011-data.html
- Centers for Disease Control and Prevention. 2016 IISAR Data Participation Rates. Atlanta (GA): United States Department of Health and Human Services; 2018 May 16 [ accessed 2018 May 22]. https://www.cdc.gov/vaccines/programs/iis/annual-report-iisar/2016-data.html
- Kolff CA, Scott VP, Stockwell MS. The use of technology to promote vaccination: A social ecological model based framework. Hum Vaccin Immunother. 2018;14: 1636–1646. doi:10.1080/21645515.2018.1477458.
- Stockwell MS, Natarajan K, Ramakrishnan R, Holleran S, Forney K, Aponte A, Vawdrey DK. Immunization data exchange with electronic health records. Pediatrics. 2016;137(6):e20154335. doi:10.1542/peds.2015-4335.
- Campos-Outcalt D, Jeffcott-Pera M, Carter-Smith P, Schoof BK, Young HF. Vaccines provided by family physicians. Ann Fam Med. 2010;8: 507–510. doi:10.1370/afm.1185.
- National Committee for Quality Assurance. The Future of HEDIS. Washington(DC): NCQA [ accessed 2018 Jul 6]. https://www.ncqa.org/hedis/the-future-of-hedis/
- National Committee for Quality Assurance. HEDIS 2019: Summary Table of Measures, Product Lines and Changes. Washington (DC): NCQA [ accessed 2018 Sept 25]. https://www.ncqa.org/wp-content/uploads/2018/08/20190000_HEDIS_Measures_SummaryofChanges.pdf