Interprofessional transformation of clinical education: The first six years of the Veterans Affairs Centers of Excellence in Primary Care Education

1

ABSTRACT

This paper describes the Centers of Excellence in Primary Care Education (CoEPCE), a seven-site collaborative project funded by the Office of Academic Affiliations (OAA) within the Veterans Health Administration of the United States Department of Veterans Affairs (VA). The CoEPCE was established to fulfill OAA’s vision of large-scale transformation of the clinical learning environment within VA primary care settings. This was accomplished by funding new Centers within VA facilities to develop models of interprofessional education (IPE) to teach health professions trainees to deliver high quality interprofessional team-based primary care to Veterans. Using reports and data collected and maintained by the National Coordinating Center over the first six years of the project, we describe program inputs, the multicomponent intervention, activities undertaken to develop the intervention, and short-term outcomes. The findings have implications for lessons learned that can be considered by others seeking large-scale transformation of education within the clinical workplace and the development of interprofessional clinical learning environments. Within the VA, the CoEPCE has laid the foundation for IPE and collaborative practice, but much work remains to disseminate this work throughout the national VA system.

KEYWORDS:

• Clinical Learning Environment
• Faculty
• Health Professions
• Interprofessional Education
• Team-based
• Trainees
• Veterans

Introduction

Despite efforts to transform education and clinical care from occurring in professional silos to interprofessional collaborative learning and practice, such transformations remain complex and challenging (Brienza, 2016; Brienza, Zapatka, & Meyer, 2014; Cox, 2013; Institute of Medicine, 1972). Most reports describe new models of education and clinical care at single programs (Long, Dann, Wolff, & Brienza, 2014; Mazanec et al., 2015; Murray, Christen, Marsh, & Bayer, 2012; Nasir, Goldie, Little, Banerjee, & Reeves, 2017; Pare, Maziade, Pelletier, Houle, & Iloko-Fundi, 2012; Ruddy, Borresen, & Myerholtz, 2013). However, implementation of interprofessional education (IPE) programs across multiple clinical care sites provides the opportunity to describe a process used for large-scale system transformation. This paper describes the first six years of a coordinated initiative within the Department of Veterans Affairs (VA) to implement the vision of the Office of Academic Affiliations (OAA) to transform education and clinical care within VA primary care settings to interprofessional team-based learning and practice to improve health professions education and primary care outcomes for Veterans. Lessons learned from our experiences may guide other large-scale transformational efforts to improve the educational preparation of our national healthcare workforce.

Background

The VA’s nationwide implementation of Patient Aligned Care Teams (PACT) in 2010 introduced fundamental changes in VA primary care based on the patient-centered medical home model (Nelson et al., 2014; Rosland et al., 2013). Soon afterwards, OAA recognized the need to redesign primary care education to maintain alignment with primary care delivery (Bowen & Schectman, 2013; Cox, 2013; Gilman, Chokshi, Bowen, Rugen, & Cox, 2014). In 2010, OAA released the first of three national requests for proposals (RFP) for VA facilities to seek funding to develop and implement interprofessional team-based curricula to achieve clinical practice-education integration (Department of Veterans Affairs, 2010). The RFP had several requirements to ensure consistency. Requirements included: partnerships with academic affiliates; incorporation of physician residents and nurse practitioner (NP) students; the inclusion of other professions as resources and expertise became available; curriculum development in four core educational domains (shared decision-making, interprofessional collaboration, sustained relationships, and performance improvement); and workplace learning as an instructional strategy. Staffing requirements included leadership teams consisting of a physician and nurse practitioner co-director and Center faculty including at least four clinician educators with protected time to fulfill curriculum development, teaching, and mentoring responsibilities. Centers were free to create strategies to fit within their local VA context. Five VA facilities (Boise, Idaho; Cleveland, Ohio; San Francisco, California; Seattle, Washington; and West Haven, Connecticut) were awarded $1 million per Center per year in January 2011 (Stage 1, 2010–2015) for training activities beginning in July 2011.

To guide multisite activities, OAA established a National Coordinating Center (NCC) consisting of individuals with expertise in education, program administration, and evaluation across the health professions. The role of the NCC was to develop IPE national policy, distribute and monitor resources, guide and facilitate Center work, identify promising educational practices to incorporate into common curricula across Centers, conduct cross-site evaluation, develop performance improvement and population registries, and communicate results to OAA, other VA staff and leadership, and the broader academic and health professions communities.

In 2015, OAA launched two additional RFPs for Stage 2, 2015–2019 (Department of Veterans Affairs, 2015a, 2015b). The five original Centers, plus two new Centers at VA facilities in Los Angeles, California, and Houston, Texas were funded for another four years at $750,000 per site per year.

Methods

Enterprise evaluation across Centers was implemented early in Stage 1. Using a mixed-methods approach, NCC evaluators conducted both formative and summative evaluation to inform ongoing intervention development and implementation. The enterprise evaluation was guided by a logic model and consisted of individual projects each guided by its own evaluation question. In Stage 2, evaluation capacity was enhanced by the engagement of an external health services research group at the Portland VA to further evaluate impacts on patients, including clinical outcomes, clinical staff, and the VA healthcare system.

This evaluation is categorized as an operation's improvement activity based on VHA Handbook 1058.05, where information generated is used for business operations and quality improvement. The overall project was subject to administrative oversight rather than oversight from a Human Subjects Institutional Review Board.

CoEPCE logic model

The logic model in Figure 1 illustrates the inputs for CoEPCE development, the multicomponent CoEPCE “intervention”, and activities undertaken to develop the intervention leading to short, intermediate, and long-term outcomes (Petersen, Taylor, & Peikes, 2013). included funding from OAA, local VA facilities and affiliates, NCC expertise and leadership, VA Office of Primary Care that directs clinical care received by Veterans, and VA informatics including the electronic medical record system. The multicomponent intervention was instituted at each of the seven VA facilities and included the Center structure and leadership, the IPE curriculum, faculty development, Center evaluation, and dissemination of best practices. Over the first six years, activities focused on the establishment of individual Center infrastructure, engagement with academic affiliates to develop new logistics required for IPE, planning and implementation of faculty development programs, development and implementation of IPE curriculum, and development and implementation of individual Center evaluation activities. These activities were undertaken by the Centers with NCC input to develop interventions that fit within the local facility context but were standardized across Centers.

Figure 1. CoEPCE logic model.

Data sources

We conducted descriptive analyses of evaluation reports and administrative data collected and maintained by the NCC. Data sources included the following:

• Coordinating Center Annual Report 2011–2012: This document describes clinical transformation, operational and evaluation activities during the first two years of Stage 1.

• Semi-annual evaluation reports: Reports were submitted by all Centers to assess Center infrastructure, curriculum, faculty, and trainees. Semi-annual reports from 2013–2014 and 2015–2016 were used in this analysis.

• Trainee data: These data were collected from Centers twice a year, and were used to describe numbers and types of trainees over the six-year period.

• Nurse Practitioner Resident Competency Tool: This tool was administered at each Center to evaluate competency of NP residents at one, six, and twelve months in domains of clinical, leadership, interprofessional team collaboration, patient-centered care, shared decision-making, sustained relationships, and performance improvement (Rugen, Speroff, Zapatka, & Brienza, 2016). NP residents rated items within each domain on a scale ranging from 0 (not performed/not observed) to 5 (able to supervise others). There was also a qualitative component where NP residents answered open-ended questions about things they do well, things they would like to improve, and short/long-term goals. Psychometric analysis of the quantitative component demonstrates high internal consistency reliability for each of the seven domains with Cronbach’s alpha ranging from 0.86–0.95 (Rugen, Dolansky, Dulay, King, & Harada, 2017).

• Participant survey: This survey assessed trainee satisfaction with the overall program, the curriculum including the four core domains, and interest in VA and/or primary care employment. The survey is administered during the spring of each year and the findings are communicated back to Centers for program quality improvement purposes. Internal consistency reliability is high with Cronbach’s alpha ranging from .83 to .93 for subscales measuring learning in each of the four core domains, program satisfaction, program practices, and systems impact (unpublished data).

Data analysis

Content analysis of textual data. The NCC Annual Report and semi-annual reports were analyzed using qualitative content analysis. Reports were uploaded into Atlas.ti software version 7 (Atlas.ti, 2017) and portions of text coded based on the logic model and time point (baseline [2011–2012], intermediate [2013–2014], or current [2016]). Queries were run in Atlas.ti to group text by code and time point. Two authors (NH and LT) reviewed query reports and abstracted key data into tables for each time point. A third co-author (KWR) reviewed the tables to confirm accuracy.

Quantitative analysis

The quantitative data from the semi-annual reports and trainee data were analyzed using Excel 2010 to generate frequencies and graphs. Statistical analyses of data from the NP Resident Competency Tool are described elsewhere (Rugen et al., 2017). Participant Survey data were analyzed using IBM SPSS version 23 (IBM Corp, 2015).

Results

Development of the CoEPCE multicomponent intervention

Center Infrastructure and Leadership

Over the six-year period, Center infrastructure was established within seven VA facilities. Initial Center leadership requirements were for a director and co-director, one physician and one NP representing the interprofessional trainee focus. The NCC realized that this implied a hierarchy, and modified the requirement to a co-director leadership model where the physician and NP were to be equal partners. However, this proved to be a difficult leadership model to sustain as there was high turnover in the NP co-director position at four of the Centers.

In Stage 2, Centers were given the freedom to change their leadership structure to a director only model, a director/co-director model, or maintain the Stage 1 co-director model. Three Centers proposed a physician director model, two proposed a director/co-director model, and two proposed co-director models.

Based on Stage 1 experiences demonstrating the importance of interprofessional leadership for clinical and educational endeavors, Stage 2 introduced new requirements for the inclusion of associate director positions for newly mandated professions of RN care managers, pharmacists, psychologists, and evaluators. Two Centers also opted to include a social work associate director.

Center space

All Centers identified space limitations as a barrier to integration of clinical care and educational activities. All required additional space for incorporation of trainees from multiple professions to participate in integrated activities such as didactic sessions, team huddles and group visits. Workplace learning was enhanced when interprofessional trainees were co-located “backstage” in a teamwork room. To address these needs, the NCC provided additional funding during the first year for the Centers to restructure their space and engaged the VA Office of Primary Care leadership to attend site visits for space assessments and development of new VA policy for architectural standards in academic primary care.

VA and academic affiliate partnerships

Over the first six years, the Centers worked to increase the number of academic affiliate partnerships. In 2011, the five Centers reported a total of 11 affiliate partnerships with schools of nursing and schools of medicine. By 2016, there were a total of 33 affiliate partnerships across seven Centers. New affiliations were established with schools of pharmacy, schools of social work, programs in management, and health services research.

One of the biggest challenges was scheduling for collaborative educational activities with different trainee professions while providing continuity between trainees and patients, trainees and PACT teams, and trainees and faculty supervisors. Consequently, the Centers had to work closely with their academic affiliates to facilitate logistics required for IPE. Academic affiliates had to adopt new ways of scheduling to accommodate learners of different professions, such as adjustment of block immersion models for physician residents so that they were present at the same time as NP students who were in clinic 1 to 2 days per week for 6–15 weeks, and NP residents who were present full time for 1 year.

Interprofessional faculty

In Stage 1, interprofessional clinical faculty consisted mainly of physicians and NPs with fewer pharmacists and psychologists. Because of the longstanding emphasis on physician training in the VA, most physician faculty had academic appointments and protected time for education. NP faculty, however, were generally clinicians with little experience in academic roles and VA facilities did not have a culture of providing NPs with protected time from their clinical role for education, leadership, faculty development, or scholarly activities. Across Centers, all physician co-directors and most physician faculty had academic appointments at their affiliated school of medicine, whereas only two NP co-directors had academic faculty appointments at their affiliated school of nursing.

By 2016, the interprofessional faculty and staff at each Center included NPs, RN Care Managers, pharmacists, physicians (internal medicine, psychiatry), psychologists, social workers, and evaluators/researchers. Four Centers reported working with their affiliates to initiate new academic appointments for their RN, NP, pharmacy, social work, and/or psychology faculty. There was variability in the rate of academic appointments by profession that followed training priorities historically set by VA, ranging from 84% for physician faculty to none for the social work faculty (Table 1).

Table 1. CoEPCE faculty academic appointments, 2016.

Faculty Profession	Number of Faculty	Has Academic Appointment n (%)
Nurse Practitioner	30	23	(77%)
Nursing^a	35	5	(14%)
Pharmacy	17	11	(65%)
Physician	82	69	(84%)
Psychology	14	7	(50%)
Social Work	5	0	(0%)
Other^b	29	10	(35%)
Total	212	125	(59%)

^aNursing includes licensed vocational nurses (LVN) and registered nurses (RN).

^bOther includes evaluators of different academic disciplines and program administrators.

Interprofessional teaching activities

In Stage 1, the Centers began developing curricula by adapting existing didactic activities or courses to address CoEPCE curricular goals. For example, most of the Centers implemented Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS®), a course to teach communication and teamwork skills (Clancy & Tornberg, 2007). The Centers focused on the development of teaching activities within a single domain, such as team conferences to develop interprofessional collaboration skills and lectures on motivational interviewing to teach shared decision-making skills.

To achieve the aim of aligning IPE with clinical care, the Centers were challenged with developing new workplace learning strategies in which teaching occurs within the context of clinical practice, linking the didactic to clinical instruction within and across professions. Each Center approached the development of workplace learning strategies to fit within their local VA context. For example, one Center developed a standardized huddle checklist because their trainees were spread throughout several PACT teams and they needed more uniformity of experience (Shunk, Dulay, Chou, Janson, & O’Brien, 2014). Another Center developed a panel management curriculum because proactive chronic disease management was an institutional priority and they had the availability of an ideal training space with technology support (Kaminetzky & Nelson, 2015). This led to each Center taking ownership of a promising practice (huddles, panel management, PACT interdisciplinary care update, polypharmacy clinic, or dyads) that could be disseminated throughout the VA system (Centers of Excellence in Primary Care Education, 2017).

In 2015, the NCC advanced a new framework based on a three-element model for non-health care settings to guide further development of interprofessional curriculum (Fink, 2003). The framework considers curriculum to include the development of learning objectives, instructional strategies (i.e., workplace learning, didactics, and reflective practice) and trainee assessment of accomplishment of learning objectives. This curriculum framework was applied across the four core domains of shared decision-making, interprofessional collaboration, performance improvement, and sustained relationships, and was also used to integrate mental health into the primary care curriculum.

By 2016, the curricula evolved so that many teaching activities integrated learning objectives across the four core domains, mental health, and instructional strategies within a single teaching activity. Examples of integrated teaching activities are listed in Table 2.

Table 2. Examples of center reported teaching activities integrating instructional strategies and core domains.

	Instructional Strategy^a			PCMHI^b	Core Domain^c
Teaching Activity	Didactic	WPL	RP	PCMHI	IPC	SDM	SR	PI
Panel Management: Quarterly diabetic panel management sessions with NP, MD, Pharmacy, and Psychology trainees. Sessions are 3–4 hours and NP/MD co-facilitated. First hour is didactic on how to use the Diabetic Registry tool, followed by use of “toolbox” to plan interprofessional interventions for a population.	Yes	Yes	No	Yes	Yes	Yes	No	Yes
Ambulatory Noon Conference: Trainees of different professions co-present an ambulatory noon conference based on a topic list developed by the CoE. Faculty facilitate trainees working together to co-present. Faculty co-evaluate each trainee pair’s presentation using formal evaluation forms.	Yes	No	Yes	Yes	Yes	No	No	Yes
Integration of Workplace TeamStepps: Time is provided for reflection on experiences when TeamStepps tools are used and the effectiveness of the implementation.	No	Yes	Yes	No	Yes	No	No	No
Enhanced Mentoring: Sessions provide opportunities for trainees and faculty to identify common interests to match trainees with mentors. Trainees and mentors check in at least twice per year.	No	No	Yes	No	Yes	No	Yes	No
Mental Health Collaborative Care Conference: Monthly collaborative care conference to develop patient-centered integrated care plans in primary care. Participants include PACT team members (RN care manager, social work, pharmacy), primary care and mental health faculty, and trainees.	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Health Psychology (HP) Coaching: HP trainees provide coaching and observe primary care trainees and faculty coach patients to provide formative feedback in real time on issues such as efficiency, agenda setting, patient-centeredness, shared decision making, motivational interviewing, risk communication, and boundary setting.	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Super Huddle: Three PACT teams meet in a super primary care huddle for 10 minutes, followed by individual team meetings for any mental health, pharmacy, social work or primary care integration needs. Super Huddle Checklist used to identify if core goals of huddle are being met.	No	Yes	No	Yes	Yes	No	No	No
Polypharmacy Clinic: Trainee-led performance improvement process to reduce polypharmacy in older Veterans and enhances trainee skills and knowledge related to safe prescribing.	Yes	Yes	No	No	Yes	Yes	Yes	No

^aInstructional strategies include didactics, workplace learning (WPL), and reflective practice (RP)

^bPrimary care, mental health integration (PCMHI)

^cCoEPCE core domains include interprofessional collaboration (IPC), shared decision making (SDM), sustained relationships (SR), and performance improvement (PI)

Center (single-site) evaluation

In Stage 1, each Center established their own evaluation program that initially focused primarily on trainees and assessment of teamwork components, such as team structure, communication, attitudes, team development, and team performance. Centers used established instruments such as the Team Development Measure (Salem-Schatz, Ordin, & Mittman, 2010) and the TeamSTEPPS® Teamwork Attitudes Questionnaire (T-TAQ) (Baker, Krokos, & Amodeo, 2008). Centers also initiated evaluation in the performance improvement domain, using instruments such as the Quality Improvement Knowledge Application Tool (QIKAT) (Singh et al., 2014) and panel management registry tools from the VA electronic record (Kaminetzky & Nelson, 2015). Trainee competency assessment within the Centers focused on NP residents using a tool developed for cross-site use, the NP Resident Competency Tool (Rugen et al., 2017), while physician residents were assessed through traditional professional accreditation methods. By Stage 2, The Centers were evaluating faculty performance on teaching activities, precepting, and mentoring. Some were evaluating patient satisfaction, and one site began to evaluate impacts of specific teaching activities on health system outcomes, dissemination, and adoption by other sites (King et al., 2017).

Short-term outcomes of enterprise evaluation

Interprofessional trainees: numbers and types

The first cohort of trainees began in academic year 2011–2012 with two mandated trainee professions, physician residents, and NP students. The Centers were encouraged to include other trainee professions such as pharmacists, psychologists, and social workers. Physician residents, primarily from internal medicine residency programs, and NP students were assigned to the CoEPCE by their affiliate training program. These trainees had to spend at least 30% of their total training time in the CoEPCE, a requirement set by the NCC to ensure an adequate “dose” of interprofessional training based on the review of physician residency program requirements.

Early in Stage 1, NP residency programs in primary care were established at each Center as core CoEPCE training programs (Rugen et al., 2014). NP residents were graduates of masters or doctor of nursing practice NP programs who participated in year-long specialized training in primary care delivery, leadership, and scholarly activities (Rugen, Gilman, & Traylor, 2015; Rugen et al., 2016, 2014).

The types and numbers of trainees by profession are displayed in Figure 2. Prior to the CoEPCE, there were no trainees receiving IPE in primary care. Beginning in 2011–2012 there were 58 trainees receiving IPE, increasing to 179 trainees in 2015–2016. Physician residents comprised two-thirds of the total CoEPCE trainees because of the historical emphasis on physician training in the VA. In addition, space limitations and preceptor availability within professions restricted the growth of other trainee professions such as NP residents and social work interns.

Figure 2. Number of core trainee positions by profession academic years 2011–2012 to 2015–2016.

Trainee competencies

n Stage 1, cross-site evaluation of professional competencies focused on NP residents using the NP Resident Competency Tool. Results from three cohorts of NP residents show significant improvement in each competency domain (p < .0001) over the year-long program as measured by both trainees and their mentors (Rugen et al., 2017). Furthermore, NP residents were rated by their mentors as able to practice without supervision in all competency domains by the end of the program (Rugen et al., 2017). The aggregated results have been used as feedback for program improvement. For example, based on quantitative findings that the clinical competency domain was among the lowest scored and qualitative findings that NP residents wanted to improve differential diagnosis skills, the NCC developed a course on this topic for NP faculty (Rugen et al., 2017).

Trainee perceptions of the CoEPCE program

Results from the Participant Survey administered to three consecutive trainee cohorts covering 2014 and earlier, 2014–2015, and 2015–2016 show an upward trend in the extent to which the CoEPCE had met its overall mission of fostering transformation of clinical education by preparing graduates of health professional programs to work in, lead and improve Veteran/patient-centered interprofessional teams that provide coordinated, longitudinal care with mean ratings of 3.7, 3.8, and 4.0, respectively, on a scale of 1–5 where 5 is best. There is also an upward trend in ratings of the overall CoEPCE learning experience with mean ratings of 3.9, 4.0, and 4.2, respectively. Similar upward trends are seen for interest in future primary care or VA employment. The 2015–2016 survey also showed differences in learning specific skills between professions, e.g., psychologists reported lower rates of learning than other professions in performance improvement skills such as using information technology to manage patient panels. More detailed longitudinal analyses are currently ongoing to explore trends in ratings on global indicators, program satisfaction, and curriculum over time.

Impacts on primary care clinic staff and patients

Cross-site evaluation of clinic staff and patients was initiated at the beginning of Stage 2. Impacts of primary care clinic staff were evaluated through semi-structured qualitative interviews with clinic staff (n = 35) working in CoEPCE teams. The overall finding was that working with trainees contributed positively to work experience, and the inclusion of clinic staff in workplace learning activities such as huddles contributed towards effective interpersonal working relationships with trainees (Newell et al., 2017). Evaluation of patient impacts come from a cohort study of primary care patients comparing patients assigned to CoEPCE teams compared to a control group of patients assigned to non-CoEPCE teams (Edwards et al., 2017). This study found that the likelihood of poor diabetic A1c control declined by 1.9% per patient per year, the likelihood of timely mental health referral increased among CoEPCE patients by 2.1%, and the likelihood of PCMHI visits increased by 1.9% per patient per year (Edwards et al., 2017).

Discussion

The introduction of PACT within VA facilities provided the opportunity for OAA to begin the transformation of the clinical education workplace to develop a collaborative-ready health workforce for the future. Our work over the first six years of CoEPCE implementation can be summarized as laying a foundation for the interprofessional clinical learning environment that is described by Simpson and colleagues (2017) as comprised of people, facilities, and processes. The “people” are the interprofessional clinical leaders and faculty, “facilities” are the clinical sites and provider readiness for IPE that are patient-centered and team-oriented, and “processes” are based on workplace-based IPE, performance improvement, and trainee placement. Our work has implications for lessons learned that may be helpful for others seeking large-scale system transformation (Table 3).

Table 3. Lessons learned for development of the interprofessional clinical learning environment within a national healthcare system.

Logic Model Component	Lessons Learned
Inputs
Funding and Resources	In VA, funding is necessary to ensure specific protected time of academic clinical staff to serve as faculty, and for development of Center infrastructure and leadership, interprofessional curriculum, interprofessional faculty, and evaluation activities. Funding was provided by OAA, a national VA program office, that embraced the vision of IPE in primary care settings to establish alignment with a new clinical care model known as PACT. Funding alone is not sufficient to achieve systems transformation because there was a myriad of challenges that required additional inputs, such as changing institutional and professional cultures and the complexity of aligning interests and logistics across all stakeholders.
National Coordinating Center (NCC)	The NCC was established within OAA to monitor compliance with VA regulations while also facilitating collaboration and fidelity to project goals. The two roles, assuring compliance and facilitating creative collaboration, could easily be in conflict requiring the NCC to develop deliberate strategies to distinguish between requirements and stretch goals. By monitoring fidelity to project goals, the NCC learned first-hand about system barriers and facilitators to IPE. The NCC addressed system barriers by seeking changes to existing local and national VA policies and procedures, developing new IPE national policies, advocating for ongoing resources through its communication with OAA, other VA leadership, and external agencies, and developing resources that could be disseminated throughout VA.
Academic Affiliate Partnerships	Because VA’s educational mission exists within the context of local community institutions and national professional standards, strong relationships are required between the VA and its academic affiliates. In this context, the academic affiliate is the sponsor of the training program and VA provides the clinical learning environment as a participating institution. Strong relationships are also required between academic affiliates to facilitate logistics and new processes required for IPE and faculty development.
Interventions
Program Structure and Leadership	Interprofessional leadership is essential to role model interprofessional behavior to staff, faculty and trainees, to work with local VA facility leadership, and to support the development of curricula involving all participating professions. Further work is required to understand the influence of factors such as individual personality, professional identity, and organizational structure on the effectiveness of interprofessional leadership. Leadership development of individuals to lead interprofessional teams should incorporate these influencing factors.
Space requirements	IPE education and collaboration between trainees, faculty, and patients require larger exam rooms and more space intentionally designed for the clinical education of groups rather than repurposed exam rooms.
Interprofessional Curriculum	Curriculum consists of learning objectives, instruction, and trainee assessment. Health professions’ trainees should experience both interprofessional and profession-specific curricula. The interprofessional curriculum should be interprofessionally co-developed. This is time-consuming but useful to understand each professions’ culture, capabilities, and roles. While it may be possible for profession-specific training programs to share learning objectives, didactic instruction, and trainee assessment strategies, workplace learning (clinical supervision) must be tailored to the local clinical environment. Advancing workplace learning requires making hidden curriculum explicit, as faculty and staff demonstrate the desired professional behaviors and attitudes that are presented to trainees in didactic instruction.
Interprofessional Faculty	Faculty development is necessary to teach skills across the four core domains and delivery of interprofessional team-based care. Faculty development should target clinical faculty and clinic staff such as RN care managers who interact with patients and trainees in the clinical setting. Formal clinical faculty academic appointments across all professions should be considered to ensure faculty development and the quality of education provided by VA clinical staff. The faculty appointment process varies by profession and therefore should be evaluated on a profession by profession basis.
Center Evaluation	Center evaluation should evaluate micro-level impacts on trainees, faculty, and patients; and meso level impacts such as institutional access and clinical metrics.
Outcomes
Short, Intermediate, and Long-Term Outcomes	Enterprise evaluation should include system outputs at the macro level to support policy and decision making, such as trainee learning, program satisfaction, patient-level clinical outcomes and costs of the program. Evaluation of interprofessional education requires interprofessional/interdisciplinary evaluation teams using mixed methods approaches. The enterprise evaluation program should be started early and include evaluators in discussions from project conceptualization.

The first lesson is that building the foundation for IPE in VA primary care settings required an initial outlay of financial resources. Because primary care IPE was a new educational model, protected time of clinicians and staff was necessary to identify needs, develop interprofessional workplace learning strategies and faculty, and shape the interprofessional clinical learning environment within each Center. Experiences across Centers could then be used to identify important features of the interprofessional clinical learning environment that could be generalized for future educational transformation.

In addition to financial resources, OAA established the NCC to link individual Centers with system transformation efforts, monitor fidelity to project goals, and monitor appropriate use of funding. The NCC facilitated interaction between individual Centers to advance a standardized enterprise-wide curriculum that could be disseminated nationally. The NCC shaped the development of national VA IPE policy informed first-hand by individual Center experiences. For example, based on identified barriers encountered by Centers, the NCC successfully modified national business rules within the VA’s electronic health record to facilitate interprofessional graduated supervision and to properly relate trainee clinical effort to workload calculations and staffing policies. Finally, the NCC contributed to national policy and professional accreditation discussions to advance IPE, such as the establishment of IPE competencies for NP resident training.

Another lesson is that strong collaborative relationships between the VA and academic affiliates, as well as academic affiliates with each other, are required to develop IPE training programs. These relationships support logistics and resource sharing, curriculum innovation, faculty development, and educational/program evaluation. The differential rate of academic faculty appointments by profession among VA interprofessional faculty highlight the need to further explore profession-specific barriers to the faculty appointment at the academic affiliate.

With respect to the multicomponent intervention, we learned that interprofessional leadership is vital to the development and delivery of effective interprofessional curricula, as well as for role modeling of leadership behaviors to trainees across professions. The high rate of turnover in leadership points to the need for deeper understanding of effective interprofessional leadership models that may be influenced by profession-specific cultural issues. Space was a key resource enabling IPE. We sought space design consultation for more efficient ways to utilize existing space. However, since actual space construction was a long-term solution we engaged the VA Office of Primary Care to address policy for architectural standards to enable IPE in VA settings. The Centers built interprofessional curricula that eventually integrated three instructional strategies, i.e., didactic, workplace learning, and reflection across four core domains of interprofessional collaboration, shared decision-making, sustained relationships, performance improvement. These curricula are exportable to the rest of the VA and the national health care system. Finally, within clinical workplace settings, interprofessional faculty include not only profession-specific faculty but all clinic staff who work collaboratively to deliver quality patient care. Faculty development for IPE should include profession-specific faculty and clinic staff who work with trainees.

Our analyses have several limitations. First, the CoEPCE Annual Report and the semi-annual reports were written by the NCC and the Centers, respectively, and therefore may be biased towards NCC or a Center’s interpretation of events. These limitations were minimized by triangulating across data sources and types of data, i.e., qualitative and quantitative to arrive at valid descriptions of what occurred. This project takes place in the VA and some of these experiences and results may not generalize beyond VA. However, the VA is the largest education and training effort for health professionals in the United States and provides education in collaboration with 135 of 144 allopathic medical schools, 30 of 33 osteopathic medical schools, and more than 40 other clinical health professions education programs (Office of Academic Affiliations, 2016), Finally, for various reasons the enterprise evaluation was initiated later than other NCC functions, and assessment of some short term outcomes was not fully initiated until the beginning of Stage 2, underscoring the need to initiate the enterprise evaluation strategy early and with a clear purpose (Reeves, Boet, Zierler, & Kitto, 2015). As we move forward, we will continually review and monitor the enterprise evaluation strategy to evaluate IPE impacts on trainees, faculty, patients, the VA population, and the health care delivery system (Cox, Cuff, Brandt, Reeves, & Zierler, 2015).

Concluding comments

Large-scale systems transformation of education and clinical care is a complex undertaking that requires dedicated work of national leadership and policymakers, clinical and academic leadership, faculty, clinical staff and the trainees themselves. Future work will focus on improving our understanding of the interprofessional clinical learning environment to support workplace learning leading to quality Veteran care. Ultimately, these models will be disseminated throughout VA to achieve system-wide transformation.

Declaration of interest

The authors report no conflicts of interest. The authors alone are responsible for the writing and content of this paper.

Acknowledgements

The authors would like to thank the following individuals for their contributions to the project: Malcolm Cox, MD; Ward Newcomb, MD; David Latini, PhD, LMSW; and Jennifer Hayes, EdM. Logic model development was informed by Bridget O’Brien, PhD, Anne Poppe, PhD, Anais Tuepker, PhD, and Mimi Singh, MD. Bridget O’Brien, PhD and Mary Dolansky, PhD, RN, FAAN provided thoughtful reviews of this manuscript. In addition, this project could not be where it is today without the leadership and staff within VA’s Office of Academic Affiliations and Office of Primary Care. Special thanks to the interprofessional leadership, faculty, clinical staff, trainees and academic affiliates of CoEPCE Centers at Boise, Idaho; Cleveland, Ohio; Houston, Texas; Los Angeles, California; San Francisco, California; Seattle, Washington and West Haven, Connecticut.