Preparing for an institutional self review using the WFME standards – An International Medical School case study

Abstract

Background: Curriculum reform poses significant challenges for medical schools across the globe. This paper describes the reforms that took place at the medical school of the Royal College of Surgeons in Ireland (RCSI) between 2005 and 2008 and the institutional self review process that accompanied these reforms.

Results: Although fully accredited with the Irish Medical Council the RCSI sought additional detailed review of all aspects of its undergraduate medical program. Five medical educationalists were invited to visit the College in 2005 and again in 2008 to act as ‘critical friends’ and guide the self review using the World Federation for Medical Education (WFME) standards which had recently been adopted in Ireland.

Conclusion: The process of institutional self review (as opposed to more high stakes accreditation) can bring about significant reform, especially when supported by a panel of ‘critical friends’ working alongside faculty to help guide and support sustained curriculum reform. The WFME standards continue to provide a useful framework to consider all medical education activities within a medical school engaged in continuous renewal. Adequate preparation for such reviews is critical to the success of such an undertaking and should be supported by a comprehensive communication strategy and project plan.

Background

Since its inception in 1784, the Royal College of Surgeons in Ireland (RCSI) has played a leadership role in Irish surgical and medical education and research. The Medical School of the RCSI dates from the 19th century with various postgraduate faculties added in the 20th century. As an international institution with an international footprint extending from Ireland to Africa, the Middle and Far East more than 60 countries are represented on its international student and Alumni body.

Curriculum reform is not new at the RCSI. A former Professor of Anatomy in the 1990s had concluded in an internal report on the medical curriculum that the division between basic sciences and clinical disciplines provided “much scope for development in terms of horizontal integration and coordination”.1 Key recommendations at that time included the need to appoint clinical course supervisors ‘with authority’; identify core curriculum material; increase time available for self directed study and reduce the burden of assessment. Most of the reforms attempted in the last two decades have been designed to specifically enhance integration of the basic sciences and clinical education and overcome the “artificial divide” between the scientific teaching in the so called “pre-clinical” years and bedside teaching in hospitals in the latter years of the program (Association of American Medical Colleges Washington DC 1984; Kaufman 1985; Kaufman et al. 1989; Cuban 1990, 1997; World Health Organisation 1991; Des et al. 1992; Cohen et al. 1994; Bloom 1995; Jolly & Rees 1998; Mennin & Krackov 1998; Bland et al. 2000).

Table

Practice points

The WFME standards continue to provide a useful framework to consider all medical education activities within a medical school engaged in continuous renewal. The process of engaging a panel of independent medical educationalists to visit the school over several years using the WFME standards can help guide and support sustained curriculum reform. Adequate preparation for the visits is critical to the success of such an undertaking and should be supported by a comprehensive communication strategy and project plan.

The WFME standards

The Executive Council of the World Federation for Medical Education (WFME) first published a position paper on the topic of international standards in medical education in 1998. Subsequently, an international Task Force was established by WFME with the purpose of defining international standards for basic [undergraduate] medical educational programmes. The main purpose of the Task Force was to develop undergraduate medical education standards that could be applied internationally. Key considerations in developing the WFME standards were as follows:

• Standards should serve as an impetus for review and perhaps change through institutional self-evaluation.

• Standards must take account of the variations in medical education between countries; due to differences in teaching tradition, culture, socio-economic potential, the health and disease spectrum, and different forms of health care delivery systems.

• Standards should not dictate content, drive quality down, prohibit educational methodology experimentation, rank schools or be used for political purposes.

• Standards should emphasise the universality of the scientific basis of medical education and that the task of medical education is to prepare physicians to care for the healthy, the ill, the disabled and the injured citizens of the world.

• Standards may serve to establish a system for national or international accreditation of medical education programmes (Grant et al. 2003, p. 6)

The standards were further refined based on feedback from international advisors and from a number of conferences around the world. The standards and Guidelines for their use are now widely used as a WFME adopted document: Quality Improvement in Basic Medical Education WFME International Guidelines.

RCSI and the WFME standards

The proposal to commence the first Graduate entry medical program in Ireland provided the impetus for a unified focus on curriculum matters at the medical school and in 2005 a panel of external educationalists were invited by the then Dean to review the curriculum using the WFME Global standards (the Basic Medical Education: Global Standards for Quality Improvement (2003). The 2005 team findings were sobering but insightful. The report commented on every aspect of the medical education program and its governance with numerous recommendations for improvement. Overall, it was the considered view of the visiting external panel that two thirds of the 36 WFME sub-areas had not achieved the quality standard and some of the basic standards had not been met. Key areas of weakness included the lack of clear statement of curriculum objectives, poor linkage between curriculum content, delivery and assessment and an inadequate evaluation strategy. Other areas of weakness identified included the need for improved curriculum governance with the need for explicit authority and control over resources to be given to the central curriculum committee to help drive forward the implementation of an integrated curriculum. The lack of technical expertise in the construction and implementation of valid, reliable, and efficient assessments was also highlighted. In particular the need for a medical education unit with the leadership, capacity, and authority needed for successful curriculum reform.

Following the results of the 2005 review the decision was taken by the new incoming Dean and his executive to initiate a new period of focussed incremental curriculum reform. The new Graduate entry program provided the ideal ‘test bed’ in which to explore new innovation in the delivery of the curriculum such as small group and case based learning (Des Marchais et al. 1992; Williams 2005; Dupuis & Persky 2008). Slowly these new approaches were trialled in the established five and six year programs where lectures still constituted the principal teaching delivery method. Several new appointments were made to the schools senior executive committee which included three new Cycle Directors replacing the departmentally based governance structures. In mid 2007 the appointment of a Professor in Medical Education saw the pace of curriculum reform increase significantly and a formal strategy for reform implemented. The previously dormant central curriculum committee (Curriculum and Assessment Board, CAB) was re-established. Reporting to CAB, several key working groups were established on the recommendation of the Professor for Medical Education to oversee the development of curriculum objectives (COWG); an assessment strategy (AWG), an evaluation strategy (EWG) and a medical education research strategy (MERG) (see Figure 1). A new curriculum database project was launched whose function was to ‘map’ the content of all modules in the curriculum. The student body became active partners in the curriculum reform process participating in several curriculum working groups and attending medical education seminars and international conferences. Under the lead of the Professor for Medical Education, a formal faculty development program was initiated with invited speakers from UK, Australia and US. Presentation and attendance at faculty sponsored activities such as curriculum forums and national and international conferences on medical education increased and enrolments in recognised postgraduate qualifications in medical education also increased.

Figure 1. Revised governance structure at the RCSI to support curriculum reform.

The follow up visit

In late 2008 the decision was taken by the Dean to seek a three year follow-up voluntary review against the WFME standards, availing where possible of the same team members as the 2005 visit. The decision was enthusiastically received by staff at the RCSI who welcomed the opportunity to demonstrate what progress had been made in the intervening three years. Compared with the 2005 visit, the November 2008 report showed significant improvement against most of the WFME standards. The visiting panel acknowledged the strong commitment of the RCSI to medical education and training and to continuous improvement. Specifically the panel recognised the significant work which had taken place in designing and developing a comprehensice set of curriculum outcomes against which each module was being mapped. These curriculum objectives were felt to be well matched to the needs of the community the RCSI served and the commitment to social responsibility, to research, and to involvement in the local and wider community. The Panel applauded the increased focus on staff development noting that a commendable number of staff had under the leadership of the Office for Medical Education identified an interest in medical education, and were undertaking a qualification in this area, attending international conferences and contributing actively to international debate. In the area of Evaluation the Panel commended the Faculty's plans to survey recent graduates with respect to their perceived preparedness for hospital practice (Figure 2) and the plans to survey several cohorts of RCSI graduates with respect to postgraduate career choices and contribution to the medical workforce. Overall 90% of the baseline World Federation for Medical Education Standards had been met and 20 of the 36 subareas received endorsement as having attained the gold standard ‘’Quality’’ marker.

Figure 2. Mean scores and confidence intervals of the 2007/2008 interns of RCSI on each domain of the PHPQ.

Discussion

Although recognised that ongoing work needs to be done to achieve all of the WFME quality standards several factors have subsequently been identified as having contributed to the success of the 2008 review.

Engaging with Faculty

Commitment to the WFME review process was evidenced by top level support within the organisation. Once the decision was taken to conduct a follow up visit, regular progress updates were sought by the RCSI governing body, the Medical Faculty Board (Figure 1). Planning for the visit was supported by a 40 page project plan and a comprehensive communication strategy developed by the Office for Medical Education which engaged internal and key external stakeholders including regulatory bodies, alumni, the department of health and all teaching hospitals. A dedicated newsletter and website provided regular progress reports in the 12 months leading up to the November 2008 review.

Engaging with the visiting team

A preliminary visit to the RCSI in February 2008 by the WFME panel established the terms of reference for the follow up visit and agreement was reached on the content of the written submission which was to provide a concise description of progress that had been made against the nine WFME standards since the 2005 visit. This submission contained specific and tangible evidence of curriculum reforms such as the revised curriculum outcomes, the new assessment and evaluation policies and a recently devised curriculum map as well as sample minutes of the newly established curriculum working group meetings.

The relationship between the visiting team and the RCSI was that of ‘critical friend’ and although staff were advised that this was a voluntary self study it was the preference of both faculty and the visiting team that the visit be conducted in an atmosphere not dissimilar to that of a high stakes accreditation visit. The desire to improve the performance of the 2005 visit was a paramount concern for the RCSI faculty, despite having received recent full accreditation with the Irish Medical Council in the interim. Throughout the planning for and conduct of the visit there was regular dialogue between the Office for Medical Education, which coordinated the visit and the visiting team. Several revisions were made to the program for the visit so as to best capture that which had changed in response to the 2005 findings. The proposed additional membership of the team was also by mutual consent but in all cases selecting academics with a strong international medical education profile.

Engaging with the WFME standards

WFME 1: Mission and objectives

The first area which the project plan prioritized was the lack of robust institutional objectives (Kassebaum et al. 1997). A 24 member Curriculum Outcomes Working Group (COWG) chaired by the Professor for Medical Education was convened to develop a set of curriculum outcomes which was informed by international sources of defined competency frameworks such as CanMEDS (CanMEDS 1996); “Tomorrows Doctors” (UK)(GMC February 2003); ‘’The Scottish Doctor’’; Medine Tuning Project (Europe) (Taskforce) and the Association of American Medical Colleges (Association of American Medical Colleges Washington DC. 1984) as well as recent graduates and other Alumni (see later). Referred to as the RCSI Medical Graduate Profile (MGP), this ‘profile’ of the ideal medical graduate was subdivided into five “themes” which provided a new framework for undergraduate learning, curriculum organisation and assessment. The use of themes to group related sub-outcomes of the MGP championed by senior academics, reflected the strategic shift away from the traditional discipline based course to a more integrated approach to medical undergraduate teaching at RCSI.

Once formulated the MGP was considered in detail by principal internal stakeholders including students and then circulated to key external stakeholders for comment. This external stakeholder consultation process included feedback from Alumni representing over 60 nations across the globe recognising the unique position occupied by RCSI as an international medical school.

The MGP informed many of the subsequent curriculum initiatives recommended by the 2005 WFME team such as the curriculum map, assessment and the evaluation strategy.

WFME 5: Academic staff/faculty

Educational support of teaching staff particularly in the context of curriculum change is challenging. Jolly (Jolly 2002) reminds us that “modifying a curriculum is likely to be difficult. Without faculty development it may well be impossible” (p. 945). Recognizing the need for regular and targeted faculty development, RCSI committed a dedicated budget to supporting the activities of the Office of Medical Education and its faculty development programs. These programs were not interpreted narrowly as merely attendance at workshops on teaching skills but rather as Irby's work attests the program of medical education seminars, workshops and forums also focused on learning theory, assessment and evaluation. A particular focus was in the area of assessment specifically item writing, OSCE station construction and marking and standard setting. International speakers were invited to present and staff were encouraged to present their work on curriculum innovation at national and international meetings. The visiting team recognized the potential for RCSI to become a centre where medical educators from elsewhere come to enhance their medical education skills.

WFME 7: Program evaluation

An evidenced based approach to curriculum reform was another key strategy articulated in the project plan. As first year graduates (Interns) are arguably well placed to comment on the perceived effectiveness of their undergraduate training in preparing them for their first postgraduate year (Hill et al. 1998) recent graduates of RCSI were asked to report on their perceived level of preparedness for hospital practice using the Preparedness for Hospital Practice Questionnaire (PHPQ) (Dean et al. 2003). This had not previously been undertaken. Eighty per cent (80%) of graduates of RCSI who were undertaking their internship in Ireland in 2008 (from class of 2007/2008) returned completed questionnaires2 and feedback obtained was used to inform the process of articulating concise curriculum outcomes as well as offering the first comparisons with responses by interns graduating from other international institutions. The PHPQ was developed and validated in Australia and allows interns to self-report their preparedness for hospital practice (Hill et al. 1998). The 41-item measure assesses the perceptions of first year doctors in relation to the adequacy of their medical training for hospital practice and has been used in a number of Australian medical schools (Dean et al. 2003; Mac Carrick 2005). Eight subscales assess key aspects of hospital care: interpersonal skills; confidence and coping; collaboration (team approach to medical care); patient management and practical skills; understanding science (as the basis of disease and therapeutics); prevention (preparedness to incorporate health promotion and disease prevention with hospital practice); holistic care (appreciation of the impact of multiple variables on patients health and disease); and self-directed learning (evaluation of the performance, identification of learning needs). Figure 2 shows the means together with their confidence intervals.

Comparing the mean domain scores with two previously-published cohorts from the University of Sydney, Australia (Dean et al. 2003) the mean scores of the RCSI graduates were found to lie intermediate between the two. Figure 3 shows the mean scores of the three groups:

Figure 3. Mean scores on PHPQ of University of Sydney problem-based learning (green); University of Sydney traditional curriculum (blue) and RCSI (red).

The results of this survey and written feedback from interns were used to inform the development of the new curriculum objectives by ensuring they focused on practical areas of preparedness for hospital practice. This was the first formal recent graduate survey conducted at RCSI and despite its limitations (Dean et al. 2003) was the first tangible evidence in support of RCSI's commitment to regular graduate evaluation.

WFME 8: Governance and administration

The re-establishment of the central curriculum committee with key working groups focussing on assessment, evaluation and curriculum objectives ensured a dedicated structure to support the responses to the 2005 review recommendations. In recognition of the potential barriers often facing curriculum committees such as lack of administrative support and “clout” to implement new ideas (Bouhuijs 1993; Neufeld et al. 1995) this committee was chaired by the Dean. For the first time RCSI had a central curriculum policy committee to establish and monitor educational policies for the school and approve major curriculum revisions (Davis & White 2002). The appointment of a Professor in Medical Education supported by a full-time project officer was further testament of the commitment to educational reform.

Conclusion

The WFME standards provide a useful framework to consider all medical education activities within a school attempting to embrace best practice.The process of engaging an international panel of medical educationalists over a three year period using the WFME standards as part of an institutional self review proved to be a useful way for RCSI (an international medical school) to ensure that all aspects of the school's aims, structures and processes were carefully explored. Adequate preparation for the visits, clear terms of reference, a comprehensive communication strategy and project plan to guide the schools efforts are all critical to the success of such an undertaking. The WFME standards continue to provide a useful framework to guide and support continuous renewal in a medical school.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

Notes

1. Monkhouse W.S. The clinical years of the undergraduate medial course in the {Institutions name}, 1991.

2. Ethics granted by {Institution name} Research Ethics Committee.