Abstract
Background: The issue of quality assurance (QA) and quality improvement (QI), being the quality of medical education intimately related to the quality of the health care, is becoming of paramount importance worldwide.
Aim: To describe a model of implementing a system for internal QA and QI within a post-graduate paediatric training programme based on the ISO 9001:2000 standard.
Methods: For the ISO 9001:2000 standard, the curriculum was managed as a series of interrelated processes and their level of function was monitored by ad hoc elaborated objective indicators.
Results: The training programme was fragmented in 19 interlinked processes, 15 related procedures and 24 working instructions. All these materials, along with the quality policy, the mission, the strategies and the values were made publicly available. Based on the measurable indicators developed to monitor some of the processes, areas of weakness of the system were objectively identified and consequently QI actions implemented. The appropriateness of all this allowed the programme to finally achieve an official ISO 9000:2001 certification.
Conclusions: The application of the ISO 9001:2000 standard served to develop an internal QA and QI system and to meet most of the standards developed for QA in higher and medical education.
Introduction
The future scenario of the health system in Europe, among many other elements, will be characterized by a shortage of medical doctors, by an increasing circulation of professionals among the European countries and by the need to guarantee excellence in medical education and quality and safety of health care. Thus, being the quality of medical education intimately related to the quality of the health care, the issue of quality, of quality assurance (QA) and quality improvement (QI) in medical education is becoming of paramount importance in Europe, not only for the benefits of medical students and doctors but above all for the European citizens (WFME Citation2003; Karle Citation2006; Karle & Gordon Citation2007; Gordon et al. Citation2008).
QA in higher and particularly in medical education is by no means only a European concern. There is an increasing interest worldwide in quality and standards reflecting both the rapid growth of higher education and its cost to the public and private purse, and the increasing awareness that human resources play an ever-increasing role in a world competitive environment.
The European Union of Medical Specialists (UEMS) has been active since 1958 in order to promote the free movements of medical specialists while ensuring the highest quality of medical care. More recently, the UEMS has invited the different UEMS specialist sections and European boards to develop a supranational quality framework for the post-graduate medical education in Europe (UEMS Citation1996). This has been essentially accomplished by looking, for each medical speciality, at harmonizing the various national curricula, by recommending criteria to which the training centres should conform and by developing a European board examination.
Furthermore, a major emphasis for providing QA has been on the accreditation of the medical educational programmes based on external peer reviews, self-evaluation procedures and site visits, authoritative decisions and publication of the final report and decision (WFME Citation2003; Sirvanci Citation2004; Karle Citation2006).
Quality in medical education, however, is not only ministerial accreditation of individual programmes or institutional accreditation criteria or common board examination. It also requires a culture of quality and QA and QI systems applied to each individual medical education programmes. This aspect of QA in higher medical institution lies mostly within each institution itself, thus providing the basis for real accountability of the academic system within any quality framework (Ho & Wearn Citation1995; ENQA Citation2000).
Based on these considerations, the Board of the Post-graduate Paediatric Training Programme of the University of Padua in Italy decided to address the issue of QA and QI going through the ISO 9001:2000 certification process. The decision was prompted also by the fact that many clinical services of the hospitals, where the post-graduate medical students get their paediatric training, were already involved in the ISO 9001:2000 certification process.
The International Organization for Standardization (ISO) develops standards for use worldwide to help companies assimilate into the world market. ISO 9001:2000 standard outlines criteria for good quality management system, offering a methodological support to those organizations that want to demonstrate their ability to provide products/services meeting specific needs and their capacity of enhancing their customer satisfaction through continual improvement (ISO 2000; Kanholm Citation2002). The key words of this standard are: ‘customer satisfaction’ and ‘continual improvement’.
The peculiarity of the quality system based on the ISO 9001:2000 standard is the fact that it is focused on a processes management approach and not on contents and thus, can be applied to any type of organization.
The series of training and learning activities that a medical doctor has to go through in order to become a specialist can be considered as ‘a sequence of processes’, thus the methodology proposed by the ISO 9001:2000 standard was felt appropriate to pursue the issue of QA and QI for the post-graduate paediatric training programme (Eriksen Citation1995; Sirvanci Citation2004).
The aim of this manuscript is to describe the experiences, the opportunities, the limits and the perspectives that the ISO 9001:2000 standard seems to offer for pursuing QA and QI in higher education.
Methods
Setting
The post-graduate paediatric training programme of the University of Padua is a ministerial accredited 5-year programme that provides a Diploma of Specialization in Paediatrics. Approximately 80% of the learning activities take place in the setting of clinical practice under supervision with increasing levels of responsibilities. The remaining of the learning activities includes formal lectures, seminars, workshops and personal studies.
At the time this experience was run, 92 residents were attending the programme and each resident was planned to rotate in at least 15 of the 25 divisions of the Department of Paediatrics of Padua and of the affiliated hospitals.
The ISO 9001:2000 standard
In order to build a quality management system based on the ISO 9001:2000 standard, the requirements that have to be met were as follows:
to establish a quality policy and quality goals to support the quality policy;
to use a ‘process management approach’, which means managing the post-graduate paediatric training programme as a series of interlinked processes;
to document the quality management system in quality policy, quality manual, procedures, work instructions and records;
to define management responsibility;
to qualify teaching resources (human resources, infrastructure and work environment);
to plan product realization processes and control them and
to measure, analyse and improve its way of functioning.
Furthermore, the processes that were identified were clustered according to the Deming cycle and its adaptation to the ISO 9001:2000 standard (Figure 1a and b). The Deming cycle is a model for continuous improvement of quality. It consists of a logical sequence of four repetitive steps for continuous improvement and learning: ‘Plan-do-check-act’ (PDCA) (Walton Citation1990; ISO 2000a; ISO 200b). The ‘plan’ phase is the phase of making strategies and plans; the ‘do’ phase is the one in which the strategies and the plans are implemented; the ‘check’ phase is the one during which the efficacy of the processes and the activities are monitored and evaluated; the ‘act’ phase is the one during which actions for improving the system are identified and planned, based on the results of the ‘check’ phase.
Results
The obligations set by the ISO 9001:2000 standard resulted in the following actions.
Quality policy and quality goals to support the quality policy
The policy for QA, the mission, the vision, the strategies, the values and the quality goals of the post-graduate paediatric training programme were developed by the Board of the programme guided by the concepts of:
meeting the educational needs of the doctors in training, the health care needs of the children and their families these future specialists will be serving;
implementing the idea of life-long learning;
respecting the health care and social structures in which these doctors will be operating; and
achieving professional satisfaction of the staff members involved in the teaching/training activities.
The ‘process management approach’
The overall process of the post-graduate paediatric training programme was fragmented in a series of 19 interlinked processes, which were differentiated into three categories as listed in .
The ‘main processes (total of 5), meaning the ones intimately related to the training activities;
the ‘supportive processes’ (total of 5) the ones which, even if not contributing directly to the training itself, are essential for the appropriate realization of the ‘main processes’; and
the ‘general processes,’ (total of 9), the ones offering general support to all the others in order to meet all the ISO 9001:2000 requirements.
Table 1. Processes underlining the activity of the post-graduate paediatric training programme
The differentiation of the processes into selected categories was done with respect to the ISO 9001:2000 standard, which recommends to cluster them according to common overall goals. More precisely, the decision to cluster the processes into three main categories was taken to replicate what was already done for the ISO 9001:2000 certification of the clinical services and thus to share some of these processes.
The distribution of these processes, according to the Deming cycle, as adapted to the ISO 9001:2000, along with ‘the mission, the vision, the values and the quality policy’, are reported in . An example of how a process – the management of the clinical training – is articulated is reported in .
Figure 1. (a) The Deming cycle (PDCA). (b) The adaptation of Deming cycle according to the ISO 9001:2000 standard for continuous quality improvement (adopted by ISO. ISO 9001:2000 Quality management systems – Requirements. International Organization for Standardization, Geneva, 2000).
Figure 2. Structure of quality management system in the paediatric training programme. Note: ER, emergency room; NICU, neonatal intensive care unit; PICU, paediatric intensive care unit; RAC, resident affair committee.
Figure 3. An example of how a process, ‘the management of clinical training’, is articulated and developed.
Documentation of the quality management system
Documents reporting the policy, the quality manual, the description of each process and the related specific procedures (total of 15), working instructions (total of 24) and records (total of 16) were produced. All documents were then shared with each member of the post-graduate paediatric training programme and finally, made publicly available in the web page of the Department of Paediatrics under quality representative control and supervision.
The procedures and the working instructions intimately related to the training activities (see , ‘main processes’) were developed keeping in mind: (i) the educational objectives and the learning outcome of the post-graduate paediatric training programme dictated also by the national regulations; (ii) the curriculum and the programme design and (iii) the teaching/training resources available. The procedures describing the medical activities of the doctors in training, the learning objectives of each rotation, the organization of the clinical activity for training purposes, the residents’ responsibilities according to their level of maturation, the medical procedures that they are expected to perform and the division chair's and tutors’ responsibilities were carefully outlined.
These procedures were written by a panel of members of the post-graduate paediatric training programme in cooperation with the director of each division. In doing so, these procedures were integrated within the ones produced for the ISO 9001:2000 certification of the clinical activities.
Management responsibility, qualification of teaching resources, planning and controlling of product realization processes
As shown in and , specific processes and procedures were related to the responsibility of the Resident Affair Committee, two to the resources management and eight to the realization of the Paediatric Education Programme.
Measuring, analysing and improving
A system to record, monitor and produce objective data regarding some of the activities of the post-graduate paediatric training programme was introduced. This system allowed to record and count the number of learning activities (e.g. lectures, seminars, rotations, workshops, electives done by each resident), the actual medical procedures performed by the residents (e.g. lumbar punctures, bone marrow aspirations), the number and contents of meetings of the Board with the teaching staff and trainees. Furthermore, a comprehensive evaluation system was also implemented. This system aimed to evaluate, using ad hoc questionnaires based on a scale of grade 1–5, the residents in terms of medical competency, professional attitudes, level of autonomy and their satisfaction (‘customer satisfaction’) in relationship to the learning outcomes that were declared for each rotation.
All this served to develop 14 objective indicators to continuously monitor the functioning of the post-graduate paediatric training programme () and to set improvement measurements.
Table 2. Indicators of the actual implementation of some selected processes
The data derived from the first 6 months of monitoring the trainees’ activities and of evaluating their performances and their levels of satisfaction allowed the Board of the programme to identify areas for improvement and subsequently to set QI goals. For example, based on the overall level of satisfaction of the residents for each rotation, as graded by the questionnaires, management activities were identified for the divisions with some weaknesses (median score less than 3) (). These consisted of a better definition of learning and educational goals of some rotations, and a revision of the training activities, particularly with reference to medical procedures.
Figure 4. Residents’ level of satisfaction related to the achievement of the learning objectives declared for each rotation.
The entire process of achieving the ISO 9001:2000 certification took almost 18 months. During this period, three internal audits of the ISO 9001:2000 standards were conducted by an expert group. They were focused on structures, processes and teaching elements with inefficiencies and weaknesses, which then allowed to identify corrective actions.
After the three internal audits, an accredited third-party registrar audited the quality system and certified that the post-graduate paediatric training programme of the Padua University met the ISO 9001:2000 requirements.
Discussion
The application of the ISO 9001:2000 standard helped the post-graduate medical educational programme of the University of Padua to develop its own internal QA and QI system and its culture of quality. A fundamental requirement of any organization is that the governance of the processes, which are the foundation of its way of functioning, is the first guarantor of the quality of its performance, irrespective of the process and of the type of performances in themselves (Kanholm Citation2002; Sirvanci Citation2004). The ISO 9001:2000 standard allowed the Board to identify, define and declare in writing, the processes and their interactions underlying professional post-graduate paediatric training in Padua and provided a system to control, monitor and improve them. All this, in brief, has meant better governance.
Prior to the application of the ISO 9001:2000 standard, some of these processes were already existing and applied, but not within such a coherent, effective, controlled and overt structure.
The application of the ISO 9001:2000 standard also prompted the use of the PDCA approach of continuous QI (). This was achieved by selecting four processes that were thought to be critical for the functioning of the training programme, by developing, for each of them, measurable indicators of functioning and thus, by producing data to be used to monitor and improve them (). One example is the measurement of the level of satisfaction of the residents (= indicator) for each specific rotation (= process: management of the clinical training). This allows to identify areas for improvement and to develop concrete action plans. The PDCA is an ongoing process which helps to build innovation into the post-graduate paediatric training programme. It could also serve, in a near future, to build benchmarking data for higher education for comparing performance of different training programmes (Caroll et al. Citation2006; Jones et al. Citation2007).
An added value of the obligations set by the ISO 9001:2000 standard was the fact that they contributed to further develop a diffuse culture for recognizing the importance of QA and QI in everybody's work, considering that all the staff members of the paediatric hospital were actively involved.
Attempts to introduce quality concepts, like ISO standard, into higher education began in the late 1980s, but to our knowledge this is the first time that it is applied to post-graduate medical training programme. Initially, there was a strong scepticism of the idea of applying a total quality management (TQM) model, as an ISO standard, to higher education institutions because of their low confidence levels of ability to manage the processes (Ho & Wearn Citation1995). Thus, the implementation of ISO standard was confined to the administrative structures and non-academic processes of the University (Sirvanci Citation2004). Although TQM developed within the manufacturing environment, it soon became evident that the benefits are equally applicable to any environment, including service entities such as higher education (Eriksen Citation1995). This can be provided by viewing the educational process as a production process – the actual process whereby resources are used to convert inputs (i.e. medical doctors) into outputs (i.e. specialists). This has been witnessed by a recent experience of a quality management programme based on ISO standard implemented for all the 34 medical schools in Germany, which had a significant impact on the development of the faculties and resulted in an improvement of the quality of medical education in term of basic science, clinical research and patient care (Dieter Citation2008, Citation2009).
Apart from these considerations, we chose the ISO 9001:2000 standard because many of the clinical services where the doctors in training learn their practice were also ISO 9001:2000 certified. A comprehensive system for QA and QI for post-graduate medical education must involve also the actual clinical practice, and this is not only for the benefits of the doctors in training but also, if not more importantly, for the patients, their families and the entire society. In fact, a quality management system is the pre-requisite for an effective clinical risk management. In this prospective, it is relevant to note that ISO 9001:2000 standard is popular in many parts of Europe and used in departments, laboratories or hospitals in virtually all EC member states (Heaton Citation2000; Sweeney & Heaton Citation2000).
The issue of QA and QI in higher and medical education has been the objectives of many international organizations and, among them, the European Association for QA in Higher Education (ENQA) and the World Federation for Medical Education (WFME). Both these organizations have developed a set of standards, procedures and guidelines on QA, such as assessing the policy and procedures for QA; approving, monitoring and periodically reviewing programmes and awards; assessing the students; assuring quality of teaching staff; assuring learning resources and student support; assuring information systems; giving public information (Eriksen Citation1995; ENQA Citation2000; WFME Citation2003). The obligations set by ISO 9001:2000 standard allowed to meet almost all of these standards. In the European prospective, this is quite important for favoring the free circulation of professionals among the various European states, thus providing national health system with the opportunity to recruit and utilize individuals trained in other countries.
The application of the ISO 9001:2000 standard requires an ‘external component’ for QA. In fact, it was an accredited independent third party, which released the ISO certification. This third party evaluation did not represent a peer-reviewed process; it was focused only on how the processes and the learning activities were planned, declared, made publicly available, measured and kept under control. This is an intrinsic limitation of this standard along with the fact that there are no guidelines for the use of the ISO standard for medical education. The consequence is that many variations in the ways the ISO is applied in the various organizations devoted to medical education may exist as already emerged for the implementation of this standard to the clinical services (Sweeney & Heaton Citation2000).
In summary, a concrete experience for developing a QA and QI system for post-graduate medical education is presented. This is not proposed as a gold standard to be applied to similar programmes, but simply as a possible referring model for developing common European framework for QA and QI in post-graduate medical education. The ISO 9001:2000 standard thus provides a solid foundation for the use of the peer-reviewed processes in post-graduate medical education in Europe.
Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.
Additional information
Notes on contributors
Liviana Da Dalt
LIVIANA DA DALT, M.D., is an Associate Professor of Paediatrics, Faculty of Medicine, University of Padua; Clinical Director, Paediatric Emergency Unit, Department of Paediatrics, University of Padua; Head, Education and Trainee Committee; Vice-director, Paediatric Residency Programme and Senior expert in Quality Management Systems in Clinical Services and in higher education.
Silvia Callegaro
SILVIA CALLEGARO, M.D., is a Paediatrician; Training researcher in Doctoral School of Developmental Medicine and Planning Sciences, Department of Paediatrics, University of Padua and Junior expert in Quality Management Systems.
Anna Mazzi
ANNA MAZZI, Ph.D., holds a Ph.D. degree in Management Engineering, Engineering Faculty, University of Padua; Owner of Research Grant in Sustainable Development, Engineering Faculty, Padua; Teaching collaborator, Engineering Faculty, Padua and Senior expert in Quality Management Systems in service sector and in higher education sector.
Antonio Scipioni
ANTONIO SCIPIONI, Ph.D., is a Full Professor, Engineering Faculty, University of Padua; Director of the Master on Strategic Environmental Management, Engineering Faculty, Padua; Director of the Environmental and Quality Studies Centre, Engineering Faculty, Padua and Member of the Accreditation Committee of SINCERT.
Paola Lago
PAOLA LAGO, M.D., is a Senior staff member of the Neonatal Intensive Care Unit, Department of Paediatrics University of Padua; Director of Quality Improvement and Risk Management Programme of the Department of Paediatrics, University of Padua and Faculty, Paediatric Residency Programme, University of Padua.
Maria L. Chiozza
MARIA LAURA CHIOZZA, M.D., is the Head, Quality Programme, University Hospital, University of Padua and Faculty, Paediatric Residency Programme, University of Padua.
Franco Zacchello
FRANCO ZACCHELLO, M.D., is a Full Professor of Paediatrics; Faculty of Medicine, University of Padua; Past Director, Paediatric Residency Programme, University of Padua and Past Chairman, Paediatric Department.
Giorgio Perilongo
GIORGIO PERILONGO, M.D., is a Full Professor of Paediatrics, Faculty of Medicine, University of Padua; Director, Paediatric Residency Programme, University of Padua and Chairman, Paediatric Department.
References
- Caroll VS, Thomas G, de Wolff D. Academic quality improvement program: Using quality improvement as tool for the accreditation of nursing education. Qual Manag Health Care 2006; 15(4)291–295
- Dieter PE. Quality management of medical education at the Carl Gustav Carus Faculty of Medicine, University of Technology, Dresden, Germany. Ann Acad Med Singapore 2008; 37(12)1038–1040
- Dieter PE. A faculty development program can result in an improvement of the quality and output in medical education, basic sciences and clinical research and patient care. Med Teach 2009; 13: 1–5
- ENQA (The European Association for Quality Assurance in Higher Education). 2000. Available from: http://www.enqa.eu/index.lasso (last accessed January 21, 2009)
- Eriksen SD. TQM and the transformation from an elite to a mass system of higher education in the UK. Qual Assur Educ 1995; 3(1)14–29
- Gordon D, Christensen L, Dayrit M, Dela F, Karle H, Mercer H. Educating health professionals: The Avicenna project. Lancet 2008; 371: 966–967
- Heaton C. External peer review in Europe: An overview from the ExPeRT project. Int J Qual Health Care 2000; 12: 177–182
- Ho SK, Wearn K. A TQM model for higher education and training. Training Qual 1995; 3(2)25–33
- ISO 2000a. ISO 9001:2000. Quality management systems – Requirements. International Organization for Standardization, Geneva.
- ISO 2000b. ISO 9000:2000. Quality management systems – Fundamentals and vocabulary. International Organization for Standardization, Geneva.
- Jones ML, Hobson RS, Plasschaert AJM, Gundersen S, Dummer P, Roger-Leroy V, Sidlauskas A, Hamlin J. Quality assurance and benchmarking: An approach for European dental school. Eur J Dent Educ 2007; 11: 137–143
- Kanholm J. ISO 9001:2000 requirements. 92 requirements checklist and compliance guide. AQA Co, Pasadena, CA 2002
- Karle H. Global standards and accreditation in medical education: A view from the WFME. Acad Med 2006; 81(12 Suppl.)S43–S48
- Karle H, Gordon D. Quality standards in medical education. Lancet 2007; 370: 1828
- Sirvanci MB. Critical issues for TQM implementation in higher education. The TQM Magazine 2004; 16(6)382–386
- Sweeney J, Heaton C. Interpretation and variations of IS 9000 in acute health care. Int J Qual Health Care 2000; 12: 203–209
- UEMS (European Union of Medical Specialist). 1996. Available from: http://www.UEMS.net (last accessed January 21, 2009)
- Walton M. Deming management at work. G.P. Putnam's Sons, New York 1990
- WFME (World Federation for Medical Education). 2003. Postgraduate medical education – WFME global standards for quality improvement. Available from: http://www.wfme.org (last accessed January 21, 2009)