Doctors or technicians: assessing quality of medical education

Abstract

Medical education institutions usually adapt industrial quality management models that measure the quality of the process of a program but not the quality of the product. The purpose of this paper is to analyze the impact of industrial quality management models on medical education and students, and to highlight the importance of introducing a proper educational quality management model. Industrial quality management models can measure the training component in terms of competencies, but they lack the educational component measurement. These models use performance indicators to assess their process improvement efforts. Researchers suggest that the performance indicators used in educational institutions may only measure their fiscal efficiency without measuring the quality of the educational experience of the students. In most of the institutions, where industrial models are used for quality assurance, students are considered as customers and are provided with the maximum services and facilities possible. Institutions are required to fulfill a list of recommendations from the quality control agencies in order to enhance student satisfaction and to guarantee standard services. Quality of medical education should be assessed by measuring the impact of the educational program and quality improvement procedures in terms of knowledge base development, behavioral change, and patient care. Industrial quality models may focus on academic support services and processes, but educational quality models should be introduced in parallel to focus on educational standards and products.

Keywords:

• educational quality
• medical education
• quality control
• quality assessment
• quality management models

Background

The mechanism for program evaluation in most medical education institutions is based on or adapted from industrial quality management models and/or accreditation. In Malaysia, the International Organization for Standardization (ISO) is widely used in higher education institutions, eg, University Malaya¹ and University Science Malaysia.² Accreditation is usually used to assess quality in the US and Canada and, again, the process is mainly based on industrial quality management models.³ Review of the latest publication from the Liaison Committee on Medical Education⁴ in June 2010 clearly indicates the emphasis put on academic services and processes. The document describes that, for accreditation purposes, institutions should provide certain activities, including ambulatory teaching, hospital teaching, problem-based learning, self-directed learning, student-centered learning, team-based learning, and communication skills training, and, if these activities are provided in the curriculum, provided other requirements are fulfilled, accreditation may be granted. There is no method to ensure whether these activities are completed successfully and if they have a desirable impact on students’ learning and knowledge base.

Academic results are used to assess achievement of outcomes, and this achievement is assessed by written and practical examination. Accreditation ensures that vetting of all examination questions is done, as evidenced by signed vetting forms, but again no consideration is given to the actual quality of the vetting process. Accreditation processes help only in ensuring the provision of required services, resources, support systems, curricular content, and recommended teaching methods and activities for students, but they cannot ensure improved quality of product as a result of these services and processes. In short, the accreditation processes being adapted from industrial models of quality assurance provide good evaluation of academic services and processes, but they fail to evaluate the impact of these processes on the final product, ie, student learning and education in terms of improved knowledge base and patient care.

ISO is originally an industrial model of quality management that was modified to be applied in higher education institutions. ISO certification ensures that the course is designed to meet the needs of customers and that the process is effective and efficient. The quality management system standards developed by ISO are intended to certify the processes and the system of an organization, but not the product.³ ISO and other industrial quality management models, eg, total quality management (TQM), assess the service component of organizations by constant improvement, management commitment, customer-driven definitions of quality, team work, and statistical techniques.⁵ These models are driven by customer satisfaction and define students in higher education institutions as customers. Institutions are urged to fulfill the needs and demands of customers and to deal with them as a matter of priority rather than as a secondary issue. This approach makes issues related to students more important and the undertaking of other activities, like education, less important.⁶

This whole process facilitates the production of technicians able to perform various tasks based on the outcomes set for the program without developing an educational base, ie, knowledge, attitude, critical thinking, and decision-making.

Researchers studying the implementation of quality assurance activities have shown that the academic staff considers these external quality assurance processes and procedures as overly onerous and detrimental to their real work of quality in teaching and learning. Researchers have also reported that industry-based quality assurance systems are unable to assure quality improvement of the educational product in any meaningful way.⁷

Performance indicators and benchmarking are commonly utilized in TQM models to assess the process improvement efforts of institutions. In higher education, these performance indicators include the student-teacher ratio, academic activity cost per student, mean completion time, student progress rate for specific competencies, and graduate employment status.⁸ Researchers suggest that these performance indicators may be the measure of the fiscal efficiency of the institution, but cannot be used to measure the quality of the educational experience for the students.⁹

Discussion

The fact that the educational component is more difficult to assess than the training component has encouraged educational institutions to adapt industrial quality assurance models. Application of industrial quality models in medical education helps in facilitating the training component in terms of competencies, but may compromise the educational component.⁹ Assessment of students by measuring their capability in fulfilling specific tasks actually promotes the training of technicians. For example, it is observed that nurses are very efficient in attaching all the monitoring devices to a critically ill patient in the red zone area of Accident and Emergency departments. They record patients’ vital signs very regularly, but most of the time no action is taken if, for example, blood pressure is noted to be very high or a Cushing’s effect is evident from a low heart rate and high blood pressure in a head injury case. This is because ISO certification demands them to provide proof that the patient is being monitored properly and that all the data are recorded regularly. ISO does not ask about the actions taken in response to monitoring, and hence quality of actual patient care is not assessed properly. ISO also does not assess the competency of doctors and other health care personnel in terms of problem-solving, patient care, and management. The best way to assess quality of medical education may be evaluation of results of all the quality improvement procedures in terms of patient outcomes and recovery.

TQM models hence should be associated with some kind of evaluation to establish whether these quality assurance activities actually reflect improved quality in terms of teaching and learning, which is the main concern in an institution of higher education. Evaluation is a process of systematic collection, analysis, and reporting of information about an audience’s knowledge, attitudes, skills, intentions, and/or behaviors regarding specific content, issues or experiences, for the purpose of making informed decisions about programming.¹⁰ Several evaluation methods are available, but no method is perfect because they all have advantages and disadvantages. The best solution is to choose the method or methods that are the easiest and the least expensive ways to provide required answers. Data should be gathered in a systematic way, so that results can be reported scientifically instead of just in the form of impressions or anecdotes.¹⁰ Audit reports of teaching hospitals, on-job assessment of students, and academic results should be assessed, evaluated, and made accessible to stakeholders and the public to comment on the quality of the program. Audit reports should include details of outcomes of surgeries and of hospital admissions ie, a comprehensive patient morbidity and mortality record. This audit record will help in identifying the actual extent of morbidity and mortality in any given teaching hospital. ISO certification in the presence of significant morbidity and mortality may reflect the actual quality of the program, and may suggest that the service component is excellent but the educational component lacks quality and, by this audit finding, actual educational quality can be assessed easily. Students’ academic results should describe the details of student knowledge, attitude, and skills level at the end of each phase and competency at the end of program, which should be against some predefined standards. On-job assessment of students during internship should be made mandatory, both for the institution for annual certification and for students to obtain medical council registration. Review and evaluation of all teaching materials and activities in the institution, ie, problem-based learning, skills training, seminars, and lectures, should be conducted regularly by qualified medical educationists in order to establish the quality and appropriateness of teaching methods and materials.

Quality of a medical education institution can only be certified if the product of that institution is competent in terms of patient care and is able to demonstrate sound knowledge, skill, appropriate attitude, critical thinking, and problem-solving abilities. Assessing quality of the system and services as done by ISO may be helpful in improving services, but is not the sole indicator of overall educational quality. In other words, we can say that by applying industrial quality models, ie, ISO or TQM, in higher education, good technicians can be produced who are able to show their command of certain procedures and skills, but they may lack the critical thinking and problem-solving aspects of learning, and hence are not competent to make decisions on their own.

Educational models

A model of quality management for higher education should consider two types of processes, ie, the services to the student body, from academic (eg, enrolment, library) to general administrative functions (eg, cafeterias, recreation). ISO is an appropriate model for the service component. The second process should cover teaching and learning functions relating to both education and research. The literature proposes a number of models¹⁰ for academic quality management. Some of the popular models include:

• The transformational model, which is a learning-oriented approach to quality and emphasizes “enhancing participants”, “adding value” to their capability, and ultimately “empowering” them.

• The engagement theory of quality, which emphasizes student, faculty (academics), and administrative engagement in teaching and learning; this theory considers student learning as the primary purpose of higher education, highlights the role that academics, administrators and students play, and provides a template for assessing quality.

• The university of learning model approach to higher education, which is pedagogic, and suggests that in teaching, research, or community involvement, the core process is one of learning (at different levels); this model suggests that quality in a university context is based mainly on the quality of learning.

• The responsive university model, which is based on the assumption that quality of relationships with the public and quality of the outcomes is important in judging the quality of the university; therefore, universities will have to be responsive and service-oriented to survive and thrive.¹¹

These quality models highlight two issues that have received a common emphasis, ie, student learning and a dynamic collaboration around it. Overall, the features of a generic model addressing quality management in teaching and learning, based on the preliminary set of models chosen above, can be summarized as offering a clear focus on “transformation” of the learners and a synergistic collaboration at the learning interface.

There is still a debate about the status of students in terms of quality management. The situation of the higher education sector in recent years has changed from being universities to organizations, and hence it has been argued that students have become “consumers” of higher education services. Some senior academics still believe that higher education is not just another service industry. The government agencies try to consider students as customers, but academic staff do not support this notion.¹²

Both the industrial and educational models highlight visible commitment and support from the senior management to continue to develop effectively. Thus, the pattern of interaction and governance required for both the approaches is the same.¹¹

Some researchers suggest that the concept of TQM is applicable to academics. Teacher-student teams are considered to be the equivalent of front-line workers in industry. The student is the teacher’s customer, as the recipient of educational services provided for the student’s growth and improvement. Viewed in this way, the teacher and the school are suppliers of effective learning tools, environments, and systems to the student, who is the school’s primary customer. In another sense, the student is also a worker, whose product is essentially his or her own continuous improvement and personal growth. Success of TQM is the responsibility of top management. The school teachers must establish the context in which students can best achieve their potential through the continuous improvement that results from teachers and students working together. Teachers who emphasize content area literacy and principle-centered teaching provide the leadership, framework, and tools necessary for continuous improvement in the learning process.¹³

Students may be treated as customers in service areas, and may be the focus of all processes, but in the teaching and research context students are participants, and the focus should be on the attributes of their learning, as determined by the content and resources that govern the curriculum design, and the delivery and assessment that govern the “enhancement” of the learner.

An approach to implement a holistic model of quality in higher education may follow Senge’s “learning organization” model for implementing learning in organizations. According to Senge, learning organizations are “… organizations where people continually expand their capacity to create the results they truly desire, where new and expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people are continually learning to see the whole together”.¹⁴

Researchers also describe that the real work of learning takes place within a deep learning cycle which includes guiding ideas, theory, methods and tools, and innovations in infrastructure. These actions are helpful in developing the holistic model of quality.¹⁵

As described earlier, the university of learning model examines the organizational characteristics of higher education from a pedagogic perspective, and researchers suggest that a comparison of the models reveals a clear basis for developing a university of learning model through the use of five disciplines, ie, systems thinking, personal mastery, mental models, building shared vision, and team learning (http://www.infed.org/thinkers/senge.htm), as described by Senge.¹⁴ Integrated use of the two models is suggested as a basis for proposing a model for quality in higher education known as the learning university model.⁶

Educational quality models in medical education describe the characteristics and various components that should be implemented to ensure that quality learning is taking place. However, in order to evaluate quality assurance, we need to develop specific performance indicators for each of the components. The performance indicators for educational models may include:

• Evaluation reports of teaching materials and methods

• Student academic results for knowledge, skill, and affective domains

• Graduate job assessment reports

• Audit reports of teaching hospitals.

Conclusion

It can be concluded that industrial quality models in medical education may be used to assess the quality of academic support services, whereby customers can be identified easily and performance indicators may be used to assess quality. For educational quality measurement we need to develop a model that can assess the actual quality of teaching and learning and its impact on students in terms of patient care.

Disclosure

The author reports no conflict of interest in this work.