891
Views
26
CrossRef citations to date
0
Altmetric
Web Paper

Improvement of residents' clinical competency after the introduction of new postgraduate medical education program in Japan

, , MD, MPH, DMSc, , , &
Pages e161-e169 | Published online: 03 Jul 2009

Abstract

Background: In 2004, the Japanese government reformed the monospecialty-oriented postgraduate medical education (PGME) program and introduced a new PGME program to develop the primary care skills of physicians by mandatory rotation through different clinical departments.

Aims: (1) to evaluate whether residents’ clinical competency improved after the introduction of the new PGME program and (2) to compare the change in competency between university and non-university residents.

Method: Surveys were conducted before and after the introduction of the new PGME, i.e. in 2004 (response rate 88%) and 2006 (88%). One in every five residents was sampled, and the study subjects were 2474 second-year residents (1762 university residents and 712 non-university residents) in 2004 who were about to complete the previous PGME program and 1166 second-year residents (487 and 679) in 2006 who were the first generation to complete the new PGME program. The clinical competency ratings were measured by the proportion of respondents reporting ‘experience’ in 24 clinical specialties and three items regarding the use of medical records and ‘confidence’ in 35 clinical skills and knowledge items in four areas (i.e basic medicine, allied areas, behavioural science and social medicine, and clinical research).

Results: Compared to the residents in 2004, the clinical experience and confidence level of the residents increased dramatically for almost all of the surveyed items in 2006, regardless of the type of teaching hospital (chi-square or Fisher's exact tests, P < 0.0001). The marked improvement in the clinical competency of university residents was noticeable, resulting in the disappearance of the historical trend for non-university residents to obtain significantly more clinical experience than university residents.

Conclusions: The new PGME program appears to have been successful at improving both the clinical experience and confidence levels of medical residents, especially at university hospitals.

Introduction

In Japan, the mandatory internship program for medical students was abolished in 1968 and instead, a 2-year postgraduate medical education (PGME) programme was introduced (Ministry of Health, Labour and Welfare Citation2006). Unlike undergraduate medical education in the United States (van der Vlugt & Harter Citation2002; Teo Citation2007), firsthand clinical practice by medical students has been limited in Japan until they complete a 6-year undergraduate program at a medical school and then pass the national board examination for physicians (Japanese Medical Practitioners Law Citation2003). Thus, the clinical aspects of the PGME are tremendously important for providing opportunities through which medical residents can learn and experience basic clinical skills and acquire the clinical knowledge they need (Teo Citation2007).

However, the previous PGME program had serious limitations. First, most medical graduates received their PGME in a single department at a university hospital, which focused the training in specific areas of clinical practice (Otaki Citation1989). This created wide differences in the levels of clinical knowledge and skill attainment among residents (Teo Citation2007). Second, the contents of the programme and training methods were not standardized (Onishi & Yoshida Citation2004). Consequently, clinical competency of participants in the PGME programme was not evaluated properly. Third, the residents’ salary was very poor, which forced them to work at second jobs to make ends meet (Inoue & Matsumoto Citation2004). As a result, the majority of residents were worn out and could not concentrate on their training (Ministry of Health, Labour, and Welfare Citation2006).

To overcome these limitations, the Japanese government reformed the PGME system in 2004 and introduced a new mandatory PGME programme (). Newly certified physicians must train at teaching hospitals acknowledged by the Ministry of Health, Labour, and Welfare (the Ministry), namely, university hospitals or non-university hospitals. The new programme was designed to provide various clinical opportunities that allow residents to obtain primary care skills and knowledge, and improve residents’ working conditions, including salaries and working environments (Onishi & Yoshida Citation2004; Kozu Citation2006; Teo Citation2007).

Table I.  Before and after the new postgraduate medical education in Japan

In this comparative study, we evaluated whether the new PGME programme has improved the clinical experience and confidence levels of residents. We assessed residents’ clinical competency before and after the implementation of the new PGME programme. Furthermore, because our previous survey reported that university residents were less experienced and confident in terms of many clinical skills and knowledge than non-university residents (Yano et al. Citation1992), we compared the change in the clinical competency between university and non-university residents.

Methods

Questionnaire

The residents completed self-administered questionnaires that were similar to those used by Yano et al. (Citation1992). The questionnaires consisted of two parts that examined the residents’ clinical experience and clinical confidence. The experience part comprised 24 questions regarding various clinical specialties and three questions pertaining to experience with the use of medical records (i.e. death certificates, clinical pathology conference (CPC) reports, and referral forms; ). The confidence part comprised 35 clinical skills and knowledge items in four major areas: basic medicine, allied areas, areas related to behavioural science and social medicine, and areas related to clinical research.

Table II.  Residents’ “no experience” of clinical skills and knowledge before (academic year 2003) and after the new PGME (academic year 2005)

The responses were each measured using a four-point Likert scale. In the experience part, residents were asked to indicate how many times they experienced different categories of clinical practice during their 2-year residency program (a, 0; b, 1–5; c, 6–10; or d, 11 + times; ). In the confidence part, for questions regarding clinical knowledge, residents answered on the basis of confidence (a, very confident; b, almost confident; c, not very confident; d, not confident at all), and for questions regarding clinical skills, they answered on the basis of performance (a, able to perform independently; b, may be able to perform independently; c, cannot perform independently; d, cannot perform at all).

Study subjects

This comparative study was a part of scientific research efforts called ‘Evaluation of the New PGME Programme’ supported by the ministry that was conducted in March 2004 and February 2006. This study was approved by the ethics committee and all participants provided informed consent based on their understanding of the survey before taking part in the study. In Japan, the academic calendar begins on April 1 and ends on March 31 of the following year. Thus, the study subjects were second-year residents in the academic year 2003 who were about to complete a 2-year residency programme just before implementation of the new PGME programme and second-year residents in the academic year 2005 who were the first generation to complete the new PGME programme. The study involved 763 teaching hospitals (123 university and 640 non-university hospitals) in academic year 2003 and 849 hospitals (104 university and 745 non-university hospitals) in academic year 2005 that were acknowledged as teaching hospitals by the Ministry (Segami Citation2003; Fukui Citation2006). In collaboration with the Ministry, we sent a questionnaire to the director of the PGME programme at each hospital requesting the first resident from every up to five in the registration list to answer the questionnaire. When fewer than five residents were in a programme, the first resident in the list was asked to answer.

Data analyses

The clinical experience and confidence levels of residents were assessed by the proportion of ‘no experience’ answers to the three other levels of experience on the 4-point frequency scale (i.e. a vs. b + c + d) and the proportion of ‘confident’ answers to the two other levels on the 4-point Likert scale (i.e. a + b vs. c + d). Statistical analyses were performed by either a chi-square or Fisher's exact test, depending on the frequency number. Comparisons of the levels of clinical experience and confidence between the ‘before’ and ‘after’ the new PGME programme groups were performed using the total sample population based on the two types of teaching hospitals (i.e. university and non-university hospitals) and between teaching hospitals in both the before and after groups. SAS version 8.12 for Windows was used for the analyses, and P < 0.05 was considered statistically significant.

Results

The response rate from the teaching hospitals in 2003 was 73% for university hospitals and 58% for non-university hospitals, whereas the respective rates for 2005 were 88% and 88%. Consequently, 2474 second-year residents (1762 residents at university hospitals and 712 residents at non-university hospitals) in the 2003 academic year and 1166 second-year residents (487 residents at university hospitals and 679 residents at non-university hospitals) in the 2005 academic year were sampled.

The comparison of clinical experience between before (i.e. academic year 2003) and after (academic year 2005) PGME reform is presented in . Compared to the before group, the proportion with no experience declined dramatically in all specialty areas among all the residents after implementation of the new PGME programme (all P < 0.0001, data not shown). When the data were divided on the basis of the two types of teaching hospitals, at least 40% of university residents answered that they had no experience in each of 10 specialty areas (i.e. infection, psychiatry, ophthalmology, orthopaedics, urology, emergency medicine, surgery, gynaecology, paediatrics, and CPC reports) in academic year 2003. In contrast, residents were more likely to have experience in these areas in academic year 2005. Except for ophthalmology and CPC reports, non-university residents had a much greater opportunity for obtaining experience in these surveyed areas already before the introduction of the new PGME.

The confidence levels of residents about their clinical skills and knowledge before and after the introduction of the new PGME programme are presented in . The confidence levels of residents improved for all question items (all P < 0.01, data not shown), except for ‘technical skills and interpretation of Medline searches’. Regardless of the type of teaching hospital, more than half the residents lacked confidence with their skills and knowledge in several areas in academic year 2003; additionally, university residents were not confident about debridement, collaboration with social workers, social welfare facilities, or diabetes patient education, while non-university residents were not confident about cross-matching for blood type. In academic year 2005, the confidence levels of residents with these items improved regardless of the type of teaching hospital, but at least 50% of the residents were still not confident with several items (namely, funduscopy and otoscopy, medical reimbursement systems, and interpretation of statistical analyses).

Table III.  Resident's confidence in clinical skills and knowledge before (academic year 2003) and after (academic year 2005) the new PGME

Marked improvement in clinical experience and the levels of confidence among university residents became noticeable after the implementation of the new PGME. They caught up with non-university residents, leading to the disappearance of the trend of non-university residents significantly obtaining more clinical experience than university residents (21 items in before vs 2 items in after in ‘no experience’ and 18 items vs. 5 items in ‘confidence’).

Discussion

This was the first nationwide study to evaluate the effectiveness of the new PGME programme enacted in Japan in 2004. Referring to the limitation of the old PGME we previously conducted in 1992 (Yano et al. Citation1992), the new PGME programme was primarily designed to improve primary care skills and knowledge of medical residents. We obtained the following three main results: after the introduction of the new PGME programme, residents were more likely to have various clinical opportunities, regardless of the type of teaching hospital; the overall confidence levels of residents greatly improved; and the improvement in clinical experience and confidence levels among university residents became more remarkable after the introduction of the new PGME. In other words, implementation of the new PGME enabled university residents to catch up with non-university residents in obtaining the basic clinical skills and knowledge needed by all physicians, resulting in a decreased difference in the residents’ clinical competence between the two types of teaching hospitals. Thus, it appears that regardless of the type of teaching hospital, the new PGME programme has achieved its original goal of improving the clinical skills and knowledge needed by all clinical physicians.

Although the overall proportion of residents reporting no experience in some specialty areas declined, almost half of the residents still lacked confidence about their clinical skills and knowledge in some areas following implementation of the new PGME programme. One reason for this may be that the clinical skills of the teaching staff are usually limited to their specialty areas because the teaching staff of the new PGME programme likely received a monospecialty program during their own residencies (Inoue & Matsumoto Citation2004; Teo Citation2007). Although the requirements for teaching staff regarding the ratio of physicians to residents and the periods of clinical experience are less strict compared to the situation before March 2004 (Yano et al. Citation1991), a methodology for evaluating teaching staff needs to be standardized. Another consideration is the variability of the PGME curriculum provided by each teaching hospital. It needs to be standardized while also covering wider areas of clinical practice, knowledge of the medical reimbursement system, social resource allocation, and skills in clinical research (Cave & Clandinin Citation2007).

One reason for the discrepancy in clinical competency of residents between the two types of hospitals may have been due to the characteristics of the hospitals. For example, in both the before and after groups, university residents were more confident with urinalysis, cross-matching, and funduscopy, which may be the areas that residents learn when caring for particular patients throughout the course of an illness such as in an inpatient setting. Non-university residents were more confident with debridement, lumbar puncture, tracheal intubation, digital rectal exam, and fracture, which may be the areas that residents learn when seeing various patients with assorted illnesses in an outpatient setting. At university hospitals in Japan, physicians with more experience have traditionally provided the care for outpatients, while residents have been in charge of the inpatients. In contrast, second-year residents at non-university hospitals may see outpatients and have more opportunities to gain clinical experience because fewer specialists and clinical departments exist at non-university hospitals than at university hospitals (Niino et al. Citation1989; Yano et al. Citation1992; Yamaoka et al. Citation1993).

The better opportunities for clinical experience and better working conditions, including salaries (the average first-year resident's annual income in 2003 was $36,917 USD for a non-university resident and $17,740 USD for a university resident) at non-university hospitals have attracted residents (Ministry of Health, Labour and Welfare Citation2006; Fukui Citation2006). Before 2004, when a residency was not mandatory, 80% of newly certified physicians started their residency at university hospitals. However, more than half of residents (i.e. 4266 residents at non-university hospitals and 3451 residents at university hospitals) began their residency at non-university hospitals in academic year 2005 after implementation of the new PGME programme (Ministry of Health, Labour and Welfare Citation2006). This shift of residents from university hospitals to non-university hospitals may explain the improvement in the outcomes as a whole. We therefore suggest that the evaluation of residency programmes must be standardized to distinguish the difference in outcomes between the two types of hospitals. It may also be necessary to clarify the educational roles of teaching hospitals (i.e. leaving the academic role to university hospitals and that of teaching clinical skills to non-university hospitals).

Some limitations of this study need to be discussed. First, the response rate was relatively high with regard to the number of hospitals, but was relatively small on a resident basis, especially for university residents in academic year 2005. The reason for this discrepancy may have been due to the lack of adequate publicity of the survey at university hospitals. Furthermore, we recruited the first resident from the registration list when a hospital had fewer than five residents. This sampling process might cause the different response rate according to the type of teaching hospital. However, so long as the two types of hospitals were analyzed separately, bias due to this process is unlikely. Second, both clinical experience and confidence level were self-reported and could differ from an assessment by the supervisor. If university hospitals set higher standards and senior professors were overcritical, residents at university hospitals may rate themselves lower than those in a less stringent environment. However, the magnitude of the potential bias may not be so large because the degree of improvement was much higher among university residents than among non-university residents without major change in supervisors. Third, as the medical environment continues to improve through the increasing availability of medical facilities and internet-accessible scientific research, the surveyed items, which were originally developed in 1992, need to be updated, especially in the areas related to the behavioural sciences, social medicine, and clinical research. Finally, this research is based on observations performed in only the 2003 and 2005 academic years and thus does not indicate the stability of the findings. However, two studies (Yano et al. Citation1992; Segami et al. 2002) that were conducted before the introduction of the new PGME both reported that the majority of residents were not confident in their clinical skills and knowledge. In addition, preliminary analyses conducted in 2005 and 2006 to investigate residents’ competency after the introduction of the new PGME showed quite similar results, indicating that resident competency had improved significantly. Therefore, we can claim a degree of stability.

In conclusion, a comparison was made between residents who completed their 2-year residency programmes under the new PGME programme and those who completed their 2-year programmes under the former PGME programme. Residents became more experienced and confident about most of their basic clinical skills and general knowledge after the introduction of the new PGME programme. Thus, the new PGME programme by mandatory rotation of various different departments seems to be fulfilling its purpose of improving primary care skills and knowledge of residents.

Acknowledgements

This study was supported in part by grant H17-Iryo-015 from the Ministry of Health, Labour and Welfare of Japan.

Additional information

Notes on contributors

Kyoko Nomura

KYOKO NOMURA, MD, MPH, PhD, is an assistant professor at Teikyo University School of Medicine.

Eiji Yano

EIJI YANO, MD, MPH, PhD, is a chairman and professor at Teikyo University School of Medicine.

Makoto Aoki

MAKOTO AOKI, MD, PhD, is a president of National Higashi-Saitama hospital.

Katsuhiko Kawaminami

KATSUHIKO KAWAMINAMI, MD, MPH, PhD is a chief of comparative policy research section at National Institute of Public Health.

Hiroyoshi Endo

HIROYOSHI ENDO, MD, MPH, PhD is a director for planning and coordination, National Institute of Public Health.

Tsuguya Fukui

TSUGUYA FUKUI, MD, MPH, PhD is a president of St. Lukes International Hospital.

References

  • Cave MT, Clandinin DJ. Revisiting the journal club. Med Teach 2007; 29: 365–370
  • Fukui T. Final report. Evaluation study of the new postgraduate medical education system. St. Luke's Hospital, TokyoJapan 2006
  • Inoue K, Matsumoto M. Japan's new postgraduate medical training system. Clin Teach 2004; 1: 38–40
  • Japanese Medical Practitioners Law. Revised ministerial ordinance for postgraduate medical education training of 2003. 2003, No. 16 (2), Stat.1
  • Kozu T. Medical education in Japan. Acad Med 2006; 81: 1069–1075
  • Niino N, Yano E, Yamakado M, Maeshiro M, Takaku H. Time study on the activity of residents. Jap J Public Health (Kosei kagaku) 1989; 36: 484–488
  • Onishi H, Yoshida I. Rapid change in Japanese medical education. Med Teach 2004; 26: 403–408
  • Otaki J. Considering primary care in Japan. Acad Med 1989; 73: 662–668
  • Segami K. Research about training teaching staff and residency program in postgraduate medical education system. 2003, Final report (Wako-city, Saitama: Dept. of Public Health Policy, National Institute of Public Health), March
  • Teo A. The current state of medical education in Japan: A system under reform. Med Educ 2007; 41: 302–308
  • Ministry of Health, Labour and Welfare. New postgraduate medical training system. 2006, Available online at: http://www.mhlw.go.jp/topics/bukyoku/isei/rinsyo/index.html (Accessed 18 February 2007) (in Japanese)
  • van der Vlugt TM, Harter PM. Teaching procedural skills to medical students: One institution's experience with an emergency procedures course. Ann Emerg Med 2002; 40: 41–49
  • Yamaoka K, Kobayashi Y, Yano E. Length of in-patient stay in teaching hospitals in Japan. Med Educ 1993; 27: 280–285
  • Yano E, Yamaoka K, Segami K, Ogawa T, Kobayashi Y, Sugita S, Niino N, Fukui T, Yamakado M, Nishizaki O. A study on the designation standard of teaching hospitals for postgraduate medical education in Japan. Byouin Kanri 1991; 28: 15–25, (in Japanese)
  • Yano E, Yamaoka K, Sugita S, Kobayashi Y, Niino N, Fukui T, Yamakado M, Nishizaki O, Ogata T, Segami K. Comparing postgraduate medical education at university and non-university hospitals in Japan. Acad Med 1992; 67: 54–58

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.