Abstract
Background: Many research articles on medical education have been published in Chinese-language journals, the majority within the past few years. However, there have been no objective studies to look at the quality of these, and their contribution to present day thinking.
Aim: This study explored the areas of focus, and the quality, of published research on Chinese undergraduate medical education.
Method: We searched three major Chinese databases, including Chinese Biology Medicine in electronic form, Chinese Journals Full-text Database, and Chinese Technological Periodicals Database, to trace the research themes and methodologies of Chinese-language research papers published from January 2000 to December 2008.
Results: The annual number of published articles on undergraduate medical education research has increased over time in China, with 70% of the articles in our 9-year study published in the past 4 years; the most popular theme was curriculum and teaching. Non-comparative studies accounted for the majority of the literature (84.6%); and comparative studies were rare.
Conclusions: Although an increase in the number of articles on medical education research in China is encouraging, more methodologically rigorous designs are needed to improve research quality. Generic and focused training on research methodology is essential to convert quantity into quality.
Introduction
The primary goal of medical education is to develop future doctors who can adapt to the conditions of medical practice in a rapidly changing health care environment and maximize healthcare quality (IIME Core Committee 2002; Chen et al. Citation2004). Knowledge has grown rapidly in the field of medicine; the profession of medicine is facing particular challenges as a result of rapid advances in biomedical sciences, information technology, and biotechnology (IIME Core Committee 2002). To meet these challenges, medical schools worldwide have undergone various medical education innovations to optimize their educational policies, curricula, teaching and learning methodologies, assessment, and organizational management. One result has been a flood of published medical education-oriented research. However, the quality of this research varies. It is expected that high-quality research findings will inform best practices and be utilized for policy decisions in education, and ultimately result in improved patient care (Bligh & Parsell Citation1999). This realization gave rise to “best evidence medical education (BEME)” standards (van der Vleuten et al. Citation2000), whose purpose is to enable “the implementation by teachers in their practice, of methods and approaches to education based on the best evidence available” (Harden et al. Citation1999), during when demand for evidence-based approaches and outcomes research in medical education increases (Hart & Harden Citation2000; Prystowsky & Bordage Citation2001; Chen et al. Citation2004; Coomarasamy & Khan Citation2004; Dauphinee & Wood-Dauphinee Citation2004). Some medical researchers predicted that the application of BEME could improve teaching methods, enhance educational and assessment levels, and provide more scientific strategies for medical education. Therefore, several systematic reviews on a wide range of subjects in medical and healthcare education have been published in the major journals dedicated to specific questions and subjects in medical education (Irby Citation1995; Bowen & Irby Citation2002; Reeves et al. Citation2002; Coomarasamy et al. Citation2003; Hebert et al. Citation2003; Coomarasamy & Khan Citation2004) to reflect the best evidence available.
Medical education research has been carried out for about 30 years in China. One result has been a flood of research papers published in Chinese-language journals. However, there has been no objective and quantitative evaluation of the quality of these researches so far; thus, it is hard to tell to what extend these researches can be used to direct health-related policy decisions.
To explore the areas of focus for Chinese undergraduate medical education and the quality of the resulting research papers, we searched three major Chinese databases for the research themes and methodologies of the medical education-related research papers published from January 2000 to December 2008 in Chinese-language journals. This is the first study to quantitatively analyze medical education research literature in China.
Materials and methods
Search strategies
We searched three major Chinese databases, including Chinese Biology Medicine in electronic form (CBM), Chinese Journals Full-text Database (CNKI), and the Chinese Technological Periodicals Database (VIP), for articles on medical education published between January 2000 and December 2008. CBM was searched by using subject indexing such as education, medicine, and undergraduate. CNKI was searched with an approximate-matching approach, using keywords of “medical education” and “undergraduate.” VIP was searched using keywords or subjects equivalent to “medical education” and “undergraduate.” We included studies that were primary or secondary research, were conducted in mainland China, and involved undergraduate medical students. Studies on nursing education and continuing medical education were not included.
Data extraction
Two authors independently extracted data according to the inclusion criteria and exclusion criteria. Discrepancies were resolved by consensus or by consulting a third reviewer. Search outcomes were as follows: 
Classification of data
We classified our included literature as follows, according to methods described in Tight's Higher Education Research and Development (Tight Citation2003) and BEME methodology, by the BEME Collaboration (BEME Citation2009).
Basic information: title, publication time, journal name, and author affiliation.
Research theme: policy and management, quality and assessment, curriculum and teaching, etc.
Study design: comparative study, non-comparative study, review, and systematic review.
Data analysis
EPIData 3.02 was used for data entry. SPSS11.5 and Microsoft Excel 2003 were used for data analysis. Basic information, research themes, and study designs were expressed as a percentage.
Results
General information
A total of 1842 articles met the criteria and were include in our study. Our results showed that the number of articles on medical education increased over time: 70.8% of the included articles were published after 2005 ().
Figure 1. Publication times of included 1842 works of literature published in Chinese-language journals from 2000 to 2008.
The included articles were published on 356 journals, 16 of which published more than 20 included articles each (), together accounting for almost half (864 articles, 46.9%) of the total number. The top five journals, which each published more than 50 included articles, were China Higher Medical Education, Northwest Medical Education China Higher Medical Education, Medical Education, Research in Medical Education, and Journal of Shanxi Medical University, together accounted for 28.77% of the total number. None of these five journals were indexed by Medline, SCI, or SSCI.
Figure 2. Chinese-language journals publishing more than 20 articles on medical education from 2000 to 2008. Notes: NME, Northwest Medical Education; CHM, China Higher Medical Education; ME, Medical Education; RME, Researches in Medical Education; JSMU, Journal of Shanxi Medical University; JYMCN, Journal of Youjiang Medical College for Nationalities; HVE, Health Vocational Education; ECM, Education of Chinese Medicine; M&P, Medicine and Philosophy; JXMU, Journal of Xinjiang Medical University; MM&H, Modern Medicine & Health; JNCCMC, Journal of North China Coal Medical College; JQMC, Journal of Qiqihar Medical College; JGTCMU, Journal of Guangxi Traditional Chinese Medical University; CJME, Chinese Journal of Medical Education; and M&S, Medicine and Society.
The included 1842 articles were written by authors affiliated with 341 institutions (data not shown). Among these, 59 institutions each published more than 10 articles (). All these 59 institutions are universities or higher education centers and publications from these institutions together accounted for 66.4% of the included articles. There were 8 institutions with more than 30 articles each, which together accounted for 19.6% of the included papers. All 8 institutions are medical universities.
Table 1. Affiliations of first authors publishing more than 10 articles in Chinese-language journals from 2000 to 2008
Research themes
A total of 850 articles (46.1%) on the theme of curriculum and teaching were published, followed in popularity by themes on quality and assessment (327 articles, 17.8%), and policy and management (299 articles, 16.2%; ). This trend remains the same over the years, with the curriculum and teaching theme (35.7–52.8%) predominating in medical educational literature.
Table 2. Theme classification, by publication time, of included 1842 works of literature published in Chinese-language journals between 2000 and 2008
Study designs
Non-comparative studies dominated the literature on medical education research, accounting for 77.5–87.6% of the amount over the observed years, while comparative studies accounted for only 9.9–18.8% (). No systematic reviews had been published.
Table 3. Study designs, by publication date, of 1842 included works of literature published in Chinese-language journals between 2000 and 2008
When study designs were compared based on research themes, we found that non-comparative studies dominated for all themes, particularly for the theme of policy and management (85.1%). Comparative studies were used in 19.6% of papers on quality and assessment, 13.7% on curriculum and teaching papers, and 11.4% on policy and management papers. There were a few non-systematic reviews: 10 papers on curriculum and teaching, 5 on policy and management, and 3 on quality and assessment ().
Table 4. Research themes and study designs of 1842 included works of literature published in Chinese-language journals between 2000 and 2008
Non-comparative studies
Since non-comparative studies dominated the included literature, we classified the non-comparative studies into six groups according to BEME methodology (BEME Citation2009) and Chinese literature searching methods: expert opinion, case report, case series, focus group, action-based, and narrative. One study could have more than one design. The most populated study method was expert opinion (). When comparing the percentage of these six study methods used in different research themes (), it was expert opinion.
Table 5. Use of six different non-comparative study methods over time of 1558 included works of literature published in Chinese-language journals between 2000 and 2008
Table 6. Study methods and research themes in non-comparative studies of 1558 included works of literature published in Chinese-language journals between 2000 and 2008
Control studies
Control studies were classified as cross-sectional, case-control, single group, before and after, time series, cohort, and experimental. Of the 254 control studies, 161 (63.4%) were cross-sectional studies; 30 (11.8%) used experimental designs, including 12 randomized control trials (RCT) and 18 non-randomized control trials; 15 (5.9%) were single group studies; 16 (6.3%) were cohort studies, 13 (5.1%) were time series; and 12 before and after studies (4.7%).
Discussion
In this study, we reviewed the literature on medical education published from January 2000 to December 2008 in Chinese-language journals. We hoped to provide insight into the current status of undergraduate medical education research in China and ways in which it can be advanced.
We found that the annual number of articles on undergraduate medical education research increased over time in China. The five leading journals that published more than 50 articles on medical education are Northwest Medical Education, China Higher Medical Education, Medical Education, Research in Medical Education, and Journal of Shanxi Medical University, accounting for 50% of the included literature. The majority (66.4%) of the articles were from medical education departments or centers, suggesting that research has become an important part of medical education in China. As in other countries, medical education in China has experienced many changes and challenges in the last few years. Chinese medical educators have also adopted innovative approaches to medical education, such as problem-based learning (PBL) (Cao et al. Citation2007; Shanley & Wang Citation2008; Wang & Zhang Citation2008), indicating that medical education is now incorporating its traditional role of knowledge spread into medical education research in China. An increase in the number of articles addressing medical education is encouraging; however, there still exist some problems, such as the low impact factor of the journals publishing these articles. In fact, none of the five leading medical education journals in China have been indexed by Medline, SCI, or SSCI. Although the quality of journals cannot always be accurately assessed by impact factor (European Association of Science Editors statement on impact factors Citation2009), studies show that impact factor may be used as a reasonable quality indicator for general medical journals (Tsay Citation1998; Saha et al. Citation2003). In fact, the most commonly used measure of journal quality in many circles is still the impact factor (Peleg & Shvartzman Citation2006).
Our study shows that the most popular theme for undergraduate medical education research in China was curriculum and teaching (46.1%), followed by quality and assessment, and policy and management. This trend accords with that found in other studies. Dimitroff et al. also found that the main concerns of medical education papers were curriculum planning (38.6%), teaching (21.2%), and student assessment (15.5%) (Dimitroff & Davis Citation1996). Similarly, Regehr (Citation2004) reported a substantial number of research articles involving the discussion and exploration of curriculum and teaching. This suggests that the key themes of medical education research are similar worldwide, which is not surprising, since medical education is an applied domain and curriculum and teaching is the core of medical education.
In our study, non-comparative studies accounted for the majority (84.6%) of the included literature and expert opinion was the most commonly used method. Comparative studies, and more importantly, RCT, which are considered the gold standard of outcome research, were rare in the assessment of educational interventions. Our findings were similar to those in other studies. Wan and his colleagues (Wan & Zhang Citation2006) also found that, although medical education research is shifting from opinion-based toward evidence-based approaches, qualitative studies still predominate in medical education literature in both China and in the other countries. In fact, 79.2% of the articles published in China Higher Education Research were qualitative studies during 2000–2004. One-fifth of the articles published in Academic Medicine in 2003 were qualitative studies (Shea et al. Citation2004). Although some experts argued that qualitative research is more appropriate in assessing the effectiveness of educational programs (Majumder Citation2004), and it is often not acceptable from an ethical point of view to randomly assign learners to receive or not to receive an intervention (Collins Citation2006), more rigorously designed methodologies have been generally encouraged in medical education research in recent years. Some medical education experts urged medical educators to produce high-quality education research by selecting and implementing the most appropriate methods (Collins Citation2006). They believe that ethical RCT designs should be used, wherever feasible and applicable, to assess the effectiveness of educational innovations and interventions (Bligh Citation2000; Torgerson Citation2002; Majumder Citation2004). According to a survey of literature on undergraduate medical education, the number of randomized control experimental studies similarly increased from one in 1969–1970 to 37 in 2006–2007 (Baernstein et al. Citation2007), indicating that there is a global tendency to use more rigorous methods in medical education research. However, we found in our study that although the annual number of published studies on undergraduate medical education interventions has been increasing in China, little change has been made in the methodological rigor of these studies. There is a need for improving medical education research methodology in China. However, being skillful at education research methodologies requires special training and experience (van der Vleuten et al. Citation2004). In developed countries, there are some international educational programs or associations, such as the Association for Medical Education in Europe (AMEE Citation2009), and the Association of American Medical Colleges (AAMC Citation2009), to support and provide training for those who are concerned with or interested in improving their educational research skills. By contrast, there is an inadequate exposure to medical educational research in undergraduate training in China and in other parts of the Asia (Majumder Citation2004), which might hinder the development of educational research, since choosing a methodology in educational research is as important as in any other type of research (Hutchinson Citation1999). In China, although institutions like the Center for Medical Education Research and Development was established with the support of the China Medical Board (CMB), and teaching methods and curricular reform were discussed during events like Medical Education Summit for the 8-year medical program in China, there has been no national center to take the lead in providing training on medical education research methodologies. In fact, medical education research in China is still rather embryonic, and no such training has been available to medical educators so far. For medical education research in China to grow and mature, it may be key to create a national center that would bring together health professions education research into a single national framework with an accessible database (Wartman Citation2004), and create opportunities for medical education researchers to meet and share ideas and resources. What's more, interdisciplinary and multi-institutional collaboration should be encouraged (Eva & Lingard Citation2008), as collaboration will enable researchers to accomplish more together than apart and to expand the impact of the research (Collins Citation2006).
Limitations
This study has several limitations. First, the classification of the research themes was not detailed enough to identify the most popular theme and the most active topics in medical education research in China. Next, we only extracted the affiliation of the first author for each article, and thus, did not examine the condition of collaboration among institutions and multidisciplinary authors, which might deserve much attention. Finally, we only explored the study designs of published literature, but we did not assess the overall quality of research being conducted.
Conclusion
In conclusion, our study found a clear increase in the number of articles on medical education research in China; however, there is a need for medical education research to use more methodologically rigorous designs and interdisciplinary, multi-institutional approaches to ensure research quality, as well as information dissemination and exchange. It is also necessary to conduct personnel training on medical education research methodology. The need for a national center for medical education research is urgent and compelling.
Acknowledgments
This project was supported by CMB of New York, nos. CMB-0082705502A01, CMB-00-721. The authors are grateful to Ms Shawna Williams for her close reading of an earlier version of this manuscript.
Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.
References
- AAMC 2009. Medical Education Research Certificate. Association of American Medical Colleges Web site. [Accessed 2009 October 7]. Available from: http://www.aamc.org/meded/start.htm
- AMEE. An International Association for Medical Education [Accessed 2009 December 27]. Available from: http://www.amee.org/index.asp?tm=59
- Baernstein A, Liss HK, Carney PA. Trends in study methods used in undergraduate medical education research, 1969–2007. JAMA 2007; 298: 1038–1045
- BEME 2009. Best Evidence Medical Education. [Accessed 2009 April 30]. Available from: http://www.bemecollaboration.org/beme/pages/start.html
- Bligh J. Medical teachers and evidences. Med Educ 2000; 34: 162–163
- Bligh J, Parsell G. Research in medical education: Finding its place. Med Educ 1999; 33: 162–163
- Bowen JL, Irby DM. Assessing quality and costs of education in the ambulatory setting: A review of the literature. Acad Med 2002; 77: 621–680
- Cao B, Cheng Zh, Cao DP. Role of PBL in reform of medical education. China Higher Med Educ 2007; 7: 1–2
- Chen FM, Bauchner H, Burstin H. A call for outcomes research in medical education. Acad Med 2004; 79: 955–960
- Collins J. Medical education research: Challenges and opportunities. Radiology 2006; 33: 639–647
- Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review. Br Med J 2004; 329: 1017–1019
- Coomarasamy A, Taylor R, Khan KS. A systematic review of postgraduate teaching in evidence-based medicine and critical appraisal. Med Teach 2003; 25: 77–81
- Dauphinee WD, Wood-Dauphinee S. The need for evidence in medical education: The development of best evidence medical education as an opportunity to inform, guide, and sustain medical education research. Acad Med 2004; 79: 925–930
- Dimitroff A, Davis WK. Content analysis of research in undergraduate medical education. Acad Med 1996; 71: 60–67
- European Association of Science Editors statement on impact factors [Accessed 2009 October 25]. Available from: http://www.ease.org.uk/statements/index.shtml
- Eva KW, Lingard L. What's next? A guiding question for educators engaged in educational research. Med Educ 2008; 42: 752–754
- Harden RM, Grant J, Buckley G, Hart IR. BEME guide no 1: Best evidence medical education. Med Teach 1999; 21(1)553–562
- Hart IR, Harden RM. Best evidence medical education (BEME): A plan for action. Med Teach 2000; 22(2)131–135
- Hebert RS, Levine RB, Smith CG, Wright SM. A systematic review of resident research curricula. Acad Med 2003; 78: 61–68
- Hutchinson L. Evaluating and researching the effectiveness of educational interventions. Br Med J 1999; 318: 1267–1269
- IIME Core Committee. Global minimum essential requirements in medical education. Med Teach 2002; 24: 130–135
- Irby DM. Teaching and learning in ambulatory care settings: A thematic review of the literature. Acad Med 1995; 70: 898–931
- Majumder M. Issues and priorities of medical education research in Asia. Ann Acad Med Singap 2004; 33: 264–266
- Peleg R, Shvartzman P. Where should family medicine papers be published - following the impact factor?. J Am Board Fam Med 2006; 19: 633–636
- Prystowsky JB, Bordage G. An outcomes research perspective on medical education: The predominance of trainee assessment and satisfaction. Med Educ 2001; 35: 331–336
- Reeves S, Koppel I, Barr H, Freeth D, Hammick M. Twelve tips for undertaking a systematic review. Med Teach 2002; 24: 358–363
- Regehr G. Trends in medical education research. Acad Med 2004; 79: 939–947
- Saha S, Saint S, Christakis DA. Impact factor: A valid measure of journal quality?. J Med Libr Assoc 2003; 91: 42–46
- Shanley P, Wang Q. Leaving problem-based learning behind: New assumptions and a revised model for case studies in preclinical medical education. Fudan Educ Forum 2008; 6: 87–91
- Shea JA, Arnold L, Mann KV. A RIME perspective on the quality and relevance of current and future medical education research. Acad Med 2004; 79: 931–938
- Tight M. Researching higher education. Open University Press, Maidenhead 2003
- Torgerson JT. Educational research and randomized trials. Med Educ 2002; 36: 1002–1003
- Tsay MY. The relationship between journal use in a medical library and citation use. Bull Assoc Med Libr 1998; 86: 31–39
- Van der Vleuten CP, Dolmans DH, De Grave WS, Van Lujijik SJ, Muijtijens AM, Scherpbier AJ, Schuwirthet LW, Wolfhagen IH. Education research at the Faculty of Medicine, University of Maastricht: Fostering the interrelationship between professional and education practice. Acad Med 2004; 79: 990–996
- Van der Vleuten CP, Dolmans DH, Scherpbier A. The need for evidence in education. Med Teach 2000; 22: 246–250
- Wan XH, Zhang XL. Evidence-based research of medical education. Chin J Evid Based Med 2006; 6: 318–320
- Wang AP, Zhang XCh. Reliability and validity of PBL teaching appraisal questionnaire. Chin J Mod Med 2008; 11: 1625–1629
- Wartman SA. Revisiting the idea of a national center for health professions education research. Acad Med 2004; 79: 910–917