2,260
Views
19
CrossRef citations to date
0
Altmetric
Research Article

Use of individualized learning plans among fourth-year sub-interns in pediatrics and internal medicine

, , &
Pages e46-e51 | Published online: 17 Jan 2012

Abstract

Background: Individualized Learning Plans (ILPs) are an effective tool for promoting self-directed learning among residents. However, no literature details ILP use among medical students.

Methods: Fifty fourth-year sub-interns in pediatrics and internal medicine created ILPs, including a self-assessment of strengths and weaknesses based on ACGME core competencies and the setting of learning objectives. During weekly follow-up meetings with faculty mentors and peers, students discussed challenges and revised goals. Upon completion of the rotation, students completed a survey of Likert-scale questions addressing satisfaction with and perceived utility of ILP components.

Results: Students most often self-identified strengths in the areas of Professionalism and Interpersonal and Communication Skills and weaknesses in Patient Care and Systems-Based Practice. Eighty-two percent set at least one learning objective in an identified area of weakness. Students expressed high confidence in their abilities to create achievable learning objectives and to generate strategies to meet those objectives. Students agreed that discussions during group meetings were meaningful, and they identified the setting learning objectives and weekly meetings as the most important elements of the exercise.

Conclusions: Fourth-year sub-interns reported that ILPs helped them to accomplish rotation goals, with the setting of learning objectives and weekly discussions being the most useful elements.

Introduction

A commitment to ongoing learning and self-improvement is now widely recognized as an essential element of medical professionalism. The Physician Charter on Medical Professionalism in the New Millennium, published in 2002 and subsequently endorsed by over 120 additional professional organizations, asserts that maintenance of professional competence through continuing self-assessment and learning is the responsibility not only of individual physicians but also of “the profession as a whole [… to …] ensure that appropriate mechanisms are available for physicians to accomplish this goal” (Blank et al. Citation2003). Furthermore, in its published statement of principles on professionalism, the American Academy of Pediatrics specifically lists self-improvement and self-awareness as two of the competencies “most appropriate for teaching and evaluation” (Fallat & Glover Citation2007)

  • In recognition of the importance of such self-directed learning, the use of individualized learning plans (ILPs) is now required of all pediatric residents under the Review Committee for Pediatrics of the Accreditation Council for Graduate Medical Education (see Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Pediatrics. Retrieved December 1, 2010. http://www.acgme.org/acWebsite/RRC_320/320_prIndex.asp; American Academy of Pediatrics. PediaLink Learning Center. Retrieved December 1, 2010. https://www.pedialink.org/index.cfm). ILPs provide a formal, program-supported mechanism by which residents self-assess, set specific process- or content-based learning objectives with corresponding plans or strategies to achieve them, and document and reassess their progress under faculty mentorship (Li & Burke Citation2010).

An emerging body of literature has begun to describe resident and faculty experiences with ILPs and to identify implementation strategies that may increase reported effectiveness of and resident satisfaction with such programs (Stuart et al. Citation2005; Li et al. Citation2009, Citation2010a, Citationb). However, to our knowledge, only one previously published report from 1991 describes medical student goal setting for a clinical rotation, and the vast majority of these students opted to select goals from a preset list instead of writing them independently (Schwiebert et al. Citation1991). Expanding upon preliminary experience with a pilot study on the use of ILPs among fourth-year pediatric sub-interns at our institution in 2009, we implemented a formalized ILP program for fourth-year sub-interns in both internal medicine and pediatrics in 2010. In this article, we detail the implementation of our program, describe trends in learning objectives generated by the students, and report data from student surveys regarding satisfaction with and perceived utility of the ILPs.

Methods

Program implementation

From July to October 2010, 23 fourth-year sub-interns in internal medicine (IM) and 27 in pediatrics created ILPs to direct learning across their one-month rotation. Participation in the exercise was voluntary and not linked to their course grade or evaluation. During the first week of the rotation, sub-interns met as a group with their faculty mentor (A.E.F. for pediatrics; E.A.S. for internal medicine) to receive instruction on ILPs and effective goal setting, complete a self-assessment of strengths and weaknesses modeled after skills and knowledge included in ACGME core competencies, define career goals, and set two initial learning objectives along with specific plans to achieve them. The faculty mentors advised students to generate learning objectives that would be specific, measureable, and achievable within the one-month rotation. During weekly group follow-up meetings, students documented self-reported progress toward the past week's objectives. Based on their perceived progress and evolving learning needs, students were given the option to continue working on the same objectives for the following week, to modify previous objectives or plans, or to create new objectives. Students discussed both successes and obstacles encountered, and fellow students and faculty mentors provided feedback and suggested new strategies for improvement. Students were encouraged but not required to share their learning objectives with attending faculty physicians or residents.

Data collection

Upon completion of the rotation, students were asked to complete a survey that consisted of (1) nine questions addressing their satisfaction with elements of the ILPs based on a 5-point Likert Scale, (2) a ranking from 1-to-5 (most-to-least useful) of five aspects of the exercise (attending weekly meetings, setting learning objectives, performing self-assessment of strengths/weaknesses, defining career goals, and exposure to ACGME core competencies), (3) a question about the optimum frequency of follow-up meetings, and (4) free response areas for listing the most and least useful elements of the project and suggestions for improvement.

The study design and protocol were approved by the Vanderbilt University Institutional Review Board.

Data analysis

The initial self-assessment of strengths and weaknesses consisted of 29 items divided into six categories corresponding to the ACGME core competencies (Patient Care, Medical Knowledge, Practice-Based Learning and Improvement, Interpersonal and Communication Skills, Professionalism, and Systems-Based Practice). For each item, students rated themselves on a line representing a continuum, with one end marked as “weakness” and the opposite end “strength.” Authors measured the position of each student response across the lines, scaled such that 0 indicated a complete weakness and 100 indicated a complete strength. Item responses were grouped into the six categories and reported as interquartile ranges and means ± standard deviation, with summary scores for each category compared between pediatric and IM groups using the Wilcoxon test.

Learning objectives were recorded weekly and sorted independently by each author into the most relevant corresponding category of ACGME core competency. Discrepancies in author assignments of objectives into specific categories were resolved by consensus discussion. The distribution of learning objectives across each category was calculated for students in the pediatric and IM groups separately and for all students combined. The percentage of students who set at least one learning objective in a category that contained one of their three lowest item scores from their initial self-assessment was also calculated.

Reponses to post-rotation survey questions were recorded as percentage distributions on a 5-point Likert scale. The distribution of student rankings of 5 individual elements of the ILP exercise from most-to-least useful was also recorded. Differences between pediatric and IM groups were examined using a proportional odds likelihood ratio test.

Results

Student self-assessment and setting of learning objectives

100% of sub-interns in IM (n = 23) and pediatrics (n = 27) between July and October 2010 volunteered to participate in the study and completed the initial self-assessment of strengths and weaknesses. All students set two initial learning objectives.

In the self-assessment, students in both the IM and pediatric groups rated themselves highest in the categories of Professionalism and Interpersonal and Communication Skills and weakest in Patient Care and Systems-Based Practice (). There were no significant differences between groups in self-ratings for any category.

Table 1.  Student self-ratings of strengths and weaknesses by category.

The highest percentages of total learning objectives generated were in the categories of patient care (45% IM; 44% pediatrics; 45% combined) and medical knowledge (20% IM; 23% pediatrics; 22% combined; ). The lowest percentages of total learning objectives were in professionalism (0% for IM and pediatrics) and systems-based practice (9% IM; 3% pediatrics; 4% combined). Overall, 72% of students (78% in IM; 67% in pediatrics) created at least one initial learning objective in a category that corresponded to at least one of their top three self-identified weaknesses, and 82% (82% in internal medicine; 81% in pediatrics) created at least one learning objective corresponding to an area of weakness at any point during the project.

Figure 1. Distribution of student learning objectives.

Figure 1. Distribution of student learning objectives.

Student evaluation of ILP project

Eighty-two percent (19/23) of sub-interns in IM and 78% (21/27) in pediatrics completed a post-rotation survey to evaluate the use of ILPs during their sub-internship (). Both IM and pediatric students expressed high confidence in their abilities to create specific and achievable learning objectives and to generate useful strategies to meet those objectives. Both groups also agreed that discussions arising during weekly group meetings meaningfully contributed to their learning experience. Students were less likely to agree that exposure to ACGME core competencies as part of the initial self-assessment of strengths/weaknesses and that definition of career goals in order to provide context to the ILP development process were useful. Correspondingly, students ranked the setting of learning objectives and weekly meetings as the two most important elements of the project, with definition of career goals and exposure to ACGME competencies least important ().

Figure 2. Elements of ILP exercise that students ranked as most useful.

Figure 2. Elements of ILP exercise that students ranked as most useful.

Table 2.  Post-Rotation Student Evaluation of ILP Exercise.

The only significant between-group difference in response to the Likert Scale questions was that pediatric sub-interns were more likely than IM sub-interns to believe that weekly reassessment of learning goals and strategies proved useful (p = 0.008). Similarly, 95% (20/21) of pediatric students agreed with weekly meetings, with only 5% (1/21) preferring to meet less frequently. In contrast, 63% (12/19) of IM students thought that follow-up meetings should occur weekly, and 32% (6/19) believed that they should take place less frequently.

Overall, students agreed that the ILP exercise as a whole helped them to accomplish more during their rotation.

Discussion

We report the first comprehensive description of an ILP program for medical students in which they were required to independently develop learning objectives with corresponding strategies for achievement, iteratively review progress, and revise weekly objectives and plans accordingly. Fourth-year sub-interns in both IM and pediatrics reported that ILPs helped them to accomplish more during their rotation, with the setting of learning objectives and strategies being the most useful element of the exercise.

Students also expressed high degrees of confidence in their abilities to create useful objectives and strategies, in contrast to reports of early resident experience with ILPs that have suggested learners often struggle in these areas (Stuart et al. Citation2005; Li et al. Citation2009, 2010). Our pilot data from last year also showed generation of plans to be an area of difficulty, so we focused advising efforts to help students create concrete and specific learning objectives that would be realistically achievable within the one-month period of their sub-internship. This improvement is in accordance with I-SMART criteria proposed by Li et al., which suggests that the most successful learners set objectives that were important, specific (with broader goals subdivided into smaller steps and with specific strategies to progress at each point), measureable, accountable (having systems to track progress with both internal and external checks on progress), realistic (achievable given time and resource constraints and including a willingness to modify as necessary), and linked to a timeline for completion (Li et al. Citation2010).

Overall, most students also found the weekly meetings and discussions with peers and faculty mentors to be useful. Although such meetings may be logistically impractical for residents, especially in light of tightening work hour restrictions, the extra direction and contact with faculty mentors and peers may be particularly useful for 4th-year medical students, who often have less oversight and less of a defined curriculum than their 3rd-year peers. A larger minority of internal medicine sub-interns (32%) than pediatric sub-interns (5%) felt that weekly meetings were too frequent. This discrepancy between groups may be secondary to differences in facilitation style (as different faculty mentors led each group), the type of students drawn to internal medicine versus pediatrics, the nature of the sub-internship specialties themselves, or other unknown factors.

While students felt that initial self-assessment of strengths and weaknesses and exposure to ACGME core competencies were less important aspects of ILP development, the distribution of strengths and weaknesses versus learning objectives across categories of competencies is informative. We anticipated that students would create learning objectives corresponding to areas of weakness. Overall, students felt they were strongest in the areas of professionalism and communication and weakest in patient care and systems-based practice. Accordingly, the greatest number of learning objectives was set in the area of patient care and the fewest in professionalism. However, the second-most common category of learning objectives was medical knowledge, and the second-least common was systems-based practice. This discordance could be explained by deficiencies in student self-assessment abilities or inadequate faculty guidance in directing learning objectives. Future iterations of ILPs may investigate whether students should be more explicitly advised or even required to develop learning objectives in areas of weakness, although 82% did generate at least one objective corresponding to an area of weakness without such a requirement. Furthermore, students may consider certain categories of competency (e.g., medical knowledge) to be more foundational at their given level of training and intentionally prioritize these areas before moving to working on other competencies (e.g., systems-based practice).

As our study is the first of its kind to describe ILP use among medical students and is limited in its sample size, more research will be required to further define characteristics of successful ILPs. A large question remains as to the optimal degree of student oversight and external feedback. Our students were not required to inform attending faculty members and residents of their ILP goals and strategies, and students did not receive formalized feedback as to their baseline strengths/weaknesses, progress in accomplishing objectives, or advice on future directions for continued learning. While this separation between the ILPs and clinical faculty was intentionally designed to minimize student anxiety about ILP performance affecting their clinical evaluations, it also may have detracted from learning opportunities. Pediatric residents and faculty have specifically identified accountability toward reaching goals as an important advantage of ILP development (Li et al. Citation2009), and residents and faculty would likely be willing to work with students to tailor clinical exposures or teaching to facilitate achievement of student goals if they were informed of them.

Furthermore, multiple studies have shown that physicians are consistently poor at self-assessment, with the lowest performers often demonstrating comparatively greater degrees of confidence, less awareness of deficiencies in skills or knowledge, and lower receptiveness to external feedback discordant with their self-perceptions (Gordon Citation1991; Davis et al. Citation2006; Violato & Lockyer Citation2006; Eva & Regehr Citation2008). Similarly, one study has suggested that medical students are more likely to base learning goals on self-perceptions of performance than on observer feedback (Eva et al. Citation2010). Lack of consistent, meaningful feedback from peers and supervisors has been implicated as a key contributor to this problem (Eva & Regehr Citation2005, Citation2008), and only with specific training in self-assessment and external input against which learners can compare self-evaluations have these abilities been shown to improve over time (Gordon Citation1991; Driessen et al. Citation2005, Citation2007; Srinivasan et al. Citation2007; Colthart et al. Citation2008). ILPs could provide an opportune forum to cultivate these skills in learners if self-evaluations of strengths, weaknesses, and weekly progress are compared to faculty or resident evaluations.

Additionally, no studies to date have attempted to create a functional measure of ILP quality, and our study was not powered to detect whether certain characteristics of ILPs, such as the number or type of learning objectives set, affect student satisfaction or perceived utility. The role of faculty mentorship also requires further investigation. Students perform significantly better in self-directed learning environments when under the guidance of mentors who have sufficient command of the subject matter to provide meaningful feedback and direction (Dolmans et al. Citation2002), and mentors are also important for helping students to set boundaries on the scope of self-directed inquiries (Eva & Regehr Citation2005). Accordingly, our faculty mentors helped students to set goals that were sufficiently challenging to push them to perform at their potential yet also realistically achievable. However, no formal curriculum for faculty development in ILP mentorship has been widely adopted or validated to date.

More robust, long-term outcome measures related to ILP use among medical student and residents are also needed. Early exposure to exercises in intentional, self-directed learning such as problem-based learning may promote increased commitment to continuing education and self-improvement in residency and practice (Shin et al. Citation1993; Schmidt et al. Citation2006; Neville Citation2009). Additionally, higher score on a propensity toward lifelong learning scale was an independent predictor of self-reported learning efficacy among pediatric residents engaged in ILP generation (Li et al. Citation2010). Such findings may suggest a need to cultivate such skills and attitudes in learners at an early point in training. However, whether ILPs or other tools that require self-reflection and goal setting in medical school translate to an increased propensity to continue such behaviors in the residency and practice remains unknown. Ultimately, the impact on patient care of any such training programs will be the most important test of efficacy.

Acknowledgments

The Office of Teaching and Learning in Medicine, Vanderbilt University School of Medicine. Harvard Macy Program for Educators in the Health Professions

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

References

  • Blank L, Kimball H, McDonald W, Merino J. Medical professionalism in the new millennium: A physician charter 15 months later. Ann Intern Med 2003; 138: 839–841
  • Colthart I, Bagnall G, Evans A, Allbutt H, Illing J, McKinstry B. The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no. 10. Med Teach 2008; 30: 124–145
  • Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA 2006; 296: 1094–1102
  • Dolmans DH, Gijselaers WH, Moust JH, de Grave WS, Wolfhagen IH, van der Vleuten CP. Trends in research on the tutor in problem-based learning: Conclusions and implications for educational practice and research. Med Teach 2002; 24: 173–180
  • Driessen EW, van Tartwijk J, Overeem K, Vermundt JD, van der Vleuten C. Conditions for successful reflective use of portfolios in undergraduate medical education. Med Educ 2005; 39: 1230–1235
  • Driessen E, van Tartwijk J, van der Vleuten C, Wass V. Portfolios in medical education: Why do they meet with mixed success? A systematic review. Med Educ 2007; 41: 1224–1233
  • Eva KW, Regehr G. “I’ll never play professional football” and other fallacies of self-assessment. J Contin Educ Health Prof 2008; 28: 14–19
  • Eva KW, Munoz J, Hanson MD, Walsh A, Wakefield J. Which factors, personal or external, most influence students’ generation of learning goals?. Acad Med 2010; 85: S102–S105
  • Eva KW, Regehr G. Self-assessment in the health professions: A reformulation and research agenda. Acad Med 2005; 80: S46–S54
  • Gordon MJ. A review of the validity and accuracy of self-assessments in health professions training. Acad Med 1991; 66: 762–769
  • Fallat ME, Glover J. Professionalism in pediatrics: Statement of principles. Pediatrics 2007; 120: 895–897
  • Li ST, Burke AE. Individualized learning plans: Basics and beyond. Acad Pediatr 2010; 10: 289–292
  • Li ST, Favreau MA, West DC. Pediatric resident and faculty attitudes toward self-assessment and self-directed learning: A cross-sectional study. BMC Med Educ 2009; 9: 16
  • Li ST, Paterniti DA, Co JPT, West DC. Successful self-directed lifelong learning in medicine: A conceptual model derived from qualitative analysis of a national survey of pediatric residents. Acad Med 2010; 85: 1229–1236
  • Li ST, Tancredi DJ, Co JPT, West DC. Factors associated with successful self-directed learning using individualized learning plans during pediatric residency. Acad Pediatr 2010; 10: 124–130
  • Medical professionalism in the new millennium: A physician charter. Ann Intern Med 2002; 136: 243-246
  • Medical professionalism in the new millennium: A physician's charter. Lancet 2002; 359: 520-522
  • Neville AJ. Problem-based learning and medical education forty years on. A review of its effects on knowledge and clinical performance. Med Princ Pract 2009; 18: 1–9
  • Schmidt HG, Vermeulen L, van der Molen HT. Longterm effects of problem-based learning: A comparison of competencies acquired by graduates of a problem-based and a conventional medical school. Med Educ 2006; 40: 562–567
  • Schwiebert LP, Crandall SJS, Brown DS. Incorporating goal setting into a third-year family medicine clerkship: A pilot project. Teach Learn Med 1991; 3: 239–244
  • Shin JH, Haynes R, Johnston ME. Effect of problem-based, self-directed undergraduate education on life-long learning. CMAJ 1993; 148: 969–976
  • Srinivasan M, Hauer KE, Der-Martirosian C, Wilkes M, Gesundheit N. Does feedback matter? Practice-based learning for medical students after a multi-institutional clinical performance examination. Med Educ 2007; 41: 857–865
  • Stuart E, Sectish TC, Huffman LC. Are residents ready for self-directed learning? A pilot program of individualized learning plans in continuity clinic. Ambul Pediatr 2005; 5: 298–301
  • Violato C, Lockyer J. Self and peer assessment of pediatricians, psychiatrists, and medicine specialists: Implications for self-directed learning. Adv Health Sci Educ Theory Pract 2006; 11: 235–244

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.