Canadian residents’ perceived manager training needs

Abstract

Background: Despite widespread endorsement for administrative training during residency, teaching and learning in this area remains intermittent and limited in most programmes.

Aim: To inform the development of a Manager Train-the-Trainer program for faculty, the Royal College of Physicians and Surgeons of Canada undertook a survey of perceived Manager training needs among postgraduate trainees.

Methods: A representative sample of Canadian specialty residents received a web-based questionnaire in 2009 assessing their perceived deficiencies in 13 Manager knowledge and 11 Manager skill domains, as determined by gap scores (GSs). GSs were defined as the difference between residents’ perceived current and desired level of knowledge or skill in selected Manager domains. Residents’ educational preferences for furthering their Manager knowledge and skills were also elicited.

Results: Among the 549 residents who were emailed the survey, 199 (36.2%) responded. Residents reported significant gaps in most knowledge and skills domains examined. Residents’ preferred educational methods for learning Manager knowledge and skills included workshops, web-based formats and interactive small groups.

Conclusion: The results of this national survey, highlighting significant perceived gaps in multiple Manager knowledge and skills domains, may inform the development of Manager curricula and faculty development activities to address deficiencies in training in this important area.

Introduction

In Canada, the Royal College of Physicians and Surgeons of Canada (RCPSC) has adopted a framework of seven core domains of competence for all specialists, called the “CanMEDS Roles” (Frank 2005). The CanMEDS framework includes the Roles of Medical Expert, Communicator, Collaborator, Health Advocate, Manager, Scholar and Professional. This framework, driving the standards of the educational mission of the RCPSC, has been incorporated into accreditation, in-training assessment, and examinations, and is systematically implemented with enhanced materials and faculty development in Canada as well as abroad (Frank & Danoff 2007; Ham & Dickinson 2008).

Within CanMEDS, the Role of the Physician as a Manager has been identified as one of the most difficult to integrate into postgraduate medical education (PGME). The CanMEDS guidelines state that “as managers, physicians function as integral parts of healthcare organisations, organize sustainable practices, allocate resources and contribute to the overall effectiveness of the healthcare system” (Frank 2005).

Similarly, in the United States, the Accreditation Council for Graduate Medical Education (ACGME) has recently launched a training framework of six competencies, including that of expertise in systems-based practice (ACGME 2009). In the United Kingdom, the Academy of Medical Royal Colleges and the NHS Institute for Innovation and Improvement have developed a Medical Leadership Competency Framework, to assist in the design of training curricula and physician personal and career development (NHS 2009). In Denmark, physicians are supported to take on leadership roles though mandatory postgraduate training, including a 10 days leadership course provided by the Danish regions and the National Board of Health (Ham & Dickinson 2008). The need for additional training in administration and health systems to effectively prepare physicians for practice management and positions of leadership is thus broadly endorsed (Alexander 1991; Bogdewic et al. 1997; Kiel 1999; Forbes et al. 2004; Somers et al. 2004; Frank 2005; ACGME 2009; NHS 2009; Stephenson 2009).

Table

Practice points

Canadian specialty residents have identified multiple gaps in Manager training domains. Identified needs in specific domains may be helpful in designing Manager curricula and faculty development activities to augment teaching and learning Manager competencies. Residents as early as their first year of training identify multiple unmet needs, suggesting it may be possible to introduce teaching manager competencies early during residency. Residents identify several preferred educational methods for learning Manager competencies, including workshops, web based materials and interactive small groups.

Despite the widespread endorsement, teaching and learning in this competency area remain intermittent and limited in most programmes. Innovative curricula and training opportunities have been described for a small number of specialty training programmes, including lectures, case scenarios, committee participation, administrative and programme assignments, and community rotations with administrative training (Alexander 1991; Iverson 1993; Cordes et al. 1996; Custalow et al. 2000; Paller et al. 2000; Babitch 2006; Gruver & Spahr 2006; Moore et al. 2006; Hemmer et al. 2007; Ham & Dickinson 2008; Stergiopoulos et al. 2009). While educators have outlined areas of knowledge central to the Physician Manager Role, these prescriptions have failed to define specific tasks and behaviors in which residents and medical students should be competent (Yedidia et al. 2000). Not surprisingly, both practicing physicians and physicians in training report deficiencies in their administrative and leadership training (Cantor et al. 1993; Cordes et al. 1996; Kiel 1999; Sinai & Hodges 1999; DeWitt et al. 2001; Sockalingam et al. 2007, 2008).

As part of its ongoing dissemination strategy, in 2007, the RCPSC began Role specific Train-the Trainer (TTT) programs to promote the implementation of CanMEDS competencies in Canadian residency programs. For some of the CanMEDS Roles, such as Communicator and Collaborator, it was recognized that there were specific methods and resources currently available to support training in these roles. The TTT programs made use of these “best practices” to inform their design where available. For the Manager Role, however, it was apparent from RCPSC accreditation surveys as well as a literature review that explicit training in this area was underdeveloped. In order to inform the design of the Manager TTT program a broad-based needs assessment was initiated by the two program co-chairs (Saleem Razack and Susan Lieff). This included a detailed review of some of the Royal College accreditation questionnaires with respect to deficiencies in Manager Role training, a Delphi consensus expert panel, an inventory of national resources for physician management as well as a survey of perceived abilities and attitudes of Canadian specialty residents with regard to manager role training. The results of this resident survey are reported in this article.

The survey attempted to answer the following questions: What Physician Manager knowledge and skill areas do residents, before any formal manager teaching, perceive to be important to acquire during their training? What are residents’ preferred learning methods for a Manager curriculum? In addition to informing the Manager TTT program, it was hoped that the survey findings might be helpful to PGME programs planning to introduce Manager curricula.

Methods

Sampling and recruitment

A questionnaire was administered to a stratified sample of specialty residents from all 17 Canadian Medical Schools. Of the 3744 specialty residents in the national RCPSC database, approximately 15% were invited to complete the anonymous survey questionnaire (N = 549). The 15% were randomly selected from their emails on file with the Royal College. Selection was stratified to elicit results based on demographics, including specialty distribution and university of training. An information/consent to participate letter with a link to the survey questionnaire was mailed electronically to residents in January 2009. A reminder email was sent to non-respondents 10 days later. Ethics approval was obtained from the University of Toronto's Research Ethics Board.

Survey questionnaire

The survey questionnaire was designed to examine residents’ perceived knowledge and skill with respect to specific CanMEDS Manager Role domains as well as resident perceptions of the importance of improving their knowledge or skills in these areas. The questionnaire was based upon a similar tool that had previously been used in a national survey of psychiatry residents, and was modified through an iterative process with Manager Role residency training leaders at McGill University and the University of Toronto to improve its relevance to any specialty trainee (Sockalingam et al. 2008).

The survey included a total of 13 distinct questions covering specific knowledge and 11 questions addressing specific skills as they pertain to the Manager Role competencies. Each knowledge or skill question was followed by specific examples to facilitate residents’ understanding of each knowledge or skill domain. For example, “Program Evaluation” was followed by the example of “evaluating how well a program met its objectives, choosing indicators, using results to inform decision-making.” It also included questions on preferred methods of learning, and participant demographics such as current level of training, age, gender, past experiences and education, and future clinical practice. A 4-point Likert scale was used to rate each knowledge or skill question. For each set of knowledge and skill questions residents were asked to rate their current ability as well as and the importance of furthering their knowledge or skills in the identified domains.

Statistical methods

All data were analyzed using SPSS version 17 software (Chicago, Illinois) employing descriptive statistics, paired student t tests, and multiple regression analyses. We determined residents’ gaps in specific knowledge (GS_k) and skill areas (GS_s) by subtracting the perceived current level of knowledge or skill from the perceived importance of furthering knowledge or skill in this specific area. A higher gap score (GS) suggests a larger difference between the residents’ current knowledge/skill in a specific area and the importance they place on furthering their knowledge or skill in this area.

Mean GSs in each knowledge or skill area were calculated and paired Student t-tests were employed to determine significant difference between all mean GSs. Total GSs were created for all GS_k and GS_s areas by summing individual knowledge and skill GSs and were used to determine predictor variables on multiple regression analysis. We hypothesized that the following five respondent characteristics might influence knowledge and skill GSs and residents’ self assessment: gender, advanced degree, past administrative experience, prior medical school education in administration and level of training. We completed two separate regression analyses using TGS_k and TGS_s as the dependent variable, and the five characteristic variables as independent variables. Statistical significance was determined at p < 0.05 for all data analyses.

Results

Demographic characteristics

Of the 549 residents contacted, 199 completed the survey, for a response rate of 36.2%, comparable to response rates obtained by the RCPSC on previous on-line member surveys. Over 81% (N = 162) of the participants were between 25 and 34 years of age and 39.3% (N = 78) were women. Chi-squared analyses were performed to determine the representativeness of the sample to the population. There were no significant differences in the distribution of gender, language, and age between the sample and the population from which it was drawn. However, the distribution of respondents’ postgraduate training level differed significantly with the sample, under representing the lower levels of postgraduate training and over representing the upper levels of postgraduate training. A comparative summary of the distribution of respondents’ and population demographics is detailed in Table 1.

Table 1.  Demographic characteristics of survey respondents

Demographic characteristics	Resident population in Canada		Survey respondents
Language	N = 3744		N = 199
	English	87.2%	English	83.9%
	French	12.8%	French	16.1%
Gender	N = 3744		N = 198
	Female	47.4%	Female	39.3%
	Male	52.6%	Male	60.6%
Age	N = 3744		N = 198
	<25	0.6%	<25	1.0%
	25–29*	33.2%	25–29	44.7%
	30–34	44.4%	30–34	36.7%
	35–39	13.6%	35–39	10.1%
	40+	8.1%	40+	7.5%
Year of training**	N = 3744		N = 198
	PGY1***	29.4%	PGY-1	11.1%
	PGY2	23.1%	PGY-2	20.7%
	PGY3	23.6%	PGY-3	23.2%
	PGY4	18.9%	PGY-4	25.8%
	PGY5***	4.2%	PGY-5	16.2%
	PGY6**	0.5%	PGY-6	2.5%
	PGY7	0.3%	PGY-7	0.5%

Notes: Significant differences in the demographics between Survey Respondents and the Resident Population in Canada have been identified by chi-squared analyses and are marked with an asterisk.

*p < 0.05, **p < 0.01, ***p < 0.001.

The majority of residents (61.3%, N = 122) envisioned an academic hospital based future practice, while 30.7% (N = 61) envisioned a non-academic hospital based practice and 8% (N = 16) an office based or rural practice. Approximately one-quarter of the residents had advanced degrees. Only 4.3% (N = 9) had a Master's in Business Administration, Master's of Health Sciences or Master's in Public Health. The most common manager related activities the residents had been involved in included participation in Resident Training Committees (N = 86), University Committees (N = 42) or being a Chief Resident (N = 60).

Perceived knowledge and skill levels

Tables 2 and 3summarize respondents’ perceptions of their knowledge and skill levels in selected areas, respectively, and the importance of furthering their knowledge or skill in these areas in decreasing order of perceived importance. Survey respondents reported the highest perceived knowledge levels in the areas of Canada's Health Care System and the Function of Regulatory Bodies. Over 55% (N = 101) of respondents reported sufficient or more than adequate knowledge in these domains. Respondents perceived themselves to be least knowledgeable in the areas of “Understanding a Financial Statement” and “Principles of Budgeting,” with approximately three quarters of respondents (N = 130) reporting limited or no knowledge in these domains. Respondents placed the greatest importance in improving their knowledge level in the areas of Physician Compensation and Health Law, with more than three quarters of respondents (N = 140) reporting these as “Quite Important” or “Essential.”

Table 2.  Resident perception of manager knowledge domains

	Perceived level of knowledge		Perceived importance
Knowledge Domain	n	mean score (SD)	n	mean score (SD)	Gap score (GSK)	Gap (%)
Physician Compensation	180	2.13 (0.62)	180	3.17 (0.75)	1.04*	26
Relevant Health Law	179	2.47 (0.68)	180	3.11 (0.80)	0.63*	16
Principles of Quality Improvement	180	2.49 (0.70)	180	2.94 (0.74)	0.44*	11
Principles of Budgeting	181	2.06 (0.75)	181	2.86 (0.79)	0.81*	20
Reading and Understanding a Financial Statement	180	2.06 (0.80)	181	2.78 (0.85)	0.73*	18
Program Evaluation	180	2.35 (0.69)	180	2.72 (0.73)	0.37*	9
Program Planning	180	2.11 (0.67)	179	2.69 (0.78)	0.58*	15
The Function of Regulatory Bodies	181	2.57 (0.63)	180	2.68 (0.72)	0.11	3
Models of Health Care Delivery	179	2.39 (0.60)	181	2.68 (0.76)	0.29*	7
Health Care Organization and Medical Staff Organizational Structures, Roles and Relationships	179	2.23 (0.59)	179	2.65 (0.74)	0.42*	11
Health Care Reform	182	2.27 (0.61)	180	2.64 (0.70)	0.37*	9
Canada's Health Care System	182	2.59 (0.59)	181	2.59 (0.75)	0.00	0
Role of Stakeholders in the Health Care System	180	2.41 (0.72)	180	2.48 (0.70)	0.07	2

Note: *p < 0.05 using a paired student t-test.

Table 3.  Resident perceptions of manager skill domains

	Perceived level of skill		Perceived importance
Skill domains	n	Mean score (SD)	n	Mean score (SD)	Gap score (GSK)	Gap (%)
Looking After Yourself and Your Career	176	2.39 (0.72)	176	3.41 (0.69)	1.02*	25.6
Time Management	176	2.88 (0.58)	175	3.28 (0.81)	0.40*	10.0
Working Effectively in Multidisciplinary Teams	175	3.26 (0.53)	173	3.12 (0.82)	−0.14	−3.5
Leadership in Clinical Practice	176	2.73 (0.64)	176	3.11 (0.73)	0.38*	9.4
Negotiating and Resolving Conflict at the Workplace	176	2.85 (0.59)	175	3.11 (0.79)	0.26*	6.5
Self-assessment and Reflective Practice Skills	175	2.80 (0.62)	176	3.10 (0.81)	0.30*	7.6
Information Technology appropriate to Health Care	176	2.81 (0.64)	176	3.05 (0.70)	0.24*	6.1
Mentorship	176	2.77 (0.60)	176	3.02 (0.74)	0.25*	6.2
Emotional Intelligence	175	3.03 (0.57)	175	2.99 (0.86)	−0.04	−1.0
Adopting Innovation and Leading Change in A Professional Team Setting	175	2.35 (0.59)	176	2.90 (0.71)	0.55*	13.8
Meeting Management	176	2.59 (0.66)	175	2.89 (0.79)	0.29*	7.4

Note: *p < 0.05 using a paired student t-test.

Undergraduate training was the source of knowledge in Physician Manager knowledge areas for 45.8% (N = 92) of respondents, with 23.4% (N = 47) having acquired most of their knowledge during medical school and 16.4% (N = 33) during residency training.

Respondents reported the highest perceived skill levels in the areas of Teamwork and Emotional Intelligence, with 96% (N = 168) of respondents reporting sufficient or more than adequate skills in Working in Multidisciplinary Teams and 85% (N = 149) reporting sufficient or more than adequate skills in the area of Emotional Intelligence. Respondents felt least skilled in the areas of Change Management and Personal and Professional Self Care, with over 60% (N = 108) of respondents reporting limited or no skills in these areas. Respondents placed the greatest importance on furthering their skill level in the areas of Time Management and Personal and Professional Self Care, with (81.7%, (N = 143) and 88.7% (N = 156) of respondents, respectively, identifying these domains as quite important or essential. Respondents reported that unlike knowledge domains, the main source of skill acquisition had been residency training (40.8%), followed by medical school training (31.5%) and undergraduate training (13.5%).

Knowledge and skill GSs

Survey respondents reported significant gaps in most of the Physician Manager knowledge areas, including: Health Law; Physician Compensation; Principles of Quality Improvement; Principles of Budgeting; Reading and Understanding a Financial Statement; Program Evaluation; Program Planning; Health Care Delivery Models; Organizational Structures; and Health Care Reform. Similarly, respondents identified significant gaps in most Physician Manager skill areas, including: Personal and Professional Self Care; Time Management; Leadership in Clinical Practice; Conflict Resolution and Negotiation; Self Assessment and Reflective Practice Skills; Information Technology in Health Care; Mentorship; Change Management; and Meeting Management. None of the variables entered into linear regression analysis, namely gender, advanced degree, past administrative experience, prior medical school education in administration and level of training predicted knowledge or skill GSs (Table 4).

Table 4.  Multiple linear regression analysis

	Predictor variables
	TGSK	P	TGSS	P
Constant	12.200		10.524
Sex	0.058	0.513	−0.043	0.636
Previous administrative education	−0.070	0.428	0.111	0.219
Level of training	−0.168	0.083	−0.092	0.351
Advanced degree	0.027	0.768	0.038	0.686
Age	0.051	0.602	0.060	0.556
F	0.847	0.519	0.601	0.699
Adjusted R^2	0.033		0.024

Notes: Sex-coded: male, 1; female, 0. Values for predictor variables expressed as standardized β.

Educational preferences

Respondents were invited to select all preferred methods of learning from a menu that included workshops, lectures, interactive small groups, web-based curricula, on-line discussion boards, mentorship and direct supervision. Respondents’ preferred leaning venues included workshops (24.1%), followed by web-based curricula (18.7%) and interactive small groups (17.1%). Least preferred learning methods included on-line discussion boards (2.6%) and direct supervision (5.4%). There were no significant differences in preferred learning methods between residents with advanced administrative training and those without.

Residents were also asked of additional Manager domains that might be important to include in training programs. Residents suggested training in contract negotiations, practice management, hiring/hiring office staff, as well as balancing the Role of Manager with that of the Health Advocate.

Discussion

Our study confirms that residents across Canada perceive training gaps in many Manager knowledge and skill domains. Specifically, residents reported significant gaps in several Manager knowledge areas, including: Health Law; Physician Compensation; Principles of Quality Improvement; Principles of Budgeting; Reading and Understanding a Financial Statement; Program Evaluation; Program Planning; Health Care Delivery Models; Organizational Structures; and Health Care Reform. Similarly, residents identified significant gaps in several Manager skill areas, including: Personal and Professional Self Care; Time Management; Leadership in Clinical Practice; Conflict Resolution and Negotiation; Self Assessment and Reflective Practice Skills; Information Technology in Health Care; Mentorship; Change Management; and Meeting Management. These specific domains may help inform novel Manager curricula and faculty development efforts across undergraduate and postgraduate training programs and define Manager-related tasks and behaviors in which medical students and residents should be competent. Given efforts underway in several countries to develop manager or leadership curricula for medical trainees, our findings may be particularly timely. In Denmark, mandatory training in medical leadership is offered to newly appointed specialists (Ham & Dickinson 2008). In Canada, the US and the UK there is increasing emphasis on integrating teaching and learning of managerial competencies throughout training with the development and implementation of competency frameworks (Frank 2005, ACGME 2009, NHS 2009). Incorporating learner perspectives into curricular planning may be a helpful approach.

Our findings both support and add to a growing literature on resident perceived Manager training needs. Our present survey of a representative sample of residents across specialty programs in Canada reveals that residents perceive gaps in most Manager domains for which their perceptions are elicited. This finding is not unexpected. It may reflect failure to address these areas in existing curricula or residents’ perceptions of the importance of furthering their mastery in these areas to successfully negotiate an increasingly complex healthcare environment. Unlike other trainees, residents have little formal contact with hospital managers, and no exposure to managerial thinking and practices.

In our study, neither gender, previous administrative experience, training level or advanced degree predicted total knowledge nor skills GSs, reflecting perhaps the difficulty in transferring knowledge or skills in a clinical setting, and the lack of opportunities to practice and develop such skills during training.

Residents identified undergraduate training as the main source of Manager related knowledge and resident training as the main source of Manager related skills acquisition. Residents as early as their first year of training find administrative domains relevant and challenging for their future practice, suggesting that managerial concepts could be introduced early during training to facilitate longitudinal reinforcement of learning and provide residents with a foundation with which to begin careers that include management duties.

Residents’ preferred leaning venues included workshops, followed by web-based curricula and interactive small groups. There was no significant difference regarding preferred learning methods between those with advanced administrative training and those without such training. Least preferred learning methods included on-line discussion boards and direct supervision. The preferred methods may reflect resident’ prior exposure to educational methods at their schools, thus limiting the usefulness of these findings. In particular, bedside teaching and direct supervision may be instrumental in teaching and learning Manager related competencies, as several of these competencies would be difficult to convey in a didactic setting and would likely require a problem based, experiential learning environment, as postulated in adult learning theories.

The diversity of methods preferred nonetheless is in keeping with previous findings (Sockalingam et al. 2008) and previous recommendations to draw form the growing body of available instructional materials and new web based educational programs in these domains (Griffith 2003).

Our study had several limitations. The response rate to our on-line survey was only 36.2%, potentially limiting generalizability of our findings. However, this rate compares favorably with those of other surveys completed by the RCPSC and our sample is representative of trainees in Canada. Although, first year postgraduate trainees are under-represented, compared to our national specialty resident cohort, level of training did not predict gaps in knowledge or skill domains. Another limitation is the nature of this survey, eliciting perceptions of residents with little or no exposure to formal teaching in Manager competencies. The results reflect perceived needs of the target audience, and may not necessarily reflect their educational gaps as identified by experts or clinical teachers (observed needs). Many of our residents, however, participated in training or other university committees or acted as chief residents, drawing from valuable experiences and perspectives.

Curriculum reform remains a challenge. As schools and programs undertake steps to reform and redesign their curricula, they will encounter several barriers to change (Halpern et al. 2001; Maggi et al. 2008). Barriers to change require strategic and operational planning and necessitate involving the highest level of leadership at each school to promote the new curricular components and provide needed support and direction. It has been suggested that it may be important to integrate this content into existing courses and bedside teaching rather than trying to carve out new time, given that resident time for educational opportunities is becoming increasingly scarce (Cox et al. 2004).

The RCPSC utilized findings of this survey to inform their Manager TTT program. It is hoped that findings will also help inform local curriculum development efforts such that the specific domains identified can be mapped into specific lessons and experiences at appropriate levels of training.

Acknowledgment

The authors wish to thank Wendy Jemmet for his administrative support in this work.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.