Validating objectives and training in Canadian paediatrics residency training programmes

Abstract

Background: Changing health care systems and learning environments with reduction in resident work hours raises the question: “Are we adequately training our paediatricians?”

Aims: (1) Identify clinical competencies to be acquired during paediatric residency training to enable graduates to practise as consultant paediatricians; (2) Identify gaps in preparedness during training and; (3) Review and validate competencies contained in the Royal College of Physicians and Surgeons of Canada (RCPSC) objectives of training (OTR) for paediatrics.

Methods: A questionnaire with 19 classification domains containing 92 clinical competencies was administered to RCPSC certified paediatricians who completed residency training in Canada from June 2004 to June 2008. For each competency, paediatricians were asked to indicate the importance and their degree of preparedness upon entering practice. Gap scores (GSs) between importance and preparedness were calculated.

Results: Response rate was 43% (187/435); 91.3% (84/92) of competencies in the RCPSC OTR were identified as important. Paediatricians felt less than adequately prepared for 25% (23/92) of competencies; 40 competencies had GSs >10%.

Conclusions: The unique approach used in this study is useful in validating OTR as well as the preparation of residents in relation to OTR. The results indicate a potential need for additional training in specific competencies.

Introduction

The fundamental objective of graduate medical education is to provide physicians with the knowledge and skills required for their future practice, ideally through supervised guidance, graduated responsibility and exposure to a broad range of medical conditions (Lesky 2007). Paediatric residency training in Canada is of 4 years duration and is designed to prepare paediatric consultants. The first 3 years are “core”. Training during the fourth year is variable in order to meet career goals of trainees. Over the past decade, roles played by paediatricians have changed and include: generalists (community-based paediatricians, hospital-based general paediatricians and academic general paediatricians), and subspecialists (The Canadian Association of Paediatric Health Centres 2008; Bannister et al. 2009). Changes in delivery of health care have included an increase in the number of children being managed as outpatients with in-patient care directed mainly to management of complex quaternary-level problems. This makes it difficult to offer comprehensive residency education solely in tertiary care hospitals. New directives on resident safety and wellness have limited resident work hours to approximately 65 h per week with no more than 24 h of continuous call. Mandatory “academic half-days”, longitudinal clinic “call back half-days” and mandatory post-call regulations have an impact on the time available for residents to adequately achieve the educational objectives of training (OTR). Residents, therefore, often comment that they are comfortable caring for acutely ill children in an inpatient setting (Lieberman & Hilliard 2006), but express discomfort managing ambulatory general paediatric problems (Veale et al. 1999; Ward et al. 2004; Chamberlain et al. 2005; Kolarik et al. 2006; Korczak et al. 2006; Narayan et al. 2006; Grant et al. 2007). These changes have also led to a continuing debate among educators across Canada as to whether the current 4-year residency training programmes adequately prepare paediatricians for their future careers.

In Canada, a group of experts [Royal College of Physicians and Surgeons of Canada (RCPSC) Specialty Committee in Paediatrics] determine the OTR, (Royal College of Physicians and Surgeons of Canada 2008a) and the specialty training requirements (STR), (RCPSC 2008b) for paediatrics. This group of experts is largely composed of physicians in academic practice who may not be fully aware of the needs of paediatricians practicing under different circumstances. Rarely do we validate their judgements. Recent studies in Canada (Lieberman & Hilliard 2006) and the US (Carraccio et al. 2004; Lesky 2007) have reinforced the need for regular programme evaluations, more standardized mechanisms for monitoring the training–practice gaps and the need for sufficient specific training to enable paediatricians to competently investigate and manage a variety of conditions. Thus, there is a need for ongoing monitoring of competencies acquired during residency training and preparedness for practice. A competence-based approach offers all medical specialties the chance to build their education programmes, on a practice-related model designed to improve performance (Dunn et al. 1985). In Canada, this would contribute to the validation and revision of the RCPSC OTR and STR in paediatrics to reflect what key competencies should be acquired during residency training. As consultant paediatric practice changes and as further changes occur in the epidemiology of serious paediatric illness (chronic diseases now more likely to cause mortality and morbidity), the competencies required of paediatricians will also constantly evolve (Accreditation Council for Graduate Medical Education 2007; Wise 2007). As the focus of training and competencies required for general paediatricians is different compared to paediatric subspecialists, this study focuses on preparation of general paediatricians for practice.

In this study, we hypothesized that graduates from June 2004 to June 2008 have attained competencies during their residency training in paediatrics in Canada that have enabled them to function as paediatric consultants. Our aims were to: (1) identify what core clinical competencies should be acquired during paediatric residency training to enable graduates to practice as consultant paediatricians during their first 5 years in Canada, (2) identify possible gaps in preparedness during training and (3) review and validate competencies contained in the RCPSC OTR for paediatrics.

Table

Practice points

Determination of training–practice gap scores allows assessment of competencies required to function as consultant paediatricians, and how residency training has prepared paediatricians for practice. Measuring the gap between importance and preparedness is very useful in validating objectives of training. This methodology could prove to be useful across medical specialties to review and validate their objectives of training, and to better define the competencies required for consultant practice. There is a need for ongoing monitoring of competencies acquired during residency training and preparedness for practice.

Methods

Survey instrument

The survey instrument used was a questionnaire. It was developed using previous studies, information obtained from community consulting paediatricians (verbal personal communication), CAPHC (Canadian Association of Paediatric Health Centres) data and Calgary Health Region data [inpatients at Alberta Children's Hospital (tertiary level) and Peter Lougheed Centre (community hospital), and patients seen in the emergency department at Alberta Children's Hospital] on the top 50 most commonly encountered paediatric problems. The questionnaire used the RCPSC OTR for paediatrics to focus on core clinical competencies (diagnosis, management and performing tasks/procedures). Using a modified Delphi method, it was pilot-tested among a total of 10 community consulting and subspecialty paediatricians for face validity. Core clinical competencies and tasks required to practice as consultant paediatricians in Canada were identified and a final questionnaire was developed. It contained 92 questions on clinical competencies with responses linked to a 5-point Likert-scale, and demographic questions. The 92 questions were grouped into 19 classification domains (Table 1). The demographic questions included: year of completion of residency training, number of years in practice, type of work and gender. The survey was designed to be completed in 15 min.

Table 1.  The Purpose of this questionnaire is to determine your opinion about what clinical competencies should be acquired during paediatric residency training so that a graduating trainee can practice as a consultant paediatrician

Conditions/disorders/disease/problems/tasks Q1. For paediatricians		Not important	Somewhat important	Important	Very important	Extremely important
Q2. For paediatricians		Not prepared	Somewhat prepared	Adequately prepared	Very well prepared	Extremely well prepared
Acute care	Obtaining intravenous access in infants, children and youth	1	2	3	4	5
	Gastroenteritis, dehydration and prescribing and managing fluids and electrolytes in all age groups	1	2	3	4	5
	Endotracheal intubation in newborn, children and youth	1	2	3	4	5
	Performing thoracentesis and chest tube placement	1	2	3	4	5
	Provision of cardiopulmonary resuscitation in multidisciplinary teams and participation in mock codes in newborn, children and youth	1	2	3	4	5
	Diagnosing and managing shock	1	2	3	4	5
	Determining if urgent consultation or transport is required for a sick infant/child/youth	1	2	3	4	5
	Applying ethical principles to provision or withholding of clinical care, patient confidentiality and informed consent	1	2	3	4	5
	Crisis resource management training (leadership, problem solving, situational awareness, communication skills and resource management) including communicating information and providing support in a crisis situation (e.g. unexpected death)	1	2	3	4	5
	Interpreting abdominal and chest X-rays and, interpreting skeletal and bone X-rays for fractures	1	2	3	4	5
GI and nutrition	Recurrent abdominal pain including constipation as a cause	1	2	3	4	5
	Failure to thrive, growth problems, gastroesophageal reflux and feeding disorders	1	2	3	4	5
	Hepatomegaly, hepatitis	1	2	3	4	5
	Obesity	1	2	3	4	5
	Iron deficiency	1	2	3	4	5
	Prescribing and managing parenteral nutrition	1	2	3	4	5
Cardiovascular	Congenital heart disease, heart murmurs	1	2	3	4	5
	Syncope, cardiac arrhythmias	1	2	3	4	5
	Interpreting ECGs in all age groups (0–18 years)	1	2	3	4	5
ENT and respiratory	Pharyngitis and tonsillitis, sinusitis	1	2	3	4	5
	Otitis media	1	2	3	4	5
	Hearing loss and hearing assessment	1	2	3	4	5
	Snoring and sleep apnea	1	2	3	4	5
	Asthma, bronchiolitis, croup, tracheo-bronchitis	1	2	3	4	5
	Pneumonia	1	2	3	4	5
	Performing and interpreting oximetry	1	2	3	4	5
	Demonstrating to patient/parent the use of devices used for drug administration for asthma and the use of an epipen	1	2	3	4	5
Allergy and immunology	Allergy and immunologic problems, drug allergy, food allergy and anaphylaxis	1	2	3	4	5
Endocrine	Type I and type II diabetes	1	2	3	4	5
	Disorders of puberty and sexual development	1	2	3	4	5
	Thyroid disease	1	2	3	4	5
Hematology and oncology	Red cell disorders, platelet disorders and hemorrhagic conditions	1	2	3	4	5
	Childhood neoplasms, leukaemia and lymphoma	1	2	3	4	5
Renal and GU	Urinary tract infection, Vesico-ureteric reflux	1	2	3	4	5
	Acute and chronic renal failure	1	2	3	4	5
	Hypertension	1	2	3	4	5
	Undescended testis, hypospadias	1	2	3	4	5
	Menstrual cycle irregularities and menorrhagia	1	2	3	4	5
Mental health	Conducting a mental status examination	1	2	3	4	5
	Anxiety disorders, phobias and obsessive compulsive disorders	1	2	3	4	5
	Depression	1	2	3	4	5
	Attempted suicide	1	2	3	4	5
	Psychosomatic conditions	1	2	3	4	5
	Violent behaviour	1	2	3	4	5
ID	Common bacterial and viral infections	1	2	3	4	5
	Infection preventive strategies and immunization counselling	1	2	3	4	5
	Uncommon but re-emerging infection-related chronic diseases such as tuberculosis, rheumatic fever	1	2	3	4	5
	Cellulitis, orbital cellulitis, necrotizing fasciitis	1	2	3	4	5
Dermatology	Acne, eczema and other dermatidites	1	2	3	4	5
	Pigmentary and vascular lesions (e.g. nevi, hemangiomas)	1	2	3	4	5
CNS and ophthalmology	Epilepsy, seizures	1	2	3	4	5
	Headaches	1	2	3	4	5
	Intracranial haemorrhage and stroke	1	2	3	4	5
	CNS infections, meningitis	1	2	3	4	5
	Floppy infant	1	2	3	4	5
	Red eye	1	2	3	4	5
	Strabismus, vision assessment	1	2	3	4	5
Newborn	Disorders associated with prematurity and low birth weight	1	2	3	4	5
	Respiratory distress syndrome and other respiratory disorders of newborn infants	1	2	3	4	5
	Common neonatal problems (e.g. jaundice, sepsis)	1	2	3	4	5
	Neonatal hypoglycaemia	1	2	3	4	5
Surgery, trauma and MSK	Appendicitis	1	2	3	4	5
	Bowel obstruction, intussusception	1	2	3	4	5
	Non-life-threatening trauma, head trauma, fractures and burns	1	2	3	4	5
	Congenital dislocation of hip	1	2	3	4	5
	Scoliosis	1	2	3	4	5
	Gait disorders, joint and limb pain	1	2	3	4	5
	Septic arthritis and osteomyelitis	1	2	3	4	5
Development/behaviour	Colic	1	2	3	4	5
	Assessing developmental milestones, psychomotor development	1	2	3	4	5
	Developmental delay, mental retardation, developmental regression	1	2	3	4	5
	Providing anticipatory guidance and well child/youth care	1	2	3	4	5
	Attention deficit disorders with or without hyperactivity	1	2	3	4	5
	Learning disability, communicating with school regarding learning problems	1	2	3	4	5
	Behaviour and conduct disorders	1	2	3	4	5
	Autism spectrum disorders, pervasive developmental disorders	1	2	3	4	5
Adolescent medicine	Assessing adolescent/youth (using HEEADSS format)	1	2	3	4	5
	Gynecomastia	1	2	3	4	5
	Risk taking (alcohol, drug, tobacco, and substance abuse),delinquency and sexually transmitted infections	1	2	3	4	5
	Contraception	1	2	3	4	5
Genetics and metabolics	Approach to investigating a dysmorphic child including Down syndrome, Turner syndrome, Fragile X syndrome, Prader–Willi syndrome, Klinefelter syndrome	1	2	3	4	5
	Genetic counselling for minor dysmorphic findings (e.g. vascular malformations, nevi, skin tags) and chromosomal anomalies (e.g. Down syndrome)	1	2	3	4	5
	Workup of infant/child with suspected inborn error of metabolism	1	2	3	4	5
Child/youth maltreatment	Managing children/youth with neglect and/or living in an abusive environment	1	2	3	4	5
	Managing children/youth who are victims of abuse (emotional, physical and/or sexual)
	Providing care to disadvantaged children and advocating for their health care needs	1	2	3	4	5
Continuing care	Children with complex/chronic problems (e.g. post-organ transplant, cerebral palsy, cystic fibrosis, chronic lung disease, short gut syndrome)	1	2	3	4	5
	Dealing with cultural and ethnic differences in a diverse patient population	1	2	3	4	5
	Assisting patients/parents in dealing with and navigating our health care system	1	2	3	4	5
	Managing an efficient work place/office	1	2	3	4	5
	Practicing cost-effective health care	1	2	3	4	5
	Performing practice-based improvement activities, conducting critical appraisal of literature and/or participating in research related to child health	1	2	3	4	5

Notes: Two questions will be asked about each condition/problem/task.

Q1. How important is the competency of diagnosing (including performing tasks) and managing the following for your practice?

Q2. How well prepared were you at the start of your paediatric consultant practice in diagnosing (including performing tasks) and managing the following?

Sampling and recruitment

The RCPSC maintains a database of all physicians who were certified as generalists in paediatrics. There were 435 paediatricians certified by the RCPSC between June 2004 and June 2008, who had completed 3 or 4 years of general paediatrics residency training in a Canadian paediatric residency training programme, and were working in Canada. All these paediatricians were invited to participate in the survey. They were graduates of the 16 Canadian paediatric residency training programmes. The questionnaire was translated into French for the graduates of the three Canadian francophone universities. The questionnaire was administered using a web-based survey format. The paediatricians were sent an electronic letter describing the importance of completing the questionnaire. If no initial response was received, reminders were sent 2 and 4 weeks after the initial request. Respondents were asked to rate the importance of each competency and how well prepared they were at the start of their consultant practice on a Likert-scale of 1 to 5. For completion of the survey, an early bird prize of $400, and a $250 prize for a later draw were given. This study was approved by the Conjoint Health Research Ethics Board of the University of Calgary. Informed consent was implied by respondent participation in the survey.

Statistical analyses

Web responses were directly entered into a secure database. The data was stripped of individual identifiers so that individual paediatricians were not identified. Data files were password protected.

Gap scores between how important a competency/task is and how well prepared the individual had been were calculated. Mean raw gap score = mean importance – mean preparedness. Mean gap score (%) = (mean raw gap score ÷5) × 100. Comparisons were made between the importance and the acquisition of the competencies using gap analysis. Two-tailed t-tests were performed to determine the significance between importance and acquisition of competencies. This methodology enables assessment of the competencies that are required to function as consultant paediatricians and how residency training has prepared paediatricians for practice by determining the training–practice gap scores.

For practical purposes, a gap score of >10% was judged to be a threshold for potential concern with regards to the need for additional training in the competency being assessed.

Results

The response rate for the survey was 43% (187/435). Of the 187 respondents, 14.4% graduated from francophone universities and 85.6% from all the other Canadian universities. A total of 174 respondents indicated their type of work: 54% (94/174) were general paediatricians (academic paediatricians 13.8%; community paediatricians 19.5% and hospital paediatricians 20.7%) and 46% were paediatric subspecialists; 64% of respondents were female.

The majority of the respondents were in their early years of practice. The numbers of years in practice following RCPSC certification were: 1 year: 79 (45.7%); 2 years: 40 (23.1%); 3 years: 29 (16.8%); 4 years: 22 (12.7%); 5 years: 3 (1.7%).

The overall mean scores for all competencies and tasks for all respondents were: importance 3.61; preparedness 3.29; gap score 6.40%; significance: p < 0.001. There were no significant gap score differences between graduates of francophone and all the other Canadian universities in the 19 classification domains.

As the focus of training and competencies required for general paediatricians is different compared to paediatric subspecialists, results from the general paediatrician responses (n = 94) only are presented in this article (Tables 2–6).

Table 2.  Responses regarding importance and preparedness (based on 5-point Likert-scale) and gap scores

Classification domains	Number of items	Mean importance	Mean preparedness	Mean raw gap score*	Mean gap score (%)**	p-value
Mental health	6	3.5	2.33	1.17	23.4	<0.001
Development/behaviour	8	3.94	2.97	0.97	19.4	<0.001
Continuing care	6	3.70	2.96	0.74	14.8	<0.001
Cardiovascular	3	3.83	3.13	0.70	14.0	<0.001
Allergy and immunology	1	4.00	3.30	0.70	14.0	<0.001
Acute care	10	3.90	3.31	0.59	11.8	<0.001
Dermatology	2	3.66	3.08	0.58	11.6	<0.001
CNS and ophthalmology	7	3.80	3.25	0.55	11.0	<0.001
Adolescent medicine	4	3.48	3.07	0.41	8.2	<0.001
Renal and GU	5	3.61	3.2	0.41	8.2	<0.001
Endocrine	3	3.56	3.16	0.40	8.0	<0.001
Surgery, trauma and MSK	7	3.59	3.24	0.35	7.0	<0.001
GI and nutrition	6	3.78	3.49	0.29	5.8	<0.001
Hematology and oncology	2	3.56	3.28	0.28	5.6	0.005
ENT and respiratory	8	3.91	3.64	0.27	5.4	<0.001
Infectious diseases	4	4.03	3.76	0.27	5.4	<0.001
Newborn	4	4.11	4.02	0.09	1.8	0.34
Genetics and metabolics	3	3.12	3.54	−0.42	−8.4	<0.001
Child/youth maltreatment	3	2.74	3.84	−1.10	−22.0	<0.001

Notes: *Mean raw gap score = mean importance – mean preparedness.

**Mean gap score (%) = (mean raw gap score/5) × 100.

Table 3.  Gap scores for all 92 competencies surveyed

Conditions/Disorders/Disease/Problems/Tasks		Mean importance	Mean preparedness	Raw gap score	Gap score (%)	Significance p (two-tailed)
Acute care	Obtaining intravenous access in infants, children and youth	3.49	2.53	0.96	19	<0.001
	Gastroenteritis, dehydration and prescribing and managing fluids and electrolytes in all age groups	4.47	4.44	0.03	1	0.698
	Endotracheal intubation in newborn, children and youth	3.84	3.29	0.55	11	<0.001
	Performing thoracentesis and chest tube placement	2.60	2.05	0.54	11	<0.001
	Provision of cardiopulmonary resuscitation in multidisciplinary teams and participation in mock codes in newborn, children and youth	3.86	3.33	0.53	11	<0.001
	Diagnosing and managing shock	4.30	4.09	0.21	4	0.028
	Determining if urgent consultation or transport is required for a sick infant/child/youth	4.37	3.87	0.50	10	<0.001
	Applying ethical principles to provision or withholding of clinical care, patient confidentiality and informed consent	4.13	3.40	0.72	14	<0.001
	Crisis resource management training (leadership, problem solving, situational awareness, communication skills and resource management) including communicating information and providing support in a crisis situation (e.g. unexpected death)	4.03	2.92	1.11	22	<0.001
	Interpreting abdominal and chest X-rays and, interpreting skeletal and bone X-rays for fractures	3.90	3.16	0.74	15	<0.001
GI and nutrition	Recurrent abdominal pain including constipation as a cause	4.06	3.82	0.24	5	0.002
	Failure to thrive, growth problems, gastroesophageal reflux and feeding disorders	4.34	3.82	0.52	10	<0.001
	Hepatomegaly, hepatitis	3.35	3.10	0.25	5	0.001
	Obesity	3.89	3.11	0.77	15	<0.001
	Iron deficiency	4.08	3.99	0.09	2	0.440
	Prescribing and managing parenteral nutrition	2.97	3.06	−0.09	-2	0.427
Cardiovascular	Congenital heart disease, heart murmurs	4.11	3.41	0.71	14	<0.001
	Syncope, cardiac arrhythmias	3.64	3.08	0.55	11	<0.001
	Interpreting ECGs in all age groups (0–18 years)	3.74	2.88	0.85	17	<0.001
ENT and respiratory	Pharyngitis and tonsillitis, Sinusitis	3.85	3.72	0.14	3	0.129
	Otitis media	4.02	3.93	0.09	2	0.320
	Hearing loss and hearing assessment	3.47	2.77	0.70	14	<0.001
	Snoring and sleep apnea	3.48	3.02	0.46	9	<0.001
	Asthma, bronchiolitis, croup, tracheo-bronchitis	4.51	4.36	0.15	3	0.075
	Pneumonia	4.44	4.34	0.09	2	0.181
	Performing and interpreting oximetry	3.68	3.51	0.17	3	0.088
	Demonstrating to patient/parent the use of devices used for drug administration for asthma and the use of an epipen	3.89	3.47	0.42	8	0.001
Allergy and immunology	Allergy and immunologic problems, drug allergy, food allergy and anaphylaxis	4.00	3.29	0.71	14	<0.001
Endocrine	Type I and type II diabetes	3.81	3.36	0.45	9	<0.001
	Disorders of puberty and sexual development	3.45	3.06	0.39	8	0.001
	Thyroid disease	3.41	3.06	0.35	7	0.001
Hematology and oncology	Red cell disorders, platelet disorders and hemorrhagic conditions	3.57	3.21	0.36	7	<0.001
	Childhood neoplasms, leukaemia and lymphoma	3.57	3.37	0.20	4	0.095
Renal and GU	Urinary tract infection, Vesico-ureteric reflux	4.23	3.97	0.26	5	0.002
	Acute and chronic renal failure	3.59	3.10	0.49	10	<0.001
	Hypertension	3.70	3.11	0.58	12	<0.001
	Undescended testis, hypospadias	3.35	3.20	0.16	3	0.083
	Menstrual cycle irregularities and menorrhagia	3.17	2.60	0.57	11	<0.001
Mental health	Conducting a mental status examination	3.20	2.51	0.69	14	<0.001
	Anxiety disorders, phobias and obsessive compulsive disorders	3.54	2.17	1.38	28	<0.001
	Depression	3.67	2.39	1.28	26	<0.001
	Attempted suicide	3.51	2.49	1.01	20	<0.001
	Psychosomatic conditions	3.57	2.41	1.16	23	<0.001
	Violent behaviour	3.47	2.01	1.46	29	<0.001
ID	Common bacterial and viral infections	4.48	4.25	0.23	5	0.002
	Infection preventive strategies and immunization counselling	4.14	3.95	0.19	3.8	0.007
	Uncommon but re-emerging infection related chronic diseases such as tuberculosis, rheumatic fever	3.47	3.07	0.40	8	<0.001
	Cellulitis, orbital cellulitis, necrotizing fasciitis	3.91	3.81	0.10	2	0.243
Dermatology	Acne, eczema and other dermatidites	3.95	3.32	0.63	13	<0.001
	Pigmentary and vascular lesions (e.g. nevi, hemangiomas)	3.2	2.86	0.34	6.8	<0.001
CNS and ophthalmology	Epilepsy, seizures	4.01	3.39	0.63	13	<0.001
	Headaches	4.04	3.71	0.34	7	0.002
	Intracranial haemorrhage and stroke	3.39	2.89	0.50	10	<0.001
	CNS infections, meningitis	4.32	4.16	0.16	3	0.015
	Floppy infant	3.89	3.25	0.64	13	<0.001
	Red eye	3.41	2.93	0.48	10	<0.001
	Strabismus, vision assessment	3.57	2.46	1.11	22	<0.001
Newborn	Disorders associated with prematurity and low birth weight	4.02	3.77	0.25	5	0.017
	Respiratory distress syndrome and other respiratory disorders of newborn infants	4.15	4.06	0.08	2	0.440
	Common neonatal problems (e.g. jaundice, sepsis)	4.24	4.25	-0.01	0	0.902
	Neonatal hypoglycaemia	4.02	4.02	0	1.0	1.000
Surgery, trauma and MSK	Appendicitis	3.61	3.55	0.06	1	0.482
	Bowel obstruction, intussusception	3.66	3.54	0.12	3	.0.170
	Non-life-threatening trauma, head trauma, fractures and burns	3.53	2.89	0.63	13	<0.001
	Congenital dislocation of hip	3.63	3.25	0.38	8	0.001
	Scoliosis	3.35	2.85	0.49	10	<0.001
	Gait disorders, joint and limb pain	3.56	2.94	0.62	12	<0.001
	Septic arthritis and osteomyelitis	3.78	3.67	0.11	2	0.260
Development/behaviour	Colic	3.75	3.43	0.32	6	0.006
	Assessing developmental milestones, psychomotor development	4.20	3.48	0.72	14	<0.001
	Developmental delay, mental retardation, developmental regression	4.15	3.33	0.82	16	<0.001
	Providing anticipatory guidance and well child/youth care	3.87	3.09	0.79	16	<0.001
	Attention deficit disorders with or without hyperactivity	4.03	2.94	1.09	22	<0.001
	Learning disability, communicating with school regarding learning problems	4.00	2.43	1.57	31	<0.001
	Behaviour and conduct disorders	3.77	2.27	1.50	30	<0.001
	Autism spectrum disorders, pervasive developmental disorders	3.84	2.81	1.03	21	<0.001
Adolescent medicine	Assessing adolescent/youth (using HEEADSS format)	3.94	3.75	0.19	4	0.077
	Gynecomastia	3.06	2.80	0.27	5	0.0130
	Risk taking (Alcohol, drug, tobacco, and substance abuse),delinquency and sexually transmitted infections	3.66	3.07	0.59	12	<0.001
	Contraception	3.27	2.66	0.61	12	<0.001
Genetics and metabolics	Approach to investigating a dysmorphic child including Down syndrome, Turner syndrome, Fragile X syndrome, Prader–Willi syndrome, Klinefelter syndrome	3.72	3.27	0.46	9	<0.001
	Genetic counselling for minor dysmorphic findings (e.g. vascular malformations, nevi, skin tags) and chromosomal anomalies (e.g. Down syndrome)	2.78	3.56	−0.78	−16	<0.001
	Workup of infant/child with suspected inborn error of metabolism	2.87	3.80	−0.93	−19	<0.001
Child/youth maltreatment	Managing children/youth with neglect and/or living in an abusive environment	2.85	3.73	−0.87	−17	<0.001
	Managing children/youth who are victims of abuse (emotional, physical and/or sexual)	2.62	3.94	−1.32	−26	<0.001
	Providing care to disadvantaged children and advocating for their health care needs	2.76	3.85	−1.10	−22	<0.001
Continuing care	Children with complex/chronic problems (e.g. post-organ transplant, cerebral palsy, cystic fibrosis, chronic lung disease, short gut syndrome)	3.28	3.88	−0.60	−12	<0.001
	Dealing with cultural and ethnic differences in a diverse patient population	3.24	3.77	−0.52	−10	<0.001
	Assisting patients/parents in dealing with and navigating our health care system	3.77	2.95	0.82	16	<0.001
	Managing an efficient work place/office	4.12	1.85	2.27	45	<0.001
	Practicing cost effective health care	3.89	2.26	1.63	33	<0.001
	Performing practice-based improvement activities, conducting critical appraisal of literature and/or participating in research related to child health	3.88	3.07	0.81	16	<0.001

Table 4.  Competencies rated as very important to diagnose and manage (mean importance ≥4)

Classification domain	Competency	Mean importance	Mean preparedness	Gap score (%)	Significance (two-tailed t-test) p
ENT and respiratory	Asthma, bronchiolitis, croup, tracheo-bronchitis	4.51	4.36	3	0.075
ID	Common bacterial and viral infections	4.48	4.25	5	0.002
GI and nutrition	Gastroenteritis, dehydration and prescribing and managing fluids and electrolytes	4.47	4.44	1	0.698
ENT and respiratory	Pneumonia	4.44	4.34	2	0.181
Acute care	Determining if urgent consultation or transport is required for a sick child	4.37	3.87	10	<0.001
GI and nutrition	Failure to thrive, growth problems, gastroesophageal reflux and feeding disorders	4.34	3.82	10	<0.001
CNS and ophthalmology	CNS infections, meningitis	4.32	4.16	3	0.015
Acute care	Diagnosing and managing shock	4.30	4.09	4	0.028
ID	Infection prevention strategies and immunization counselling	4.26	3.91	7	<0.001
Newborn	Common neonatal problems (e.g., jaundice, sepsis)	4.24	4.25	−0.01	0.902
Renal and GU	Urinary tract infection, Vesico-ureteric reflux	4.23	3.97	5	0.002
Development/behaviour	Assessing developmental milestones, psychomotor development	4.20	3.48	14	<0.001
Development/behaviour	Developmental delay, mental retardation, developmental regression	4.15	3.33	16	<0.001
Newborn	Respiratory distress syndrome and other respiratory disorders of newborn infants	4.15	4.06	2	0.440
Acute care	Applying ethical principles to provision or withholding of clinical care, patient confidentiality and informed consent	4.13	3.40	14	<0.001
Continuing care	Managing an efficient work place/office	4.12	1.85	45	<0.001
Cardiovascular	Congenital heart disease, heart murmurs	4.11	3.41	14	<0.001
GI and nutrition	Iron deficiency	4.08	3.99	2	0.200
GI and nutrition	Recurrent abdominal pain including constipation as a cause	4.06	3.82	5	0.002
CNS and ophthalmology	Headaches	4.04	3.71	7	0.002
Acute care	Crisis resource management training including communicating information and providing support in crisis situation	4.03	2.92	22	<0.001
Development/behaviour	Attention deficit disorders with or without hyperactivity	4.03	2.94	22	<0.001
Newborn	Disorders associated with prematurity and low birth weight	4.02	3.77	5	0.017
Newborn	Neonatal hypoglycaemia	4.02	4.02	0	1.0
ENT and respiratory	Otitis media	4.02	3.93	2	0.32
CNS and ophthalmology	Epilepsy, seizures	4.01	3.39	13	<0.001
Allergy and immunology	Allergy and immunology problems, drug/food allergy, anaphylaxis	4.00	3.29	14	<0.001
Development/behaviour	Learning disability, communicating with school regarding learning problems	4.00	2.43	31	<0.001

Table 5.  Competencies rated as less than adequately prepared for (mean preparedness score <3.00)

Classification domain	Competency/task	Mean preparedness
Continuing care	Managing an efficient work place/office	1.85
Mental health	Violent behaviour	2.01
Acute care	Performing thoracentesis and chest tube placement	2.05
Mental health	Anxiety disorders, phobias and obsessive compulsive disorders	2.17
Continuing care	Practicing cost-effective health care	2.26
Development and behaviour	Behaviour and conduct disorders	2.27
Mental health	Depression	2.39
Mental health	Psychosomatic conditions	2.41
Development and behaviour	Learning disability, communicating with school regarding learning problems	2.43
CNS and ophthalmology	Strabismus and vision assessment	2.46
Mental health	Attempted suicide	2.49
Mental health	Conducting a mental status examination	2.51
Acute care	Obtaining intravenous access in infants, children and youth	2.53
Renal and GU	Menstrual cycle irregularities and menorrhagia	2.60
Adolescent medicine	Contraception	2.66
ENT and respiratory	Hearing loss and hearing assessment	2.77
Development and behaviour	Autism spectrum disorders, pervasive developmental disorders	2.81
Cardiovascular	Interpreting ECGs (0–18 years)	2.88
Surgery, trauma and MSK	Non-life threatening trauma, head trauma, fractures and burns	2.89
Acute care	Crisis resource management	2.92
Development and behaviour	Attention deficit disorders with or without hyperactivity	2.94
Surgery, trauma and MSK	Gait disorders, joint and limb pain	2.94
Continuing care	Assisting patients/parents in dealing with and navigating health care system	2.95

Table 6.  Competencies/performing tasks with gap scores >10% and p < 0.001

Classification domain	Competency/task	Mean importance	Mean preparedness	Gap score (%)
Continuing care	Managing an efficient work place/office	4.12	1.85	45
Continuing care	Practicing cost effective health care	3.89	2.26	33
Development and behaviour	Learning disability, communicating with school regarding learning problems	4.00	2.43	31
Development and behaviour	Behaviour and conduct disorders	3.77	2.27	30
Mental health	Violent behaviour	3.47	2.01	29
Mental health	Anxiety disorders, phobias and obsessive compulsive disorders	3.54	2.17	28
Mental health	Depression	3.67	2.39	26
Mental health	Psychosomatic conditions	3.57	2.41	23
Acute care	Crisis resource management	4.03	2.92	22
Development and behaviour	Attention deficit disorders with or without hyperactivity	4.03	2.94	22
CNS and ophthalmology	Strabismus and vision assessment	3.57	2.46	22
Development and behaviour	Autism spectrum disorders, pervasive developmental disorders	3.84	2.81	21
Mental health	Attempted suicide	3.51	2.49	20
Acute care	Obtaining intravenous access in infants, children and youth	3.49	2.53	19
Cardiovascular	Interpreting ECGs (0–18 years)	3.74	2.88	17
Development and behaviour	Developmental delay, mental retardation, developmental regression	4.15	3.33	16
Continuing care	Assisting patients/parents in dealing with and navigating health care system	3.77	2.95	16
Continuing care	Performing practice-based improvement activities	3.88	3.07	16
Development and behaviour	Providing anticipatory guidance and well child/youth care	3.87	3.09	16
Acute care	Interpreting abdominal/chest X-rays and skeletal/ bone X-rays for fractures	3.90	3.16	15
GI and nutrition	Obesity	3.89	3.11	15
Development and behaviour	Assessing developmental milestones, psychomotor development	4.20	3.48	14
Acute care	Applying ethical principles to provision or withholding of clinical care, patient confidentiality and informed consent	4.13	3.40	14
Cardiovascular	Congenital heart disease, heart murmurs	4.11	3.41	14
Allergy and immunology	Allergy and immunology problems, drug/food allergy, anaphylaxis	4.00	3.29	14
ENT and respiratory	Hearing loss and hearing assessment	3.47	2.77	14
Mental health	Conducting a mental status examination	3.20	2.51	14
CNS and ophthalmology	Epilepsy, seizures	4.01	3.39	13
Dermatology	Acne, eczema and other dermatidites	3.95	3.32	13
CNS and ophthalmology	Floppy infant	3.89	3.25	13
Surgery, trauma and MSK	Non-life-threatening trauma, head trauma, fractures and burns	3.53	2.89	13
Renal and GU	Hypertension	3.70	3.11	12
Adolescent medicine	Risk taking (alcohol, drug, tobacco, and substance abuse), delinquency and sexually transmitted infections	3.66	3.07	12
Surgery, trauma and MSK	Gait disorders, joint and limb pain	3.56	2.94	12
Adolescent medicine	Contraception	3.27	2.66	12
Acute care	Endotracheal intubation in newborn, children and youth	3.84	3.29	11
Acute care	Provision of cardiopulmonary resuscitation in multidisciplinary teams and participation in mock codes in newborn, children and youth	3.86	3.33	11
Cardiovascular	Syncope, cardiac arrhythmias	3.64	3.08	11
Renal and GU	Menstrual cycle irregularities and menorrhagia	3.17	2.60	11
Acute care	Performing thoracentesis and chest tube placement	2.60	2.05	11

Table 2 presents responses regarding importance and preparedness in the 19 classification domains. Nine of these domains had gap scores >10% indicating that there were specific competencies within these domains that would require ongoing review and consideration for additional training during residency training.

Table 3 depicts gap scores for all 92 competencies surveyed. These gap scores are based on the responses of general paediatricians when asked to indicate the importance of each competency as well as their preparedness at the start of their practice. Paediatricians rated 84 (91.3%) competencies to be important for practice, and indicated that they felt adequately prepared for 69 (75%) of the competencies surveyed.

Table 4 presents 28 competencies that were rated as very important to diagnose and manage. Paediatricians indicated that they were adequately prepared for 24 (85.7%) of these competencies and very well prepared for 9 (32.1%).

Table 5 presents 23 competencies for which paediatricians indicated that they were less than adequately prepared.

Table 6 presents competencies and tasks where the gap scores were >10%. A total of 40 competencies were identified with gap scores >10% suggesting a threshold for potential concern with regards to the need for additional training in these specific competencies. One factor to consider is the importance of the competency.

Four competencies with gap scores ≥30% were identified: (1) managing an efficient work place/office, (2) practising cost-effective health care, (3) learning disability and (4) behaviour and conduct disorders.

Discussion

Canadian general paediatricians identified the vast majority of competencies contained in the RCPSC OTR for paediatrics to be important. They also felt adequately prepared for the majority of competencies surveyed.

Domains where preparation rated as less than adequate

General paediatricians indicated that their preparation for practice was less than adequate in 3 of the 19 domains (mental health, development/behaviour and continuing care). For all competencies in mental health (anxiety disorders, phobias and obsessive compulsive disorders, depression; attempted suicide; psychosomatic conditions and violent behaviour), specific development/behaviour competencies (attention deficit disorders with or without hyperactivity; learning disability, communicating with school regarding learning problems; behaviour and conduct disorders; autism spectrum disorders and pervasive developmental disorders), and specific continuing care competencies (assisting patients/parents in dealing with and navigating the health care system; managing an efficient work place/office; practising cost-effective health care) the mean preparedness score for diagnosing and managing each of these conditions/tasks was <3.00, indicating that preparation for practice was less than adequate irrespective of where they received exposure (in-patient or outpatient hospital; ambulatory or community setting) in order to attain the competency of diagnosing and managing the condition and/or task.

Competencies where preparation rated as well prepared

Competencies rated as very important and for which general paediatricians felt they were very well prepared included: asthma, bronchiolitis, croup, tracheo-bronchitis; pneumonia, common bacterial and viral infections; CNS infections; gastroenteritis, dehydration, prescribing and managing fluids and electrolytes; diagnosing and managing shock; common neonatal problems including respiratory disorders and hypoglycaemia. Recent graduates felt well prepared for competencies within the child/youth maltreatment domain that were previously identified as weaknesses (Ward et al. 2004).

Competencies rated as less important

Some competencies were felt to be less important, but preparation to diagnose and manage these conditions was adequate. These included: genetic counselling; work up of infant/child with suspected inborn error of metabolism; managing children/youth with neglect and/or abuse; providing care to disadvantaged children and advocating for their health care needs. These findings are congruent with changes that have occurred over the past decade in residency training that have improved residents’ self-reported preparation for assessing community needs (Cull et al. 2003; Kaczorowski et al. 2004) and participating in child advocacy efforts (Macnab et al. 1998; Flaherty et al. 2006).

Competencies rated as less than adequately prepared

Within the 19 domains, general paediatricians felt that they were less than adequately prepared for 23 specific core competencies including: 6 in mental health; 4 in development/behaviour; 3 in acute care; 3 in continuing care; 2 in surgery/trauma/MSK; 1 each in CNS; cardiovascular; ENT; renal/GU; and adolescent medicine.

These findings are similar to those of previous Canadian studies (Macnab et al. 1998; Veale et al. 1999; Lieberman & Hilliard 2006; Grant et al. 2007; Korczak et al. 2009) and multiple other US studies (Camp et al. 1997; Roberts et al. 1997; Mulvey et al. 2000; Blumenthal et al. 2001; Blendon et al. 2002; Garfunkel et al. 2005; Freed et al. 2009a, 2009b, 2009c, 2009d; Leslie 2009). Lieberman and Hilliard (2006) found that paediatricians certified in Canada between 1999 and 2003 had received excellent training in most paediatric subspecialties and felt adequately prepared for their careers. However, paediatricians had identified gynaecology, child mental health/psychiatry, surgical subspecialties, behaviour and developmental paediatrics, adolescent medicine, and palliative care as areas of weakness in training.

Training needs

The high numbers of competencies with gap scores >10% among paediatricians suggests a need for additional training in selected areas. We recognize that the competencies required of paediatricians are constantly evolving and may be different for different types of paediatricians. This observation is in keeping with the Residency Review and Redesign project (R³P) participants conclusions in the US that residency training should be tailored to meet the diverse career needs of individuals who choose to care for disparate patients, in diverse settings, with a variety of health-related needs (Jones et al. 2009; Leslie 2009). Residency training programmes should allow flexibility and take a patient-based and family-centred perspective, tailoring training to the diverse and emerging health care needs of the paediatric population (Jones et al. 2009; Leslie 2009). Forrest et al.'s (1999) survey of the referral practices of general paediatricians to specialists suggested that educators should ensure that the 50 most commonly referred conditions (which were similar to the competencies surveyed in this study) are emphasized during paediatric residency training.

As it is unrealistic to expect general paediatric residency training programmes to provide all learning that every paediatrician might conceivably need, residency training programmes will need to determine what expectations are reasonable. It will be important to ensure that making improvements in one area does not occur at the expense of training in other areas (ACGME 2007).

Limitations of study

The overall response rate for this survey was 43% which may limit the generalizability of the results to the population of general paediatricians. However, the results are based on the responses of 94 general paediatricians (response rate of 54% within the overall sample) with respectable representation from the academic, community and hospital domains as well as gender representation similar to the population of practicing paediatricians in Canada (Canadian Medical Education Statistics 2009).

This study is based on self-reported perceptions which can introduce an element of bias as well as general inaccuracy resulting from limitations in memory recall. There is mitigation in the memory recall effect as 69% of the respondents were in their first or second year of practice. Recent graduates are more likely to accurately remember what happened during residency training.

Of course the ideal study to determine the importance of competencies and preparedness in relation to these competencies would require the keeping of a log and in-practice assessment. Implementing such studies is extremely difficult and a literature review did not identify such studies.

This is the first time that these scales for importance and preparedness have been used. The scales were designed to be parallel. However, there may be some systematic bias in the scaling. This could somewhat skew the gap analysis. However, the relative measures and gaps should be consistent and meaningful.

This study deals only with general paediatricians. Programme needs are complex and before determining changes to be made to programmes, a full analysis of the needs of subspecialists would be necessary.

Conclusions

Canadian general paediatricians identified the vast majority of competencies contained in the RCPSC OTR for paediatrics to be important and they also felt adequately prepared for the majority of the competencies that were surveyed. These findings are similar to those of previous studies conducted both in Canada and the United States. The number of competencies with preparation less than adequate, and gap scores >10% suggest a need for additional training in selected areas during residency training, whereas the competencies rated as not important will require ongoing review by training programmes. Given that healthcare is changing rapidly, the competencies required of paediatricians will also constantly evolve and will be different for different types of paediatricians. The needs of general paediatricians and those of paediatric subspecialists should therefore be constantly monitored.

The methodology used in this study differed from others in that it linked specifically to OTR rather than only asking questions about domains. It is also unique in measuring not only preparedness but also the importance for practice of competencies, as well as the gap between importance and preparedness. This approach is therefore very useful in validating the objectives themselves as well as the preparation of residents in relation to these objectives. This methodology could prove to be useful across medical specialties as an on-going process to review and validate their OTRs, and to better define the competencies required for consultant practice.

Definitions

• Competency: “a complex set of behaviours built on the components of knowledge, skills and attitudes”¹⁴

• Competence: “one's ability to perform a task”¹⁴

• Core years: In the Canadian context, the term “core years” refers to the first 3 years of paediatric residency training during which a resident is expected to gain adequate experience in both in-hospital and ambulatory facilities of a children's hospital or of the paediatric department of a general hospital. In addition, appropriate experience in community-based child health services and training in the comprehensive care of children with physical and psychosocial challenges must be obtained. During this period, a resident is also expected to learn the skills to work collaboratively with other medical and health disciplines dealing with infants and children, and acquire the professional attitudes to work with other health disciplines in a variety of health care service models.

• Core competencies: In the Canadian context, the term “core competencies” refers to the competencies required to practice as consultant paediatricians in Canada.

Acknowledgement

This study was funded by The Alberta Children's Hospital Foundation.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.