What are we preparing them for? Development of an inventory of tasks for medical, surgical and supportive specialties

Abstract

Background: Internationally, postgraduate medical education (PGME) has shifted to competency-based training. To evaluate the effects of this shift on the outcomes of PGME appropriate instruments are needed.

Aim: To provide an inventory of tasks specialists perform in practice, which can be used as an instrument to evaluate the outcomes of PGME across disciplines.

Methods: Following methodology from job analysis in human resource management, we used document analyses, observations, interviews and questionnaires. Two thousand seven hundred and twenty eight specialists were then asked to indicate how frequently they performed each task in the inventory, and to suggest additional tasks. Face and content validity was evaluated using interviews and the questionnaire. Tasks with similar content were combined in a total of 12 clusters. Internal consistency was evaluated by calculating Cronbach's alpha. Construct validity was determined by examining predefined differences in task performance between medical, surgical and supportive disciplines.

Results: Seven hundred and six specialists (36%) returned the questionnaire. The resulting inventory of 91 tasks showed adequate face and content validity. Internal consistency of clusters of tasks was adequate. Significant differences in task performance between medical, surgical and supportive disciplines indicated construct validity.

Conclusion: We established a comprehensive, generic and valid inventory of tasks of specialists which appears to be applicable across medical, surgical and supportive disciplines.

Introduction

Calls from society for more transparency with regard to the content and quality of medical education have been the prime drivers of the fairly recent shift to competency-based postgraduate medical education and training in many, mainly Western, countries (Maudsley et al. 2000; Jones et al. 2001; Leach 2001; Carraccio et al. 2002; Frank 2005; Scheele et al. 2008; Lillevang et al. 2009). The introduction of competency-based curricula was based on the assumption that such training would provide future physicians with better preparation for the full scope of contemporary medical practice or, in the words of the Royal College of Physicians and Surgeons of Canada, would train ‘better physicians’ (Frank 2005). While many met this transformation of postgraduate medical education (PGME) with enthusiasm, it also stirred up considerable unease (Grant 1999; Talbot 2004; ten Cate & Scheele, 2007; Reeves et al. 2009). Others have pointed to the huge investment of time, money and effort that has to be expended to reform medical education or have questioned whether these efforts can actually be expected to result in better physicians (Cooper 2006; Hays 2007). Although any curriculum should be evaluated regularly, the misgivings regarding the recent changes make it only the more urgent to evaluate the effects of competency-based postgraduate medical training using sound and appropriate measurement instruments.

Although the obvious way to evaluate the new postgraduate training programmes would be to ascertain whether they actually produce better physicians, the feasibility of this approach seems questionable, mainly due to the varied and often conflicting definitions of what is meant by a ‘good’ physician (de Bont 1997; Ashcroft 2002; Letters 2002; Miles & Leinster 2010). In order to circumvent this problem, we opted for a different approach, looking for ways to develop an evaluation instrument based on the tasks a physician is expected to be able to perform. This approach was prompted by the consideration that, basically, the goal of each phase of the medical education continuum should be to ensure optimal preparation of students and trainees to move on to the next phase of training or to professional practice. For senior postgraduate trainees, the obvious next step is independent performance of the full scope of the tasks routinely performed by a specialist. This suggests that outcomes of PGME can be effectively evaluated by determining to which extent training programmes deliver trainees who are competent to perform these tasks.

Although several studies have investigated how well recently registered specialists felt prepared for clinical practice, none of them covered the full scope of independent practice. Moreover, these studies focused mainly on competencies related to the role of medical expert and relied on a variety of insufficiently validated outcome measures (Liebelt et al. 1993; Lenton et al. 1994; Roberts et al. 1997; Beckett et al. 2006; Benstead 2006; Card et al. 2006; Lieberman & Hilliard 2006). To remedy the lack of research data and to accommodate the shift from input-oriented to output-oriented training, it has been advocated that a systematic approach to programme evaluation be designed that is uniformly applicable to all PGME programmes (Higgins et al. 2005; Musick, 2006). The present study was designed to systematically develop a generic inventory of the tasks of specialists that can be used in evaluating how well specialists are equipped for professional practice after completing their training.

Task inventories are a common feature of job analysis in human resource studies (Oswald 2003; Cascio & Aguinis 2005; Singh 2008). Although widely used in vocational training (Blackmore 1999), job analysis has no tradition in the professions, partly because professional groups tend to resist this type of evaluation of their activities (Blackmore 2000). In vocational training, the primary aim of the job analysis is to align the content of training programmes with the competencies required for independent practice. Systematic job analysis of the work of specialists could be similarly helpful in evaluating and improving the content of PGME programmes.

Apart from its usefulness as an instrument for evaluating training programmes, there are other potential uses for a comprehensive inventory of the tasks of specialists. The introduction of competency-based training has led to widespread concern about how to tease apart and translate to clinical teaching practice the different competencies that play a role in complex professional task performance (Grant 1999; Leung 2002; ten Cate & Scheele 2007; Jones et al. 2011). Also, clinical work consists of tasks rather than competencies, and competency development is judged through observation and assessment of’ task performance in daily practice. To help bridge the gap between theory and practice, Entrustable Professional Activities (EPAs) have been introduced (ten Cate 2005; ten Cate & Scheele 2007). EPAs are defined as those professional activities that operationally define a profession and that a trainee can progressively be trusted to perform competently. Although very valuable, EPAs are limited to (critical) situations for specific specialties. In our opinion, in addition to rather specific EPAs and broadly defined competencies, an inventory of generic tasks of specialists could have value as an instrument to facilitate evaluation and improvement of curricula, training methods, coaching and assessment in specialty training.

Table

Practice points

Competency-based postgraduate training programmes promise to prepare trainees better for the full scope of medical practice, but an inventory of this practice is lacking. A comprehensive and generic inventory of tasks of medical specialists can be used to evaluate outcomes and content of specialist training programmes. Procedures commonly applied in Human Resource Management can be employed to develop an inventory of tasks. An inventory of 91 tasks is applicable to all specialties and reflects the major differences in task performance frequencies between typical medical, surgical and supportive specialties.

Methods

Setting of the study

The study was conducted in the Netherlands among specialists representing all 28 medical specialties with accredited training programmes for surgical, medical, or supportive disciplines (Table 1). Training programmes are offered in eight, geographically defined ‘education regions’. Participants were recruited in the north-eastern region from the three different hospital settings known in the Netherlands: (1) University Medical Centres, which provide tertiary care, undergraduate medical education and postgraduate specialty training, with a strong involvement in research; (2) large (>500 beds) general teaching hospitals providing secondary care (with some referral functions) and undergraduate and postgraduate training and (3) small (<500 beds) district hospitals providing basic secondary care.

Table 1  Overview of three groups of Dutch disciplines

Surgical specialties	Medical specialties	Supportive specialties
Cardiac surgery	Cardiology	Microbiology
General surgery	Geriatrics	Nuclear medicine
Neurosurgery	Internal medicine	Pathology
Oral surgery	Neurology	Radiology
Orthopaedics	Paediatrics	Anaesthesiology
Plastic surgery	Psychiatry	Clinical genetics
Gynaecology	Rheumatology	Radiotherapy
Ophthalmology	Dermatology
Otorhinolaryngology	Gastroenterology
Urology	Rehabilitation medicine
	Respiratory medicine

Note: Bold type indicates a typical representative of a discipline.

In line with established ethical standards (Eva 2009; ten Cate 2009), and the legal requirements in the Netherlands, all participants were informed that participation was voluntary and the data would be treated confidentially and anonymously.

Specificity of tasks

Our main aim in creating a task inventory was to design an instrument to evaluate whether postgraduate specialty training programmes succeed in preparing trainees for the full scope of tasks they are expected to perform in professional practice. Since we aimed to create one inventory for surgical, medical and supportive specialties, there were two requirements to fulfil: the tasks had to be formulated in a sufficiently generic way and tasks characteristic for surgical, medical, or supportive specialties had to be incorporated as well. So, on the one hand, we incorporated tasks that are not specifically discipline related, such as ‘keeping up to date with the professional literature’ or ‘collaborating with administrative support staff’. On the other hand we included tasks like ‘performing surgical procedures in the operating theatre’ and ‘performing a laboratory test or imaging procedure’, because these tasks are routinely performed by surgical and supportive disciplines, respectively. We took care, however, to exclude tasks that were only a differentiation of a more general task, such as ‘performing coronary bypass surgery’, which is limited to one surgical subspecialty only.

Item development, initial phase

Since there is no established way to perform a job analysis of the tasks of specialists, we conducted an exploratory study in separate phases (Table 2). Using methods from job analysis in human resource management (Cascio & Aguinis 2005), we first performed a pilot study involving document analysis, observation and interviews. To identify the specific and generic tasks of every discipline, we reviewed curricular documents of all specialties. Subsequently we analyzed field notes taken while observing a typical working day of 10 specialists from different specialties and transcripts of interviews with 17 specialists about their tasks in daily practice. During the whole procedure, in an iterative process, we used initial findings as input for further data collection and analysis, continuously integrating and differentiating tasks to arrive at an optimal level of aggregation. Finally, a web-based questionnaire survey was sent to 328 specialists in order to establish task frequencies, validate findings and elicit critical comments and suggestions. This led to an initial inventory consisting of 73 tasks.

Table 2  Stages of inventory development

	Method	Subject	Result
Initial item development phase	Document analysis	Curricular documents of all accredited specialties
	Observation	Ten specialists (surgical, medical and supportive)
	Interview	Seventeen specialists (surgical, medical and supportive)	Initial inventory of 73 items
	Questionnaire	Tasks frequency Questionnaire using initial inventory returned by 328 specialists of all accredited specialties	Refinement needed: – Better reflection of supportive discipline – Resolvement of ambiguities
Advanced item development phase	Observation & Interview	2 × micriobiologist 2 × pathologist
	Interview (method of advanced item development phase)	28 specialists of all accredited specialities	Advanced inventory of 89 items
Final validation	Questionnaire	Task frequency Questionnaire using advanced inventory returned by 706 specialists of all accredited specialties	Final inventory of 91 items

Item development, advanced phase

After the initial item development phase, we concluded that two important adjustments had to be made. First, several tasks had to be reworded because some concepts were interpreted differently across disciplines. ‘Procedures’ and ‘surgery’, for example meant different things to surgeons and gastroenterologists. Second, the initial inventory needed further adaptation to improve applicability for the diagnostic/laboratory activities of the supportive disciplines. Therefore, we further observed typical representatives of the supportive disciplines, two specialists in pathology and two specialists in medical microbiology, during a typical working day, taking field notes to document tasks, activities and subjective impressions. Next, we conducted semi-structured interviews with the same specialists to verify our observations and gain further insight into their work. Subsequently 28 specialists from all specialties were interviewed by JP an BB for approximately one hour to explore the comments from the questionnaire concerning the initial inventory, determine whether new tasks should be added and seek confirmation that all the relevant tasks had been covered at an appropriate level of aggregation. In order to ensure that proper attention was paid to any differences between hospital settings, we purposively selected interviewees from each of the three settings. The interviews were guided by a topic list and by the preliminary inventory. Starting questions were:

1. Which tasks are typical of your specialty and distinguish it from other specialties?

2. Which of your tasks are typical of your work and distinguish it from the work of your colleagues from your own specialty?

3. Which tasks are most distinctive for you in the following environments: department, hospital, region and country (the professional association of your specialty)?

4. Do you wish to make any further comments or suggest additions to the task inventory?

Before each interview, the interviewer analysed how the field of activity of the specialty in question was described in curricular documents, in order to identify specialty-specific tasks. We carefully checked that the task descriptions in the inventory were clear and the wording easy to understand and unambiguous. The respondents were asked to give examples of tasks and we compared examples given by different respondents. The interviews were audio-recorded, transcribed and analysed using qualitative data analysis software (Atlas.ti, Berlin, Germany). In an iterative process, the results of the analyses were used as input for subsequent interviews.

Validation phase

The analysis of the field notes and the interviews resulted in an advanced task inventory that was submitted to respondents in a web-based questionnaire. To ensure that the inventory was relevant to all hospital settings, we asked each of the nineteen hospitals in the north-eastern education region to distribute the questionnaire among all specialists. Respondents were asked to indicate the frequency with which they performed each task on a seven-point scale: 1 = never; 2 = rarely; 3 = every year; 4 = every six months; 5 = every month; 6 = every week; 7 = every day. The questionnaire also collected demographic data and asked about respondents’ experience as a specialist. In order to promote reflection and evaluation, tasks with resembling content were distributed and presented in 12 clusters. For member validation we invited comments, suggestions and additions at the end of each cluster of related tasks and at the end of the questionnaire.

We assessed face, content and construct validity of the inventory and the internal consistency of the clusters of tasks. (Terwee et al. 2007) Face validity – a qualitative judgement of the extent to which an instrument measures what it is supposed to measure – was assessed in the semi-structured interviews. Content validity – the extent to which the concepts of interest are captured by the measurement – was evaluated by examining the number of additional tasks proposed by the respondents and by analysing their comments and remarks. To further satisfy the criterion of content validity, we made sure that there were no tasks included in the inventory that had no relevance according to any of the responding specialists. Moreover, clusters of similar tasks were expected to show internal consistency in terms of frequency scores. Finally, we determined the construct validity – the extent to which a score relates to other measures in a way that is consistent with predefined hypotheses – by determining whether differences in the frequency with which medical, surgical and supportive specialists performed specific (clusters of) tasks were in the expected direction, i.e. were in accordance with the following assumptions.

1. Specialists in surgical and medical disciplines perform tasks directly related to patient care more frequently compared to specialists in supportive disciplines.

2. There are no or only small differences in the frequency of performance of the generic tasks.

3. Specialists in surgical disciplines perform tasks related to surgical procedures more frequently compared to specialists in medical and supportive disciplines.

4. Specialists in supportive disciplines perform diagnostic tasks more frequently compared to specialists in surgical and medical disciplines.

Statistical analysis

Descriptive statistics were calculated to analyse task frequencies. Next, to determine whether the frequency scores reflected characteristic differences between the three disciplines and to establish construct validity, we compared the specialties that were the most typical representatives of their discipline. Over the years the boundaries between medical, surgical and supportive specialties have become increasingly blurred. Clinical genetics, for example was traditionally regarded as a supportive discipline, but today's clinical geneticists have regular contacts with patients, involving not only genetic counselling but also taking patient histories, performing physical examination and initiating further investigations. Accordingly, clinical genetics can no longer be considered to be a typical representative of supportive specialities. Consequently, this was not considered to be a suitable specialty to determine whether the frequency scores reflected characteristic differences between the three main disciplines. In order to circumvent this problem, the authors decided by consensus, which specialities were the most typical representatives of their discipline. The criterium for typical surgical specialties was: the relative importance of surgical procedures. Surgical specialties score high on both aspects, medical specialties score low on surgical procedures and high on direct patient contacts and supportive specialties score low on both aspects (Table 1).

To enable statistical comparisons, the tasks were grouped into twelve clusters based on homogeneity of content. Cronbach's alpha was calculated to determine the internal consistency of each cluster, with >0.7 being considered adequate (Bland & Altman 1997). To test whether the frequency scores of the task clusters differentiated between the three disciplines, Games-Howell's post hoc procedure was used to perform multiple comparisons in SPSS (Field 2009).

Results

As a result of the observations and interviews in the advanced item development phase, 26 new tasks were added to the initial inventory of 73 tasks. The special attention we paid to the supportive disciplines resulted in eight new tasks. Thirty-four task descriptions were reworded to improve precision and avoid ambiguity. For example, the ambiguity of the terms ‘procedures’ and ‘surgery’ was resolved by defining the different locations where they are performed, such as the operating theatre, office setting etc. Ten tasks were discarded as being further specifications of already included tasks. For example ‘taking a history’ and ‘performing a physical examination’ form an integral part of a patient consultation. Other tasks were discarded because they were interpreted as a skill or cognitive activity rather than a task (e.g. ‘identifying gaps in my own knowledge’). This resulted in an advanced inventory of 89 tasks, which was further tested using a web-based questionnaire.

Seventeen of the 19 hospitals in the education region we studied (two small district hospitals declined to participate) distributed the questionnaire to all 2728 medical specialists working in these hospitals. We received 706 (38%) completed questionnaires (Table 3), yielding 112 comments, which resulted in addition of two tasks: ‘e-mail correspondence with patients’ and ‘asking another medical specialist to perform a treatment as co-attending physician’. Four respondents proposed new tasks, including ‘discussing withdrawal of life support in newborns with severe congenital abnormalities’, which were considered too specific to warrant inclusion in the inventory. Similarly, several respondents proposed tasks which were already covered by the inventory. However the large majority of the comments concerned personal information (working part-time or in different settings), or referred to personal opinions regarding general developments in the healthcare system affecting the work of specialists. Examples of such comments are: ‘I am concerned that the present emphasis on competencies will go at the expense of clinical knowledge’ and ‘increasing budget cuts force medical specialists to do more and more scut work.’

Table 3  Demographics

	Surgical specialists	Medical specialists	Supportive specialists	Total sample
N	99	227	71	706
University medical centre	38%	28%	22%	29%
General hospital	46%	46%	53%	46%
District hospital	16%	26%	25%	25%
Mean age in years (SD)	48 (8)	49 (8)	49 (8)	48 (8)
Mean experience in years (SD)	12 (9)	14 (9)	14 (8)	13 (9)
Female	13%	43%	24%	33%
Male	87%	57%	76%	67%

The average frequency of task performance ranged from 6.7 (every day) to 1.9 (rarely). At least 15% of respondents indicated that they performed each task every six months at least (the complete inventory and accompanying frequency scores can be found in Table 4).

Table 4  Overview of all the tasks with mean frequency scores and percentage of respondents performing the task at least every six months

	Total	Surgical	Medical	Supportive	% at least every six months
N	706	99	227	71
Consultations and visits	4.96	5.42	5.37	2.78
Preparing for a consultation, patient visit or ward rounds	6.20	6.56	6.63	3.79	90
Performing a treatment as the attending physician	5.93	6.41	6.53	2.48	87
Consultation with a new outpatient	5.91	6.06	6.24	3.01	88
Consultation with a patient with an acute or urgent illness (in the clinic or on the ward)	5.88	6.11	6.22	3.28	90
Follow-up of a patient with short-lasting illness	5.48	5.97	6.07	2.08	81
Performing on-call services (either in house or by telephone)	5.48	5.74	5.63	4.90	94
Follow-up of a patient with chronic illness	5.41	5.86	6.13	2.28	80
Participating in a multidisciplinary consultation (clinic)	3.56	4.74	3.89	2.21	54
Leading an inpatient ward	3.29	3.92	3.73	1.85	43
Doing consultations (in another hospital or location) as a locum	2.47	2.79	2.61	1.90	30
Communication with patients and their family	4.52	4.71	5.11	2.01
Explaining diagnosis, treatment plan and alternatives to a patient	6.30	6.46	6.64	3.54	93
Handling a third party in patient contacts	6.06	6.18	6.37	3.90	93
Speaking on the telephone with patients and/or their family members	5.56	5.66	6.14	2.80	87
Discussing management and treatment with a patient's family members	5.28	5.62	6.03	1.77	84
Breaking bad news to a patient	4.99	5.26	5.41	1.76	82
Discussing consent (for diagnostic procedures, organ donation, medication or surgery) with patients or their legal guardians	4.83	5.10	5.41	1.90	76
Supporting a patient's family	4.56	4.65	5.56	1.39	72
Discussing advance directives with patients	3.78	4.14	4.59	1.08	58
Dealing with inappropriate (e.g. demanding or aggressive) behaviour by patients or family members	3.58	3.79	4.18	1.70	56
Discussing medical errors with patients and their family members	3.21	3.58	3.57	1.82	47
Being actively involved in end-of-life decisions, such as withholding or withdrawing treatment, providing palliative care and euthanasia	3.11	3.01	3.82	1.38	44
Discussing palliative care and end-of-life decisions with patients	3.04	3.09	3.62	1.11	43
Emailing with patients *
Patient related administration	5.06	5.39	5.56	2.67
Keeping a patient's chart up-to-date	6.21	6.45	6.78	2.73	90
Writing or dictating letters about patients	6.15	6.41	6.6	3.87	91
Filling in diagnosis forms and similar administrative procedures	5.57	5.91	6.37	2.73	79
Negotiating with a health care insurance provider about compensation for a treatment	2.32	2.8	2.51	1.34	27
Transfer. admission and discharge	4.20	5.08	4.92	1.48
Admitting a patient	5.01	5.83	5.77	2.00	79
Discharging a patient	4.79	5.79	5.64	1.59	74
Transferring a patient to another ward or unit	4.14	4.92	4.86	1.37	65
Transferring a patient to another hospital	3.57	4.19	4.35	1.28	60
Transferring a patient to another health care institution (e.g., nursing home or rehabilitation centre)	3.48	4.67	3.97	1.15	53
Collaboration with specialists	5.31	5.45	5.58	4.45
Ordering laboratory tests or imaging procedures	6.27	6.46	6.72	3.59	93
Offering advice to a colleague who has consulted you about a patient	5.77	5.72	5.77	6.10	96
Preparing for patient handover or clinical conference	5.73	5.59	5.95	6.28	91
Participating in multidisciplinary conferences	5.60	5.69	5.77	6.30	92
Consulting a colleague from a different medical specialty about a patient	5.60	5.78	5.89	3.34	96
Presenting a patient at a clinical conference or handover	5.45	5.58	5.67	5.55	89
Consulting with a general practitioner (or dentist) about a patient	5.44	5.39	5.73	4.96	93
Performing a treatment at the request of a fellow physician	5.25	5.38	5.36	4.17	87
Coordinating different aspects of care for an individual patient	5.18	5.86	6.03	2.00	98
Consulting a fellow medical specialist about a patient	4.95	5.17	4.85	5.38	86
Leading a multidisciplinary patient care team	3.19	3.65	3.70	1.31	42
Asking another medical specialist to perform a treatment as co-attending physician*
Collaboration with non-specialists	4.92	5.28	5.51	3.33
Collaborating with administrative support staff	6.71	6.60	6.80	6.87	98
Collaborating with nurses	6.20	6.55	6.65	3.73	91
Collaborating with support staff (e.g., dieticians, physical therapists)	5.49	6.03	6.25	2.31	84
Collaborating with nurse practitioners	4.75	5.30	4.88	4.20	70
Doing bedside ward rounds	4.73	6.06	5.77	1.54	71
Collaborating with laboratory personnel	4.64	4.32	4.53	6.82	68
Collaborating with psychosocial professionals	4.39	4.34	5.46	1.31	70
Doing chart-based ward rounds	4.32	5.09	5.56	1.90	65
Collaborating with spiritual or religious counsellors	3.01	3.19	3.70	1.27	43
Management	4.28	4.55	4.13	4.28
Performing management and organisation activities for the partnership, ward or unit	4.81	5.29	4.50	4.89	75
Contributing to organisational changes of the unit or department	4.55	4.68	4.46	4.62	78
Participating in hospital committees (e.g., infection or M&M committee)	4.20	4.35	4.17	4.15	73
Participating in regional or national committees (e.g., scientific society committees)	3.54	3.95	3.40	3.49	63
Education	4.20	4.43	4.17	3.36
Supervising medical students and clerks	5.40	6.03	5.71	3.80	86
Supervising residents	5.36	6.12	5.51	4.62	78
Teaching students and residents	4.46	5.01	4.63	3.80	80
Teaching nurses, laboratory personnel or nurse practitioners	3.67	3.72	3.82	3.37	64
Contributing to changes in undergraduate and postgraduate medical education	3.37	3.77	3.47	3.03	49
Contributing to the continuing professional development of colleagues (e.g., medical specialists, dentists, family physicians)	3.25	3.54	3.28	3.08	47
Providing medical information to groups (e.g., patient support groups or parents)	2.60	2.82	2.79	1.83	26
Continuing professional development	4.37	4.51	4.39	4.33
Keeping up with the literature in the field	5.87	5.80	5.94	5.94	98
Staying up-to-date in one's field by attending continuing professional development courses	4.96	4.84	4.97	4.97	99
Discussing critical incidents, medical errors or complaints during a conference in a structured fashion	4.70	5.05	4.56	4.55	87
Contributing to the development of the field in my specialty	4.58	4.69	4.50	4.65	81
Dealing with feedback from fellow medical specialists	4.57	4.81	4.68	4.54	75
Providing feedback to fellow medical specialists	4.49	4.72	4.59	4.44	75
Improving one's competence in medical education by attending faculty development activities	2.97	3.24	3.13	2.80	37
Improving one's competence in organisation and management by attending continuing professional development activities	2.80	2.97	2.74	2.72	33
Research	2.90	3.64	2.89	2.66
Presenting a critical review of a medical research paper	3.62	3.97	3.63	3.32	62
Performing scientific research	3.30	4.39	3.16	3.14	42
Writing a research paper	2.75	3.61	2.79	2.49	31
Preparing and submitting grant applications	1.93	2.61	1.97	1.69	16
Surgical and non-surgical procedures	3.39	4.85	2.30	2.00
Preparing for a procedure or surgery	4.58	6.34	3.28	2.75	67
Preparing a patient for a procedure or surgery (e.g., anaesthesia, sterile draping)	3.93	6.14	2.26	2.39	52
Performing procedures and minor surgery in an office setting	3.74	5.32	2.43	2.54	55
Performing procedures or surgery in the operating theatre	3.48	6.18	1.84	1.87	47
Performing procedures on the ward or in an intensive care unit (e.g., bone marrow aspiration or central line placement)	2.81	2.91	2.68	2.10	38
Leading a surgical, trauma or resuscitation team	2.74	5.14	1.70	1.14	33
Performing procedures or surgery in a day treatment centre (e.g., bronchoscopy or ERCP)	2.44	1.95	1.91	1.68	28
Diagnostics	3.84	3.49	3.42	6.67
Writing a report about results of laboratory testing or imaging	4.75	4.62	4.74	6.51	67
Based on results of laboratory tests or imaging procedures, providing advice to a colleague on differential diagnosis, prognosis or treatment options	4.44	3.94	4.38	6.82	68
Determine which laboratory test or imaging procedure is most appropriate to answer a clinician's query	4.28	4.15	4.00	6.63	60
Judging whether a clinician orders the most appropriate laboratory test or imaging procedure to solve the diagnostic issue at hand	4.16	3.97	3.97	6.48	57
Retrieving additional clinical information following a request for laboratory testing or imaging	4.03	3.92	3.56	6.54	58
Performing a laboratory test or imaging procedure	3.31	2.97	2.28	6.80	41
Notifying an applicant in writing of the results of laboratory tests or imaging procedure	2.89	2.26	2.22	6.89	34
Notifying an applicant by telephone of the results of laboratory tests or imaging procedure	2.85	2.10	2.18	6.66	35

Notes: Items are measured at a seven-point-scale ranging from never to daily.

*Task was added as a result of the survey: no data available.

The internal consistency of the clusters of tasks was above the desired threshold of 0.70 (Bland & Altman 1997; Spiliotopoulou 2009) (Table 5). There were significant differences in frequency of performance of clusters of tasks between surgical, medical and supportive specialists (Tables 5 and 6). Tasks involving direct contact with patients (clusters 1–6) were performed more often by surgical and medical specialists than by supportive specialists. The only significant, albeit small, difference between surgical and medical specialists related to communication with patients and their families. The generic tasks (clusters 7–10) showed few and relatively small significant differences between the three groups of specialists, although supportive specialists performed fewer tasks related to education. Unexpectedly, surgical specialists performed significantly more management and research related tasks. To determine whether the latter finding might be due to oversampling of surgeons from the university medical centre, we performed an additional ANOVA with work setting (university medical centre, general hospital and district hospital) as covariate. Work setting proved to be significantly associated with the frequency of research tasks, F(1381) = 108.04, p < 0.001), but after controlling for the effect of work setting there remained a significant effect of specialty on the frequency of research tasks, F(2381) = 9.92, p < 0.001), suggesting that oversampling of surgeons in an academic setting was not the sole cause of the difference. In line with our hypotheses, surgical specialists performed significantly more surgical procedures than other specialists (cluster 11). Finally, supportive specialists performed significantly more diagnostic tasks compared to surgical and medical specialists (cluster 12).

Table 5  Comparison of frequency scores between disciplines: results from ANOVA and Cronbach Alpha

	Surgical	Medical	Supportive
	Mean (SD)	Mean (SD)	Mean(SD)	F (2394)	p	Cronbach α
1. Consultations and visits	5.42 (0.85)	5.37 (0.77)	2.78 (1.40)	227.76	0.000**	0.80
2. Communication with patients and family	4.71 (0.88)	5.11 (0.81)	2.01 (0.91)	372.69	0.000**	0.91
3. Patient related administration	5.39 (1.13)	5.56 (1.05)	2.67 (1.41)	98.44	0.000**	0.73
4. Transfer, admission and discharge	5.08 (1.32)	4.92 (1.14)	1.48 (1.01)	261.41	0.000**	0.91
5. Collaboration with specialists	5.45 (1.00)	5.58 (0.68)	4.45 (0.94)	52.67	0.000**	0.73
6. Collaboration with non specialists	5.28 (1.02)	5.51 (0.85)	3.33 (0.76)	171.01	0.000**	0.76
7. Management	4.55 (1.24)	4.13 (1.19)	4.28 (1.39)	4.28	0.014*	0.70
8. Education	4.43 (0.90)	4.17 (0.83)	3.36 (1.00)	26.76	0.000**	0.75
9. Continuing professional development	4.51 (0.79)	4.39 (0.71)	4.33 (0.75)	1.53	0.217	0.71
10. Research	3.64 (1.41)	2.89 (1.39)	2.66 (1.15)	13.89	0.000**	0.83
11. Surgical and non-surgical procedures	4.85 (0.86)	2.30 (1.28)	2.00 (1.52)	165.02	0.000**	0.83
12. Diagnostics	3.49 (1.53)	3.42 (1.57)	6.67 (0.81)	142.76	0.000**	0.86

Notes: *p < 0.05.

**p < 0.01.

Table 6  Post hoc comparisons: 95% confidence intervals of mean differences between disciplines

	Surgical vs. medical		Surgical vs. supportive		Medical vs. supportive
Cluster	Lower limit	Upper limit	Lower limit	Upper limit	Lower limit	Upper limit
1. Consultations and visits	−0.19	0.28	2.19**	3.08**	2.17**	3.00**
2. Communication with patients and family	−0.64**	−0.16**	2.37**	3.03**	2.81**	3.39**
3. Patient related administration	−0.59	0.42	1.41**	2.37**	1.73**	2.59**
4. Transfer, admission and discharge	−0.20	0.52	3.18**	4.02**	3.10**	3.78**
5. Collaboration with specialists	−0.37	0.16	0.67**	1.38**	0.85**	1.42**
6. Collaboration with non-specialists	−0.51	0.04	1.63**	2.27**	1.93**	2.44**
7. Management	0.09*	0.78*	−0.21	0.77	−0.59	0.28
8. Education	−0.01	0.50	0.62**	1.32**	0.41**	1.04**
9. Continuing professional development	−0.09	0.34	−0.10	0.47	−0.18	0.30
10. Research	0.35**	1.16**	0.52**	1.45**	−0.17	0.62
11. Surgical and non-surgical procedures	2.26**	2.84**	2.31**	3.26**	−0.24	0.70
12. Diagnostics	−0.37	0.51	−3.60**	−2.74**	−3.58**	−2.91**

Notes: * p < 0.05.

**p < 0.01.

Discussion

The outcomes of the study suggest that it is feasible to develop a generic inventory of the tasks of medical specialists that is applicable to surgical, medical and supportive specialties and has adequate face, content and construct validity as well as internal consistency.

The final inventory of 91 tasks met the predefined criteria of being applicable to all 28 specialties with training programmes in the Netherlands while also reflecting the major differences in tasks between typical medical, surgical and supportive specialties. Frequency of task performance ranged from every day to rarely. Although frequency of performing a task does not necessarily equate importance or relevance of the task, all tasks were performed regularly by at least 15% of the respondents and no tasks were denoted as irrelevant.

The significant differences in frequency of task performance were in the expected directions, and any differences in generic clusters were small. Supportive specialists reported lower frequencies of tasks related to education, which can be explained by the non-involvement of these specialties in undergraduate clinical training and by their limited role in patient education, due to the nature of their work (few contacts with patients). The significantly higher frequency of research and management tasks reported by surgical specialists was unexpected. We ruled out that it was solely attributable to oversampling of surgeons from an academic setting. Although this unexpected result may reflect reality, it remains to be fully explained. However, the comparisons of the inherent differences between medical, surgical and supportive disciplines in this study were not directed at defining the content of the disciplines but rather at using the typical differences between disciplines for establishing the validity of the task inventory. Given the generally good fit between results and expectations and the small size of the unexpected effects, it seems unlikely that the unexpected differences should give cause to question the validity of the inventory.

Strengths and limitations

To our knowledge, this was the first study to produce a comprehensive inventory of the tasks of specialists. The methodological rigour of the study, exemplified by triangulation of findings from document analyses, observations, interviews and questionnaires and the iterative process of data collection, is in accordance with the standard methods of job analysis in human resource management research, which are assumed to guarantee optimal comprehensiveness and objectivity (Tuckett 2005; Oliver-Hoyo & Allen 2006; Schwandt 2007). The comprehensiveness (content validity) of the inventory is supported by the fact that only two new tasks were introduced by the respondents to the questionnaire. The inclusion in the interviews of specialists from every specialty and the 706 questionnaires returned by respondents from all specialties and each of the different hospital settings suggest that the inventory is representative of all 28 specialties involved in specialty training in the Netherlands.

A general limitation of job analysis is that it inevitably relies on implicit and explicit conceptions that determine what is being measured and valued (Blackmore 2000). Our goal was to develop an instrument to evaluate 28 postgraduate training programmes, and we took great care to ensure that the inventory was comprehensive and relevant to all specialties. Consequently the scope and content of the inventory are specifically suitable for this purpose. The frequency scores showed that supportive disciplines score relatively low on a substantial amount of tasks, especially patient related tasks. Because we devoted extra attention to the task of the supportive disciplines during the advanced item development phase, we have no reason to doubt the completeness of the inventory. Ultimately, the inventory was designed to evaluate several programmes simultaneously by asking recently registered specialists to indicate how well their training programme has prepared them for practice. Respondents then have the option to indicate whether a specific tasks is a part of their routine work. This makes the inventory generically applicable because respondents would have deleted redundant tasks themselves. However, when the inventory is used for other purposes, e.g. within a single specialty, the applicability of the inventory might be enhanced by removing redundant tasks and adding tasks with special relevance to that specialty or by appending specialty specific Entrustable Professional Activities.

We believe, the inventory can be used internationally, since most tasks seem to be relevant to the work of specialists in other Western countries. Nevertheless, account should be taken of international differences in culture, legal system and the organisation of the healthcare system (Lynn Payer 1996). For example, tasks like ‘Avoiding litigation’ or ‘Setting up/running a business’, while not sufficiently relevant to warrant inclusion in the Dutch inventory, might have to be added to ensure validity in USA settings.

Future research

As a next step, the inventory needs to show whether it can meet its purpose. Several questions need to be addressed. Which tasks do recently registered specialists feel worst or best prepared for by their training programme? And will competency-bases training lead to better prepared specialists? Although we believe the inventory to be internationally applicable, a cross-national validation study needs to be conducted to support this claim.

Conclusion

We established a comprehensive, generic, representative and valid inventory of tasks of medical specialists in the Netherlands. As a component of an advocated systematic approach (Higgins et al. 2005; Musick 2006), this inventory can be used to evaluate outcomes and content of training programmes and as an aid in curriculum development, training, coaching and assessment.

Acknowledgements

The authors would like to acknowledge and thank Professor Janke Cohen-Schotanus, Abe Meininger, Professor Yvonne Steinert and Professor Bernard Nijstad for their help and advice. Moreover, we thank Wilko van den Bergs, Janne Allema and Anna Yedema for their invaluable contribution to the pilot studies. Dr Andre Schultz and Mereke Gorsira are greatly acknowledged for their assistance in translation. Finally, we thank all specialists who contributed to this study.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

Notes

*For the purpose of this article, the term ‘specialist’ refers to a physician who has completed a postgraduate specialty training programme in a medical, surgical or supportive specialty.