The development and evaluation of a Professional Self Identity Questionnaire to measure evolving professional self-identity in health and social care students

Abstract

Background: Professional self-identity is a ‘state of mind’ – identifying one's-self as a member of a professional group. Delayed professional self-identity is a barrier to successful transition from student to professional. Current trends in medical education limit student doctors’ legitimate peripheral participation and may retard their developing professional self-identity compared with other health and social care students.

Aims: Develop a tool to monitor the development of professional self-identity to operate across the different health and social care professions and evaluate the tool with student doctors before wider data collection.

Method: Content analysis of relevant curricula, mapped to professional standards documents, defined initial content. Field tests across 10 professional groups refined questionnaire items. A cross-sectional study on 496 student doctors evaluated validity on the basis of internal structure and relationships with external variables.

Results: The 9-item questionnaire indicates a three-factor structure reflecting ‘interpersonal tasks’, ‘generic attributes’ and ‘profession-specific elements’. Students with greater previous experience of health or social care roles, and students with a more positive attitude to qualification had significantly more advanced scores than their peers. Scores advanced through the curriculum showing step changes after the start of clinical attachments.

Conclusions: The data provides sufficient evidence of validity with student doctors to justify wider data collection.

Context

Professional self-identity is a ‘state of mind’ – identifying one's-self as a member of a professional group. As a student prepares for a professional role they begin to ‘feel’ like a member of the profession. One mechanism for this gradual change in self-identity is legitimate peripheral participation in the activities of the profession (Lave & Wenger 1991). Clinical contact, role modelling, uniform and reflection also play a part (Goldie et al. 2007).

Acquisition of a professional self-identity is by nature subjective. It is affected by social, demographic and personality factors (Chamberlain et al. 2005). Nevertheless, it is important because it is a pre-requisite for accepting the responsibilities and obligations of the professional role and it can be key to developing the confidence to work as a qualified professional in the student's chosen profession (Stockhausen 2005). In other words it is a necessary foundation for professionalism. Delayed professional self-identity is a barrier to successful transition from student to professional (Schwertner et al. 1987).

Current trends in medical education limit student doctors’ legitimate peripheral participation and may retard their evolving professional self-identity compared with other health and social care students. As a first stage to quantifying this potentially serious phenomenon, we set out to develop and evaluate an instrument to measure developing professional self-identity across all the health and social care professions. The instrument is intended as a research tool to understand the curricular features that contribute to the development of professional self-identity. The tool is not intended to analyse the nature or content of professional self-identity, nor is it designed for the purpose of individual assessment. It may however be a useful reflection aid for developmental purposes and this is discussed at the end of the article.

Table

Practice points

Delayed professional self-identity can be a barrier to successful transition from student to professional. We present a generic tool to measure the development of professional self-identity across health and social care students. Our data suggests that the tool is a valid research measure for studying the development of professional self-identity in doctors. It is not intended for individual assessment. We now intend to use the tool to diagnose curricular features that influence the development of professional self-identity.

Methods

The University of Sheffield Ethical Committee reviewed the protocol and granted permission for the study. The mapping, field testing and data collection were undertaken as an undergraduate research project.

Defining content

Definitions of professionalism vary even with a single professional group. We made no attempt to define or elucidate the nature of professional self-identity. Instead, we left it in a ‘black box’ into which respondents might read their own tacit perceptions. We did however structure the instrument to reflect some domains of professional activity, common to the health and social care professions, within which professional self-identity might operate.

The second author examined the Quality Assurance Agency benchmarking statements for curricula in Medicine(Quality Assurance Agency for Higher Education 2002), Nursing (Quality Assurance Agency for Higher Education 2001a), Physiotherapy (Quality Assurance Agency for Higher Education 2001b) and Social Care (Quality Assurance Agency for Higher Education 2001c) and identified nine common themes. The themes were: teamworking, communication, patient or client assessment, cultural awareness, ethical awareness, using patient or client records, dealing with emergencies, reflective practice and teaching.

The relevance of each theme for professional practice was then checked by mapping onto existing professional standards documents. For each health care profession, the professional standards document sets out the core elements of the professional's role before outlining relevant performance standards for each. We took the core headings from ‘Good medical practice’ (General Medical Council 2006) for doctors, ‘The Nursing and Midwifery Code of professional conduct’ (Nursing and Midwifery Council 2004) for nurses, ‘Core standards of physiotherapy practice’ (The Chartered Society of Physiotherapy 2005) for physiotherapists, and ‘Health Professions Council standards of proficiency’ (The Health Professions Council 2007) for other allied health professions. No equivalent national professional standards document existed for social care at the time of the study. The second author and three research students (EJ, GB & CD) independently mapped the nine themes onto the main headings in these standards documents. All four corresponded over their mapping choices for all nine items. No additional themes were identified.

The second author operationalised each theme into a self-statement which allowed the respondent to say how they currently identify themselves along a scale between ‘first day student’ and ‘qualified doctor/nurse/social worker etc.’ when engaged in each professional activity.

Field testing

EJ, GB & CD field tested the instrument with small number of students and practitioners from a range of health and social care programmes on an iterative basis. They re-drafted the instrument after each test until no further changes were suggested.

Forty-four respondents from 10 professions contributed to the field trials. The main alterations to the instrument were as follows:

1. substituted the term ‘patients or clients’ for the term ‘patients’ following trials with social care,

2. substituted the term ‘ethical or moral’ for the term ‘ethical’ because of repeated confusion between ‘ethical’ and ‘ethnic’ and a tendency to restrict the meaning of ‘ethical’ to issues of consent and confidentiality only,

3. substituted the phrase ‘teaching juniors/peers’ for the phrase ‘teaching others’ because many students did not recognise their teaching roles without prompting,

4. altered the sequence of questions to improve comprehension,

5. introduced a ‘not-applicable’ response option for all the scales.

The final version of the questionnaire is appended. In addition to the scales, respondents are also asked to give their gender and their stage or year of progression through their programme.

Evaluation with student doctors

Administration

The questionnaire was administered online. A notice was posted within the online learning environment accessible to all 1334 student doctors at the University of Sheffield, School of Medicine. The curriculum covers four phases of learning over 5 years. Students were not contacted directly, and participation was entirely voluntary. Students were offered the incentive of a professional behaviour credit for their portfolio if they completed a questionnaire.

To aid the evaluation, participants were asked for two pieces of additional information. They were asked to declare any experience of work or training in health or social care before they enrolled on the MBChB course. They were also asked to choose any appropriate adjectives from a list of 19 options to describe their attitude to qualification.

Analysis

Reliability is the precision or reproducibility of a measurement (Crossley et al. 2002). Traditionally, reliability is evaluated by examining the agreement between different respondents (inter-rater reliability), or the same respondent on different occasions (test, re-test reliability), or different elements of the assessment (inter-item variation) or by examining several possible sources of variation is a single generalisability study. However, these approaches are inappropriate for the Professional Self Identity Questionnaire (PSIQ) for the following reasons.

1. There is only one respondent who can meaningfully gauge the self-identity of the subject, and the subject is him or herself.

2. Self-identity is a state of mind and is intrinsically unstable. It is likely to occupy the emotional realm rather than the rational or structural realm. There is no good reason to expect consistency on repeat measurement – even within a short timescale.

3. The items of the instrument are not intended to offer repeat measures of a single domain – rather to contextualise the construct of interest within a range of professional activities.

Validity is the degree to which the measurement reflects what is intended. Here it is possible to propose and test hypotheses which would be expected to follow if the PSIQ provides a valid measure of professional self-identity (Downing 2003). The hypotheses are listed below:

‘If the PSIQ is a valid measure of professional self-identity then …

1. … the internal structure of students’ responses across the nine areas of professional activity will not be random, rather they will reflect expected performance constructs such as interpersonal skills and humanism.’ Exploratory factor analysis (principal axis factor extraction with varimax rotation in SPSS 14.0) tested this hypothesis.

2. … more senior students, who have experienced more training, will respond with higher scores.’ The relationship between year of study and students’ scores are presented for visual interpretation; no statistical test is applied.

3. … more senior students, who have experienced more training, will regard more items as applicable to their self-identity.’ The relationship between year of study and students’ response rates are presented for visual interpretation; no statistical test is applied.

4. … those students who have had more extensive prior experience in health and social care will score more highly.’ Blind to the scores, Crossley categorised the statements of prior experience into three codes (none/minimal work-experience – 1, extensive work experience/voluntary work – 2, actual employment – 3). Approximating these codes as an ordinal categorical variable, Spearman's correlation co-efficient tested the relationship between prior experience in health and social care and students’ scores.

5. … those students with higher scores will have a more positive attitude to qualification.’ Crossley and a fifth investigator (PS) independently coded the 19 adjectives as ‘predominantly positive’ (+1), or ‘predominantly negative’ (−1). The coding agreed in every case. Approximating the attitude balance (positive minus negative) as numeric data, Pearson's correlation co-efficient tested the relationship between students’ scores and their attitude to qualification.

Results

Four hundred and ninety six out of a total of 1334 student doctors (37%) responded to the notice to participate and returned a questionnaire. Response rates deteriorated across the phases of the course from 69% in phase 1a to 9% in phase 4. Because of the very low response rate, we compared phase 4 responders and non-responders in gender and quartile grades. The whole of phase 4 had 61% female and the year's median quartile grade was 4. The responders in phase 4 were 59% female and their median quartile grade for the year was also 4.

Responses yielded a 3-factor solution (Table 1). The latent constructs suggested by these factors are: ‘profession-specific tasks’ (conducting assessments, using records, dealing with emergencies and teaching), ‘generic attributes’ (cultural awareness, ethical awareness and reflection) and ‘inter-personal tasks’ (teamwork and communication). The internal consistency of the questionnaire measured by Cronbach's alpha was 0.93.

Table 1.  Factor structure of responses

Item	Factor 1 loading	Factor 2 loading	Factor 3 loading
Teamwork	0.42	0.39	0.46
Communication	0.31	0.41	0.78
Conducting assessments	0.63	0.21	0.59
Cultural awareness	0.34	0.53	0.43
Ethical awareness	0.20	0.71	0.22
Using records	0.52	0.46	0.31
Dealing with emergencies	0.64	0.23	0.22
Reflection	0.34	0.60	0.26
Teaching	0.55	0.39	0.24

Scores increased progressively across the four phases of the programme (Figure 1). There were no student doctors in phase 2 at the time of the evaluation.

Figure 1. Mean PSIQ score by phase of course for each of the nine themes.

Valid response rates were initially low for all the ‘profession-specific tasks’ and these increased across the phases of the course. However, respondents mostly regarded the generic and interpersonal attributes as ‘applicable’ to their self-identity from the start of the course (Figure 2).

Figure 2. Valid response rates by phase of course.

Responses displayed a modest positive relationship between prior health and social care experience and PSIQ score across all nine elements (Spearman's correlation co-efficient 0.05–0.22). This relationship was strongest, and reached statistical significance at the level of p < 0.05, for ‘teamwork’, ‘conducting assessments’, ‘teaching’ and ‘dealing with emergencies’.

Similarly, responses displayed a modest positive relationship between PSIQ score and attitude to qualification across all nine elements (Pearson's correlation co-efficient 0.07–0.21). This relationship was strongest, and reached statistical significance at the level of p < 0.05, for ‘conducting assessments’, ‘teaching’ and ‘dealing with emergencies’.

Discussion and conclusions

This manuscript takes the view that professional self-identity is a pre-requisite for taking up professional responsibilities. This self-perception is quite separate from any external indicator of professional preparedness but is important in its own right.

Main findings

We describe the development of an instrument to measure professional self-identity, the PSIQ, presenting the process of defining its content as evidence of content validity. We also present the following evidence in support of its validity:

1. The factor structure of responses suggests intuitive latent variables influencing responses.

2. Student doctors with more pre-programme experience of health or social care, and more senior students with more experience of the MBChB course, who might be expected to have a more developed professional self-identity, score more highly using the PSIQ. This is especially the case for the profession-specific tasks.

3. Student doctors in the early years of the MBChB programme, who have limited experience of their profession, regard the profession-specific tasks (conducting assessments, using records, dealing with emergencies and teaching) as least-applicable to their self-identity.

4. Student doctors with high PSIQ scores, especially in the profession-specific tasks, report a more positive attitude to qualification as might be expected from students with a more highly developed professional self-identity.

Strengths and limitations

The PSIQ is simple, but the content is robustly defined and is specifically designed for the purpose of making comparisons across all health and social care professions. This is important for a research tool designed to study the effect of curricular and programme experiences on the development of professional self-identity. This evaluation is limited to student doctors, even though the questionnaire will be used across other health and social care professions. A wider evaluation would support more general conclusions about validity, but we judged that a single-profession evaluation should be conducted first to avoid wasting resources.

The overall response rate is low (37%) which raises the possibility that the respondents are an atypical group that is not representative of the whole cohort. This is a particular risk for the most senior student doctors in phase 4 where the response rate was only 9%. But analysis of phase 4 PSIQ respondents on gender and quartile grades indicate that they were not different from the rest of their year group, at least in these respects.

The authors could not conceive of a method to evaluate the precision or reproducibility of a measurement of professional self-identity. The evaluation provides a wide spectrum of evidence suggesting that the PSIQ is a valid measure of professional self-identity, but each source of evidence is open to critique.

The factor analysis would have provided stronger evidence if the authors had identified the expected latent variables in advance since all factor analysis is liable to retrospective interpretation. Readers must judge whether the variables identified are indeed intuitive or not.

The anchor statements ‘first day student doctor’ and ‘qualified doctor’ are directly related to programme progression so that it is likely that actual progression through the phases of the MBChB course will influence students’ responses over and above their evolving self-identity. This may account for the positive relationship between PSIQ score and phase of the course. However, these were most universal and clear anchor statements that the authors were able to conceive and they did not prevent some phase 1a students from choosing the ‘qualified doctor’ response option and some phase 4 students from choosing responses adjacent to the ‘first day student’ response option.

The positive relationship between professional self-identity and attitude to qualification could be explained by a confounding variable that is positively related to both – for example personality factors. However, even if it is subject to personality factors, professional self-identity probably remains an important pre-requisite for practice.

The pseudo-variables ‘attitude to qualification’ and ‘prior experience’ were derived using statistical approximations. These approximations are described so that readers can appraise their validity.

Conclusions and further work

The PSIQ is likely to provide a valid measure of developing professional self-identity – at least in student doctors.

These measurements also suggest some further interesting hypotheses:

1. Extra-curricular factors are important in the development of the generic and interpersonal elements of professional self-identity since almost all students are able to rate their development in these areas from the onset of the programme.

2. Prior healthcare experience has the greatest impact on profession-specific elements of professional self-identity, teamwork and ethical awareness since these are the elements rated significantly more highly by students with greater prior experience.

3. Confidence in dealing with emergencies and teaching about one's profession appear to be key determinants of students’ attitudes to qualification.

4. The start of clinical attachments (in this case phase 3a) causes a step-change in the development of professional self-identity.

The authors now plan to make longitudinal measurements within cohorts of health and social care students and to use fluctuations in individual students as a means of discovering what curricular and extra-curricular experiences have the greatest positive and negative impact on the development of professional self-identity.

We plan to compare the development profile across heath and social care programmes to diagnose curricular features that influence the development of professional self-identity. We would like to extend our longitudinal study into the early post-graduate years to examine the predictive validity of the professional self-identity developmental profile as a means of identifying strengths and difficulties in transition to professional employment.

The tool is not intended for individual assessment purposes. However, it could be a powerful prompt for self-reflection and self-monitoring (Eva & Regehr 2008) and it would be worth exploring its utility for these purposes.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Additional information

Notes on contributors

Jim Crossley

JIM CROSSLEY is a Consultant Paediatrician and Associate Director of Teaching at Chesterfield Royal Hospital NHS Foundation Trust, and a Senior Educational Research Fellow in the Academic Unit of Medical Education, University of Sheffield. He has published validations of various performance assessment methods and on assessment methodology and psychometrics including the application of generalisability theory.

Pirashanthie Vivekananda-Schmidt

PIRASHANTHIE VIVEKANANDA-SCHMIDT, D.Phil., is a Lecturer in Medical Education at the Academic Unit of Medical Education, University of Sheffield. Her research interests include validation tools, assessment methods and medical ethics including medical professionalism.