The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part II: Organisation & Administration

Figures & data

Table

Practice points

The assessment team would need to adopt new roles and responsibilities when setting up a new OSCE programme. A nominated OSCE lead needs to have an oversight of all aspects of the OSCE programme. An OSCE question bank needs to be developed and maintained in order to have a pool of quality assured and peer reviewed stations for use in various examination sittings. Examiner and Standardised Patient training are important elements of quality assurance and standardisation process. Post-hoc psychometrics provide valuable data for further quality assuring the OSCE questions and the programme as a whole.

Table 1  An example of an OSCE blueprint

Topics	Procedural skills	Clinical examination skills	History taking	Total number of questions
Acute medicine	Q.1			1
ENT		Q.2	Q.4	4
		Q.3	Q.5
Paediatrics	Q.6			2
	Q.7
Geriatrics		Q.8	Q.10	3
		Q.9
Total number of questions	3	4	3	10

Figure 1 Flow diagram for the development of an OSCE question bank.

Table 2  Types of OSCE stations

Station type	Description	Examples	Benefits	Limitations
Observed station	An examiner is present throughout the duration of testing	Communication skills Procedural skills Clinical examination	Direct observation allows assessment of the higher levels of the learning domains. Immediate feedback can be given for formative assessment.	Resource intensive with one examiner needed per station
Unobserved station	No examiner present throughout the testing period. Answers may be submitted on paper either after each station or following the completion of the examination	Interpretation of clinical information e.g. X-rays, pathology specimens, blood results. Prescribing skills. Information Technology Skills.	Examiners not required on such stations.	No direct observation of performance. OSCE station may not be necessary and alternative assessment tools may be as effective at assessing the cognitive skills in question.
Technology enhanced station	A station involving the use of technological advances such as part-task trainers or high-fidelity manikins to assess skills that would otherwise be difficult to assess in the OSCE format.	Intimate clinical examinations with use of part-task trainers, e.g. rectal examination. Complex decision making skills and the management of acutely unwell patients with the use of high-fidelity manikins.	Increases the scope of potential OSCE stations, allowing assessment of learning domains which could not be assessed effectively using traditional OSCE stations.	Personnel must be trained in the use of the equipment. Equipment failure. Initial cost of new equipment and maintenance costs.
Linked stations	Two consecutive stations are based upon the same clinical scenario or information. These may be observed or unobserved.	Observed examination of the respiratory system in the first of two stations. Unobserved documentation of findings and management plan in the second station.	Greater number of skills can be assessed per scenario Efficient use of examiner resources.	Needs careful circuit planning, as candidates can neither start on the second of a pair of linked stations nor end the examination on the first.

Table 3  Guidance for completing the question writing template

Template section	Information required
Station information
Subject/Topic	Chosen as described at the beginning of this section.
Level of candidate	The competencies that need to be demonstrated should be appropriate for the level of training of the candidates.
Competencies to be assessed	As described in part one of this Guide.
Station duration	This should generally be standard for all stations as determined by the OSCE committee as described in the examination length section.
Information for the site organisers
Standardised patient (SP) age and sex	This information is key for the organising team to find appropriate SPs for the stations.
Resources and equipment needed	The organising teams need this information to equip the station and to maintain standardisation across sites if an examination is taking place at multiple sites.
Setting up the station	This information is needed to ensure that the position of the chairs, tables and couches do not interfere with the tasks in questions. For example candidates should be able to approach the SP for examination from the right side and the examiners are able to view the whole process from where they are seated.
Instructions for candidates
What is the scenario (briefly)?	This is the key information for the candidates about the station.
Who and where they (Candidates) are?	It is essential for the candidates to know in what capacity they are having a consultation with the patient, so that they can provide appropriate information to the patients and the examiners.
What has already happened	If the scenario is setup in such a way that a nurse has seen the patient and some information about the blood pressure and temperature etc. could be available to the candidates if they asked for it, then it should be made clear at this stage.
What are the candidates expected to do	These are the skills to be demonstrated by the candidates at a given station, e.g. take a focused history and examine the respiratory system
What are the candidates not expected to do?	If due to limitation of time candidates are not expected to perform General Physical Examination as a part of the respiratory system examination then it should be clarified here.
Supplemental data	If any investigations could be provided to the candidates then it should be included in this section. For instance, ‘if you require the results of any relevant blood tests you can ask the examiners
Template section	Information required
Information for the examiner
Brief background to the scenario	This section is similar to ‘What is the scenario?’ section above.
Examiner's role	It is very important to make clear here, in what way the examiners are allowed to interact with the candidates if at all. Are they supposed to observe only or ask some questions as well?
What are the objectives of the station	Again similar to ‘What are the candidates expected to do?’ section above.
What information they might be able to provide the candidate?	The examiners, if required, can provide information such as reports of blood tests or X-rays.
What information they should not provide the candidate?	The examiner should not volunteer any information candidates are expected to explore for themselves.
Clinical information relevant to the station	If examiners from disciplines other than the one in question are used for assessment then this information should be able to enable them to mark the candidates fairly. This becomes even more important, especially if generic marking schemes are to be used for assessment as described later in this Guide.
Simulated patient information (All the information in this section is essential in maintaining the standardisation of the station, especially if the examination is being conducted at split sites)
Who are they?	As set out in the candidates’ instructions earlier.
Their social/economic background (if applicable)	This needs to be brief, considering the station length and time allowed for the candidates to complete this task.
Presenting complaint and past history	Should be succinct and focused for the purposes of the station.
Details of current health problems and medications	Again this information should be relevant and to the point.
Details of their concerns/perceptions	If a station involves dealing with SP concerns then it is important to standardise this aspect.
What they should say (their agenda) and what they should not say	In certain stations this information would not be needed, for instance where the candidates are expected to examine only.
What they should ask (questions)	The SPs should be allowed to ask standard questions when talking to the candidates.
Specific standardisation issues (specific answers to specific questions, please stay with the script)	This information could be vital if the candidates are expected to ask some particular questions.

Table 4  Comparison of binary checklist and rating scale

Station on history and examination of an asthmatic patient: two possible types of checklist for scoring the examination of the chest
Binary check list	Checklist using rating scales
Candidate performs an examination of the chest 1. Introduction   Yes [ ] No [ ] 2. Obtaining consent   Yes [ ] No [ ] 3. Appropriate exposure   Yes [ ] No [ ] 4. Professional approach   Yes [ ] No [ ] 5. General physical examination   Yes [ ] No [ ] 6. Inspection   Yes [ ] No [ ] 7. Palpation   Yes [ ] No [ ] 8. Percussion   Yes [ ] No [ ] 9. Auscultation   Yes [ ] No [ ] 10. Advising patient at the end to cover up the exposed areas and thank for cooperation.   Yes [ ] No [ ]	Candidate performs an examination of the chest   1 [ ] 2 [ ] 3 [ ] 4 [ ] 5 [ ] 1. Unstructured approach 2. Structured approach but completes less than 50% of key steps. 3. Structured approach but completes more than 50% of key steps. 4. Structured approach and completes a majority of key steps. 5. Structured approach and completes all key steps. Such a check list should then be accompanied by a list of key steps. Uses alcohol rub before and after examination and, when appropriate uses gloves Seeks permission to examine, and explains the nature of examination Offers/Asks for chaperone where appropriate Asks the patient if any areas to be palpated or moved are painful Positions the patient correctly and comfortably, then uses a methodical, fluent and  correct technique Does not distress, embarrass or hurt the patient unduly Examines, or suggests examining, all the relevant areas Completes the task, covers up the exposed areas and thanks the patient
Score awarded X/10
Add the scores and award out of 10

Box 1 Outcomes for examiner training

Learning outcomes for examiner training sessions
To understand the scope and principles of the OSCE examination
To maintain consistent professional conduct within the examination
To understand and use the scoring rubric in order to maintain standardisation
To provide written feedback on performance if required in summative examinations
To provide verbal feedback at the end of station in the formative examinations
To ensure confidentiality of the candidates’ marking sheets
To understand the procedures for inappropriate or dangerous behaviour by candidates

Box 2 Common administrative tasks for OSCE

Common administrative tasks for the OSCE

Allocation of students to examination centres

• If examinations are to be held at multiple sites, planning is required to ensure that wherever possible examiners do not know the candidates and any candidates with disabilities are sent to centres with appropriate facilities.

Transport and reporting instructions

• Candidates must be provided with comprehensive instructions about where to report at the examination centre. In some circumstances transport may need to be arranged for large groups of candidates.

Distribution of paperwork

• Station information, candidates’ lists and mark sheets need to be printed, collated and distributed to all examination sites. Mark sheets should be pre-populated with candidates’ details to minimise time required during the examination.

Selection of standardised patients

• Once equipped with the station information it is necessary to identify appropriate SPs from the trained pool for all stations. Commonly, more than one SP for each station is identified, as fatigue may occur if the station is to be run several times in the day. In addition it is also advisable to invite a number of reserves. They should receive their scripts and reporting instructions in advance.

Selection of examiners

• Once the station information is known appropriate examiners must be selected from the trained pool, taking into consideration the decisions made regarding expert versus non-expert examiners. Reserve examiners should always be invited.

Table 5  Common problems and troubleshooting tips

Problem	Potential solution
Variable performances by SPs affecting station standardisation	Occasionally SPs may change their behaviour between candidates or provide unsolicited information. Robust selection and training procedures should minimise these issues. Examiners should also be aware of this potential problem and be willing to intervene between candidates if necessary.
Equipment failure	There should always be spare equipment readily available at hand. If candidates lose a lot of time waiting for spare equipment it may be possible for them to retake the station at the end of the examination.
Unpredictable behaviour of candidates	Nervous candidates under stress can often act in unpredictable ways. In particular, getting lost in the venue or on the OSCE circuit. Adequate support staff should be available to help direct candidates and answer any queries. Examiners may have to prompt candidates to move on at the correct time if bells or voice commands are missed.
Removal of instructions or equipment from stations by candidates	Instructions can be firmly secured to a table or lectern. Examiners and support staff should be vigilant for candidates leaving stations with equipment
Removal of mark sheets or station information by examiners	This may preclude the station form being used in subsequent sittings and examiners should be warned that no documentation must leave the station. Support staff collecting documentation prior to examiners leaving the station can reduce the chance of this occurring.

Box 3 Examination day briefings

Information for examination day briefings
Candidates briefing
• A description of the circuit including their start stations, rest stations and pilot stations.
• Reminders of rules and regulations
• Quarantine procedures
• Fire procedures
Examiners briefing
• The objective of the examination, e.g. formative/summative
• To check students’ identity at the start of the station
• An overview of the scoring rubric and how to complete the mark sheets
• The importance of keeping stations and candidates’ scores confidential
• Not to talk to the students any more than what is allowed in the script
• To treat all candidates equally
• The procedures for reporting concerns about candidates
• Completing feedback after the examination
• Fire and quarantine procedures
Standardised patient briefing
• The importance of standardisation between candidates
• Their role in providing feedback
• Rest-breaks and refreshment facilities
• Fire procedures

Figure 2 Elements of OSCE quality assurance.

Table

Author

AQ

Subject/Topic

Asthma

Level of the Candidate (choose at least 1)

[x] Year 3

[ ] Year 4

[x] Year 5

Competencies (Essential Field)

Please choose 1 to 3

[x] Clinical Examination Skills

[ ] Communication & Consultation Skills

[x] History Taking

[ ] Hand Over Skills

[ ] Procedural Skills

[ ] Mental Health Assessment

[ ] Professionalism

[x] Application of Knowledge

[ ] Prescription

[ ] Data Interpretation

[ ] Diagnostic Skills

[ ] Management Planning

[ ] Ethics

Station Duration (please do not modify this field)

8 min

Information for the site organisers

SP age & sex

Mark/Mary Freeman, age 32

Resources & Equipment needed

1. Paper and pen, in case the candidate wishes to make notes

2. Alcogel™

3. Couch for the patient

4. Stethoscope

5. Water bottle and glasses

Setting up the station

1. Examiner's chair should be positioned so that he/she can observe faces of both candidate and simulated patient.

2. No desk or table is necessary. If one is present, it should NOT form a barrier between the candidate and the patient.

Table

Brief Background to the scenario

Mark/Mary Freeman is a 32-year-old patient who is attending for their annual asthma review

Examiner's Role

Your role is to observe the history taking process and to assess the candidate's examination of the respiratory system

What are the objectives of the station or what is expected of the candidate?

• The candidate is expected to take a brief, focused asthma history from this patient to assess his/her asthma control and make to suggestions as appropriate

• The candidate is also expected to examine this patient's chest

The candidate has NOT been asked to take a detailed history or perform a general physical examination

What information they might be able to provide the candidate?

• If the candidate attempts to perform a general physical examination, please ask them to move on to examining the chest

• If candidates attempt to perform tactile vocal fremitus and whispering pectoriloquy please ask them to omit these and move on

What information they should not provide the candidate?

• Please do not repeat instructions to the candidate

• Please do not interrupt or prompt candidate or ask any questions

Clinical information relevant to the station

Key points in Asthma history taking;

1. Onset

2. Duration

3. Family History

4. Occupation

5. Smoking history

6. Housing, soft toys

7. Aggravating and relieving factors, e.g. exercise, weather, allergies, etc.

8. Cough

9. Wheeze

10. Perennial versus Seasonal

11. Diurnal variation/night time symptoms

12. Affect on lifestyle

13. Hospital admissions

14. Treatment including steroids

Key Points in Examination of the Respiratory System;

(A good student would complete the examination at the front before moving to the back)

1. Proper exposure

2. Inspection for symmetry and shape

3. Palpation of trachea, chest expansion (vocal fremitus not necessary in this case)

4. Percussion, front and back including the clavicles

5. Auscultation at front and back, (whispering pectoriloquy not necessary in this case)

British Thoracic Society (BTS) guidelines

BTS/Sign Guidance on pets in asthma: There are no controlled trials on the benefits of removing pets from the home. If you haven’t got a cat, and you’ve got asthma, you probably shouldn’t get one.

BTS/Sign Guidance on smoking and asthma: Direct or passive exposure to cigarette smoke adversely affects quality of life, lung function, need for rescue medications and long-term control with inhaled steroids.

BTS/Sign Guidance on control of asthma: The aim of asthma management is control of the disease. Complete control is defined as:

• no daytime symptoms

• no night time awakening due to asthma

• no need for rescue medication

• no exacerbations

• no limitations on activity including exercise

• normal lung function (in practical terms FEV1 and/or PEF >80% predicted or best)

• minimal side effects from medication

Simulated Patient Information

Who they are?

Mark/Mary Freeman, age 32

Their social/economic background if applicable?

• You are currently working in a telesales call centre (trying to sell fitted kitchens)

• You live in your own small terraced house, with your partner and one daughter

• You bought your daughter a cat 4 months ago (for her birthday)

• Your partner smokes 20 a day and you smoke occasionally – when you go out socialising. You know you shouldn’t, but you don’t think an occasional one harms your chest. You’ve been doing this since you were 16

• You have 2–3 drinks (pints of beer or large glasses of wine) when out socialising – which is once a week or less often

• You go to the gym very occasionally

• Your mood is normal (not anxious or depressed)

History

• You have had Asthma most of your life – you think it was diagnosed when you were about 5 or 6

• It seemed to be much worse as a child – you had been under a hospital clinic, but you have never had to stay in hospital

• As a teenager, it seemed to go much better – you rarely seemed to need treatment

• In your adult years, it hasn’t troubled you too much, though you have always liked to have a blue inhaler (salbutamol) handy, just in case

• If asked, you have never taken a course of steroid tablets by mouth and you do not own your peak flow meter

• One inhaler usually lasted you several months, as you only needed it every few days (but see below for current situation). When you use it, you take 2 puffs at a time (as instructed), which relieves the coughing and wheezing noises within 5 min or so)

• You have an elder brother with asthma, a your daughter has mild eczema and your father has had hay fever all his life

• Your asthma is not seasonal

• It does not vary by day or night

Details of their concerns/perceptions

• You also wonder if the cat might be to blame. On the other hand, you had a dog when you were young, which didn’t affect you, so you don’t think you are allergic to animal fur

• You are not really worried about your asthma – you don’t think it's anything serious, but you’re happy to come for a check-up

What they should say (their agenda) & what they should not say

You think the reason why your asthma is worse is because of working longer hours. There is a fair amount of stress at work, as the company's sales are not doing well. You don’t feel that you are getting over-stressed; it's just the general atmosphere at work

What they should ask (questions)

Please do not ask any questions

Specific Standardisation issues (specific answers to specific questions, please stay with the script)

If the candidate asks for your thoughts about why your asthma has worsened very early on (before making any effort to establish rapport), say you believe it's the stress at work. But if the candidate asks the same question when you have established some rapport, mention your belief about it being the stress at work, but also say: It's been ever since my daughter's birthday, I suppose. If they probe why you think that might be, mention the cat.

Table

How easy will it be to find an SP for this question?

Very easy

Relatively easy

Difficult

Table

Are the tasks in this station achievable in 8 Minutes?	Yes	No
Is this topic taught in curriculum?	Yes	No
Where is it taught in curriculum?
The attributes being tested are expected of a FY1 doctor.	Yes	No
Would this question discriminate between good and poor students?	Yes	No
Would this station be able to recreate an atmosphere close to real patient encounter?	Yes	No
If a candidate passes this station would that be able to extrapolated to competence in workplace?	Yes	No
Does this question test what it is supposed to test – as outlined in primary and secondary competences?	Yes	No

Table

Would this question benefit from piloting if unsure about any of the above?	Yes	No
Can this question be used in other years as well?	Yes	No
If yes please indicate which year