A predictive validity study of medical judgment vignettes to assess students’ noncognitive attributes: A 3-year prospective longitudinal study

Figures & data

Table

Practice points

In this prospective study, we investigate the predictive validity of a semi-structured interview for admissions to medical school based on medical judgment vignettes: (1) ethical decision-making (moral), (2) relationships with patients and their families (altruistic), and (3) roles and responsibilities in professional relationships (dutiful). Our semi-structured interview based on clearly defined and scored medical judgment vignettes that focus on the assessment of medical students’ noncognitive attributes establishes a promising protocol with high reliability and evidence of predictive validity of clinical performance over a 3-year period.

Table 1.  Overview of Kohlberg's stages of moral development

Pre-conventional level
Stage 1: Obedience and punishment orientation	Focus on avoidance of punishment by not breaking the rules.
Stage 2: Individualism and exchange orientation	Acceptance of alternative views as right and wrong is determined by what satisfies the individual's particular needs.
Conventional level
Stage 3: Good interpersonal relationships orientation	Meet the expectations of what is right because people expect it as part of mutual interpersonal relationships.
Stage 4: Maintaining social order orientation	Emphasis is on obeying social order, respecting the dignity of all while conforming to the laws of the group or institution.
Post-conventional level
Stage 5: Balance of social contract and individual rights orientation	Conceptualize society in a theoretical manner, stepping back from existing society and considering the relativity of group and individual values with respect to what society ought to uphold.

Table 2.  Means (SDs) and raw scores of medical students (n = 26) on the moral, altruistic, and dutifulness medical judgment vignettes

Medical judgment vignette	Stage 1	Stage 2	Stage 3	Stage 4	Stage 5	Mean (SD)
Moral: Ethical decision-making in medicine	7	6	6	3	4	2.65 (1.41)
Altruistic: Relationships with patients and their families	4	10	8	2	2	2.54 (1.10)
Dutiful: Roles and responsibilities in professional relationships	7	10	2	6	1	2.38 (1.24)
Medical judgment vignettes total score (Min. 3–Max. 15)						7.58 (2.87)

Table 3.  Excerpts of various responses to the ‘altruistic’ medical judgment vignette

Pre-conventional level
Stage 1: Obedience and punishment orientation	‘… I don’t think it's the doctor's role to suggest to people that they go elsewhere. He/she should stick to what they know best, mainly medicine, and that includes describing the limitations of medicine.’
Stage 2: Individualism and exchange orientation	‘… I think that as far as I know, it's always really up to the patient to determine what course they want to go on. If they want to pursue an alternative treatment, then I think that if the doctor has explained everything that he can to them …’
Conventional level
Stage 3: Good interpersonal relationships orientation	‘I think you’re always kind of the moderator on these. Especially in the role of family physician, I think you have to pursue both sides of this … that it's a horrible decision to have to make. Especially for a 15-year-old girl … I think the family doctor could play a role in somehow moderating this …’
Stage 4: Maintaining social order orientation	‘… I guess the family doctor would have to say that if they are confident that the 15-year-old girl and her mother have made sound decisions in their practices, and he/she would have to defend them in their decisions. And that he would probably need to outline the conflict to this governing body because that seems to be of pretty big importance to this case.’
Post-conventional level
Stage 5: Balance of social contract and individual rights orientation	‘The family doctor can offer the treatments that are available through the Western system which in this case seems to be ongoing chemotherapy. So, the family doctor's obligations are to outline the options available, which are to be on chemotherapy or not be on chemotherapy … As a family doctor, I would try to work with the mother and daughter to try and blend the two approaches and find a middle ground they might be comfortable with …, but also helping the father and mother realize that the social benefits to their daughter of good family relations are going to do way more for their daughter's health than either of the two treatment modalities.’

Table 4.  Correlations between ITER items in clerkship and scores from the traditional six judge panel and medical judgment vignettes (i.e. moral/altruistic/dutiful) interview formats

		Medical judgment vignettes
ITER Items (for all seven mandatory clerkship rotations)	Six panel judges total	Moral	Altruistic	Dutiful	Total
1. Fund of knowledge (basic and clinical) and understanding of disease mechanisms	0.55**	0.21	0.35	0.03	0.25
2. Clinical skills: History taking and physical examination	0.20	0.34	0.49*	0.24	0.46*
3. Communication skills: Patients interviews and oral case presentations	0.21	0.39*	0.22	0.23	0.37
4. Record keeping: conscientious case write-ups and progress patient management notes	0.09	0.39*	0.16	0.22	0.35
5. Problem solving consistency and strong patient management skills	0.21	0.47*	0.45*	0.50**	0.62**
6. Professionalism and responsibility: Assumes patient care responsibilities and is conscientious	0.05	0.51**	0.26	0.46*	0.55**
7. Motivation: Self improvement and educational initiative	0.23	0.50**	0.20	0.30	0.45*
8. Relationships: Sensitive to patients and fulfils responsibilities as a team member	0.18	0.42*	0.21	0.19	0.37

Note: *p < 0.05, **p < 0.01.

Table

For the past 5 years, a 45-year-old woman has been suffering from muscle degenerative effects of terminal Amyotrophic Lateral Sclerosis (ALS). Since being diagnosed with ALS, her condition has steadily deteriorated but most rapidly in the last year to the point that she is confined to a motorized wheel chair and is now completely dependent on 24-h nursing care for most aspects of her daily life. Although there is no cure for her condition, it is expected that she will live for at least another 2–3 years, but in that time her health will continue to deteriorate to the point that she will completely lose her ability to speak and will eventually require the use of a life support system to stay alive (i.e. ALS is a disease that will ultimately disable all body movement, and yet a person with ALS can continue to be kept alive artificially). In support of her request to die at home, her medical doctor of 15 years has agreed to set up a self-administered IV system that will increase the rate of her morphine injections over a set period of time. At a certain point, the system will release what will be a lethal injection of morphine to end her life.

Probing Questions:

• Do you think the doctor made the right decision in helping her to commit euthanasia?

• Should the doctor lose his license and, hence, ability to practice medicine?

• Should the doctor be arrested and serve time for his actions?

Table

A 15-year-old teenage girl is suffering from swollen glands and complains of being lethargic. After a number of tests the family doctor and specialists confirm the diagnosis of astrocytoma, grade 4, brain cancer (i.e. advance stage of development). The family is informed that there is no known cure for this type of cancer, but the doctors’ suggest that they can slow the process if they begin to address the disease immediately through ongoing chemotherapy. After a few days, the mother and daughter decide to end the chemotherapy treatments, which they say are leaving the girl feeling constantly sick and disorientated. Instead, the mother and daughter decide to pursue a variety of alternative non-toxic therapies outside of the recognized medical system of practice (e.g. herbology, nutritional modification, vitamin therapy). The girl's father, however, is in direct conflict with his wife and daughter and wants them to return to the original chemotherapy treatment plan. After a frustrating week of family discussions, the father has decided to take legal action against his wife for sole custody of his daughter in support of his decision to get her back into chemotherapy.

Probing Questions:

• What should the family doctor say to the mother and daughter about the medical system's ability to provide care at the cancer treatment centre?

• What should the family doctor say to the mother and daughter regarding their decision to pursue alternative therapies?

• What should the family doctor say to the father when he comes to ask for assistance in pursuing support from the doctor in convincing his wife and daughter to continue chemotherapy treatment?

• The specialists and health authority administration are requesting further information on the patient's care, how should the doctor respond?

Table

A doctor with little experience working in Canada has just asked you for assistance with a patient, and his family, that was checked into emergency for a knife wound about two hours ago. The knife wound is not serious, but the patient and family are being difficult in cooperating with the acting doctor, nursing staff, and the police officers in the treatment of the injury. The 17-year-old male patient is of aboriginal descent and has consumed a considerable amount of alcohol. While at a party on the Reservation, he got into a fight with another young aboriginal male his age and was stabbed twice in the arm before the fight was broken up. The patients’ mother and father also live on this high-needs Reservation and simply want the doctor to patch up and release their son so that they can take him home. They are refusing to lay charges against the other young man and ignore the frustrated police officers’ attempts to answer any further questions. After the patient has been bandaged and treated, the family leaves quickly.

Probing Questions:

• The new Canadian doctor asks you to explain why the aboriginal parents were so reluctant to assist the doctor and police in helping their son out. What do you say?

• The new Canadian doctor asks you to explain what some of the issues are facing the health of aboriginal peoples in Canada. What is your response?

• In a follow-up visit with the patient and his mother and father what should the doctor suggest the family do next?

• Upon further investigation, you find that there is a history of aboriginal patients that come through the emergency room, get various inflictions ‘patched up’, and leave as quickly as possible. What can be done to improve this situation?