Abstract
Background: Although essay questions are used in the admissions process in many medical schools, there has been little research on how applicants respond to essay questions.
Aims: The purpose of this study was to explore how applicants to medical school approach essay questions used in the selection process.
Methods: Qualitative analysis was conducted on 240 randomly selected essays written by individuals applying to a single Canadian medical school in 2007 using a modified grounded theory approach to develop a conceptual framework which was checked in interviews with applicants.
Results: Three core variables were identified: “balancing service and reward,” “anticipating the physician role,” and “readiness.” We described the overall approach of applicants as “taking stock,” writing about their journeys to the selection process, their experiences of the process itself, and about their anticipated future in medicine.
Conclusion: Our findings suggest a disconnect between the approach of the applicants (to “show themselves” and be selected as individuals) and the stated intent of the process (to select applicants based on “objective” criteria). Our findings raise important questions about how applicants represent themselves when applying for medical school and suggest that it is important to understand the applicant's point of view when developing questions for selection processes.
Introduction
Recent studies on the selection processes employed by medical schools have highlighted the importance of establishing and measuring the traits thought to be desirable in aspiring medical students (Norman Citation2004; Benbassat & Baumal Citation2007; Prideaux et al. Citation2011). While many studies have focused on improving the way personal interviews are used in the selection process (Lemay et al. Citation2007), little has been written about the rationale, utility, or effectiveness of the essay question as a selection tool, despite its widespread use. Some have suggested that there is little evidence to support the continued use of the essay (Salvatori Citation2001), while others point out that using essays is a good way to complement selection based only upon academic performance (Gilbert et al. Citation2002). It has also been suggested that writing skills such as those required to write an essay should be included in admissions processes because they are an essential part of healthcare education and practice (Chur-Hansen & Vernon-Roberts Citation2000; Showalter & Griffin Citation2000).
The evidence that exists thus suggests that the admissions essay may be an important part of the selection process, but few authors have considered how applicants approach essay questions, nor how the written form can be used to more effectively select applicants for medical school. Given this relative lack of information in the literature about the role of the essay question in the selection of medical students, this study was designed to examine what applicants wrote in response to a series of essay questions employed in the selection process in a single year at a single medical school. It should be noted that the study was not intended to consider the value of the questions themselves, nor how they were created, nor to develop “better” questions. Our purpose was to examine the applicants’ responses to essay questions with the aim of better understanding how applicants approach questions used in selection for medical school.
Methods
This study was based on the analysis of three different essay questions utilized in the selection process at our medical school. Essay questions were developed by a 10-person Admissions Committee; they were intended to test for the traits of altruism, understanding of professionalism, and reflection/self-assessment, and are presented in . The essay on volunteerism was submitted online at the time of the initial application, and the other two essays were completed by applicants immediately after the completion of the multiple mini-interview (MMI) process, which was implemented at our school as described in a recent paper (Brownell et al. Citation2007). Each essay on volunteerism was marked by four assessors, while the essays on professionalism and reflection were each marked by a single assessor. Marks for the online essay contributed to the score used to decide whether applicants were invited for interview. Marks from the two other essays made up 10% of the score used for the final ranking for admission. This study was conducted independent of the admissions process; analysis was performed after admissions decisions were made and investigators did not have access to any of the scores assigned. Essays for analysis were selected at random from the 417 applicants interviewed and were anonymized before analysis commenced. All applicants gave permission for the use of their written submissions for research purposes. Ethical approval was obtained from our university's Conjoint Health Ethics Review Board.
Table 1. Essay questions
A pilot study conducted on 30 randomly selected essays on “reflection” was conducted to determine whether the material submitted by applicants was suitable for analysis. The essays were analyzed by a single reader (Jonathan White), who employed iterative reading of texts, selection of key words and phrases to capture the central ideas and concepts expressed by the writers, to develop a coding scheme in which the ideas expressed were categorized. This iterative process of reading–analysis–coding was repeated on additional essays until no new concepts were derived, and data saturation was achieved.
Based on the findings of the pilot study, the main analysis was conducted on 60 additional examples of each of the three essays, focusing on the specific topics addressed in each essay. Once a coding scheme had been generated, 10 more examples of each essay were selected at random and compared to the coding scheme to insure that data saturation had been achieved. Thus, a total of 240 essays were analyzed (70 for each of the three essays, plus 30 in the initial pilot study).
All essays were read by two reviewers (Jonathan White and Jocelyn Lockyer). The text of each essay was analyzed using a modified grounded theory methodology. Grounded theory is a qualitative research method designed to develop a well-integrated set of concepts that provide a theoretical explanation of a social phenomenon which is “grounded in the data of experience.” The purpose of this approach was to examine what applicants wrote about in each of the essay questions presented to them, and to derive a theoretical framework to describe the patterns of response observed. The conceptual framework derived using this approach is intended to serve as a way of expressing the relationships between the ideas and concepts generated by data analysis. This study was designed after reading the guidelines and suggestions presented by a number of experienced authors (Given & Olson Citation2003; Cote & Turgeon Citation2005; Kennedy & Lingard Citation2006; Kuper et al. Citation2008). Our process for analysis involved iterative reading/coding as outlined and reflective journaling and charting to record observations on the underlying themes, concepts, and connections in the data as analysis proceeded. A coding structure and a core variable were first developed for each essay analyzed. Essays and the coding structures derived from them were reviewed at intervals by the other authors, who suggested ways in which the coding structures could be refined and improved. Then, using common themes and ideas expressed in answer to the different questions, the data were reviewed across the three essays to look at commonalities and differences in the approach taken by the applicants to the essays. This last step allowed us to develop a conceptual framework which described the observed applicant response. All quotations presented in “Results” are from the essays unless otherwise noted.
After completion of primary analysis and generation of a conceptual framework, interviews were held with a number of applicants who were involved in the admission process in 2007. Applicants were invited to participate by email and two sessions lasting 60 min each were held, with a total of 20 of the 417 applicants; sessions were tape-recorded and transcribed. The findings of our analysis were discussed with these participants to gain another perspective on the role of the essay in the admission process using participant checking/data triangulation. To stimulate discussion, applicants were asked to consider the following question: how did you decide to answer the essay questions?
Results
For the essay on volunteerism, “Balancing Service and Reward” was identified as the core variable. Applicants expressed their understanding of the concept of volunteerism, described their experiences of volunteering and how they had been influenced by it, and related this to their motivation for wanting to become a doctor. Their responses suggested that they were willing to enter a career in order to serve others, but that they also wished to gain personal satisfaction and reward. Most of them described that there is a balance of service to others and personal reward involved in both volunteering and medicine, and that neither are purely selfless activities. The rewards involved were described in terms of tangible effects such as new knowledge, skills, and experience, and also in terms of intangible effects such as satisfaction, intellectual engagement, meaning, and personal growth.
Service given to enrich the life of another results in the enrichment of ourselves.
While I feel that the students that I worked with were able to benefit from the knowledge and perspective I brought, I believe that I learned as much, if not more from my experience.
For the essay on professionalism, “Anticipating the Physician Role” was identified as the core variable. Applicants discussed the concept of professionalism as it related to themselves as individuals and applicants, and as it was likely to relate to them as medical students and physicians of the future. Most wrote about applying themselves to a serious task, meeting obligations and expectations, and obeying rules of behavior. Applicants framed the decision to become a professional as a lifelong commitment affecting all aspects of a person's life. Applicant responses suggested that they were prepared to put their own interests second to concepts and ideas larger than themselves in order to proceed with a career in medicine. There was a clear connection between the ideas about service and reward noted in the essay on volunteerism, and the balance described in this essay between the responsibility and opportunities they anticipated being given as medical students and physicians, and the obligations and rules which they understood to be associated with these roles.
As an applicant to medical school, the statements provided reveal the gravity of the responsibility and the fashion in which an individual should act when accepting this position.
A potential physician must take him or herself seriously with their responsibilities … physicians are expected to … understand the seriousness of their responsibilities that society has trusted upon them.
For the essay on reflection, “Readiness” was identified as the core variable. Applicants wrote about a variety of topics relating to their experience of the interview: their own performance, the interview process itself, their personal qualities, and how the skills they demonstrated might relate to those expected of a medical student or physician. Clearly, many applicants found the interview process a new and challenging experience: some described feeling forced by the process to consider unanticipated aspects of themselves, and described a sense of surprise at being able to cope with unanticipated situations. The interview process was also framed as an opportunity for applicants to test themselves by putting previously developed skills to use. Some skills required related to organization, time management, and creativity (e.g., quick thinking, problem solving), while others related to more innate qualities (empathy, respect, trust). Applicants reflected on the meaning of the performance they observed in themselves, expressing the opinion that through the interview process they had been tested and found “ready to start the road to becoming a doctor.” The idea of foreshadowing was especially relevant here, as applicants linked the tasks demanded of them to the anticipated future tasks of the medical student/physician role. There was also description of a sense of completeness and synthesis (“bringing it all together”), suggesting that participating in the interview process had had an affirmatory effect for some applicants, informing them that they were indeed “on the right path” and were prepared for the tasks ahead.
Though I understood that I have many of the qualities listed above before I attended the MMI, answering questions that called these values into play proved to me that I am able to think clearly and utilize strategies I have developed when placed under pressure.
Being able to work through these challenges has reassured me that I will be able to face the many challenges ahead as a pursuer of a life in medicine.
The thing that was even more instilled upon me during the MMI was that I know that I am ready to start the road to becoming a doctor.
Conceptual framework “Taking Stock”
The conceptual framework generated to explain applicant responses observed across all the essays was given the name “Taking Stock.” We hypothesized that applicants had responded to these questions by “taking stock” of themselves and their lives considering three distinct “frames of reference”: their past experiences (how did I get here?), their current status (where am I now?), and their anticipation of the future (where am I going?). This approach imagines the applicant pausing to reflect at an important moment in their journey to becoming a physician, as they transition from mere applicant to future doctor-to-be, looking back at where they have been, considering where they stand today, and looking forward to what the future holds. The three frames can be summarized as “why do I want do be a doctor? (past),” “will I become a doctor? (present),” and “how will I live as a doctor? (future).” Each “frame” was noted across all three essays, although there was a strong association between particular frames and particular essays. provides examples of quotations for these perspectives. provides a conceptual diagram.
Figure 1. Conceptual framework: taking stock.
Table 2. Representative quotations from the essays
Taking stock: The past
This frame of reference was observed most often in the essay on volunteerism, but was also noted to a lesser extent in the other essays. The types of questions which were addressed in this frame of reference were:
How did I get here?
Why have I applied?
How have my past experiences prepared me?
Applicants discussed why they had chosen to enter medicine, gave an account of the experiences which had brought them to this choice, and outlined the steps they had taken to prepare themselves for a medical career. They also described their past feelings of needing to find meaning, purpose, and satisfaction in serving others which they hoped to address by becoming a physician. This frame was also represented by applicants’ accounts of hard work, sacrifice, and suffering, their sense of having been prepared and tested for entry into a professional career, and their inclusion of stories from the past relating to principles and issues raised during the application process. In some answers, there was a tangible feeling of applicants reflecting upon their past experiences and deriving new meaning from them through the act of writing.
Taking stock: The present
This frame of reference was observed most strongly in the essay on reflection, but was also noted in the other essays. The types of questions which were addressed in this frame of reference were:
What have I learned about myself in the application process?
Do I have the skills I need for this process? Am I properly prepared?
Am I ready to proceed into medical school?
Am I good enough to be chosen? Do I have what they are looking for?
Does the application process reflect what it will be like as a physician?
Within this frame of reference, applicants discussed their current abilities as perceived through the lens of the application process, describing the process as a challenge or a test of their abilities and their preparedness, and often concluding that they had been tested and found ready. They reflected upon their immediate-past performance in the application process and described skills that they considered they possessed and others which they currently lacked. They also described their expectations and experience of the process itself, discussing how it felt to undergo the interview process, asking how it related to the abilities expected of a medical student or physician, and questioning whether the process was indeed capable of selecting the best applicants. The dominant theme in this frame of reference was “preparation-testing-readiness” – many applicants described a sense of satisfaction at having successfully completed the process and finding themselves ready to proceed.
Taking stock: The future
This frame of reference was observed across all the three essays. The types of questions which were addressed in this frame of reference were:
Do I have the skills I need for medical school and medical practice?
Do I have the skills I will need, or can I develop them?
How will I change in medical school?
What will be expected of me as a professional physician?
How will I derive satisfaction and meaning from my life?
Applicants wrote about their anticipation of a career in medicine, and about the hopes and fears associated with this. The essay on professionalism allowed them to discuss their understanding of the obligations and expectations which would be placed upon them as physicians, and to reflect upon their understanding of what it means to be “a professional” for the rest of their lives. Applicants described a career in medicine as “hard” or “grueling,” looking forward to the associated challenge, and wondering if they did indeed possess the skills needed to succeed. There was discussion about whether the skills needed were innate, or if some could be developed in medical school. Applicants also looked forward to the rewards of a career in medicine, describing the associated opportunities for personal development and anticipating the satisfaction associated with being a doctor. There was a sense of reflection and integration in many of the essays, as if applicants were using the written form to describe a growing realization of the commitment that would be expected of them in the future, and of their own readiness to undertake the task of becoming and being a doctor.
We have represented our conceptualization of the taking stock process in which presents the past, present, and future along with the predominant themes and subthemes for that frame.
Participant checking
In the participant checking interviews, applicants confirmed the main findings of the conceptual framework. Applicants expressed the idea that they had approached the essays as a way to “show themselves” and “tell their own story” in a subjective way which they felt was missing from other parts of the admission process (quotations from interviews):
“You need a balance of subjective and objective – That's where the essays come in – you really did get a chance to show yourself” … “I like communicating with writing, so that was nice for me to like write, a story of your life” … “I really liked when we had to write autobiographical essays and stuff (at other schools), because it felt like people could bring out what was most important to them” … “I want to be able to prepare for the serious objective parts, but also get to know me as a person.”
Discussion
Applicants appeared to approach the essay questions as a way to tell their own subjective story; this approach was different from what was intended and stated by the school. Faced with an essay question supposed to be about altruism or professionalism, applicants wrote mostly about themselves. They wrote about why they applied, how they prepared, and how they would like to live as a doctor even when the questions did not ask directly about these things, and they described perceiving value in the more subjective, “personal” components of the selection process. This was an interesting phenomenon to encounter in an era of efforts to increase the objectivity of the admission process (Lemay et al. Citation2007). Having prepared such a statement about their lives to date and their ambitions for the future (presumably in expectation of being asked “Why do you want to be a doctor?”), we hypothesize that applicants may have taken the opportunity of essay-writing to make sure that this material about themselves and their stories were not overlooked. It is possible that this inclusion of personal material would have occurred regardless of the “official” topic of the essay question.
Applicants considered the concepts of volunteerism, professionalism, and reflection as applied to themselves, and framed their responses in terms of their past experiences, their present capabilities, and their anticipation of their future roles as physicians. One work that bears a particular relevance to our findings is that of Niemi et al. (Citation2003) in which 109 students were asked to write an essay on the topic of “my work and future as a physician in the 21st century,” on their first day of medical school. Essays were analyzed using iterative content analysis to identify a number of “representations” including thoughts on personal development and growth, thoughts about the physician as a person, doubts about the future, thoughts about challenge and work, and the changing role of the physician in society.
It is not surprising that applicants to medical school want to write about themselves as part of the admission process, and it is common for the admission process to include questions which require the applicants to comment explicitly on their own qualities and to speak or write about themselves. From the applicant's point of view, the process is highly personal: applicants wish to present themselves to the school in the best possible light, and emerge from the selection process as one of the “chosen few” deemed good enough to enter medical school. In discussing this work with others involved in the field, we have encountered broad acceptance of our findings that there is a “disconnect” between the approach of the applicants (to “show themselves” and be selected as individuals) and the intent of the process (“objective” selection based on the assessment of specific traits). Our study is the first to put a name to this “disconnect,” which appears to be widely understood by those engaged in the admissions process, but has not been named or discussed in the admissions literature. We consider the qualitative methods employed here, especially suitable when exploring the importance of phenomena considered so “obvious” that they are easily overlooked.
It is important that questions used in such a high-stakes process do actually measure what they are intended to measure: mere face validity is not enough (Kulatunga-Moruzi & Norman Citation2002). Our findings suggest that if the essay question is to function as an effective selection tool, those responsible for designing questions should understand the applicant's approach. We suggest that the applicant's perspective should be considered when new essay questions are created, and we encourage question developers to “think like an applicant” when considering the range of responses that questions may receive. We further recommend that consideration be given to how the essay question can be used to allow applicants to demonstrate the attributes desired by the process, as distinct from giving accounts of their personal experiences. We also suggest that questions which ask applicants directly about themselves be treated with special caution. It may also be useful to employ situational questions (“What would you do?”) instead of behavioral questions (“What did you do?”) in the selection process (Klehe & Latham Citation2006). An alternative approach would be to use the essay question as a way of understanding the perspectives of individual applicants, perhaps using their responses as a basis for counseling and guidance at an early stage of the application process.
The applicants interviewed expressed the wish that they could tell their own personal stories and be selected on the grounds of their individual merits, as people. If it is true that applicants write about themselves in the essay question because they fear that there is nowhere else in the process for these details to be considered, it may be helpful to deliberately design selection processes to include an explicit opportunity for applicants to talk about themselves and their strengths. Such an approach might reduce the need for these details to be expressed in other parts of the process. This echoes a point made by Albanese in 2003 that the selection process should retain at least some elements that “place a human touch on what is a highly stressful, high-stakes process for all involved.”
Our school has made significant changes to its selection process as a result of this study. We have completely changed the way that we employ essay questions in selection and no longer use them to ask applicants directly about themselves or their past experiences. Instead, we have developed situational essay questions intended to allow applicants to demonstrate the abilities that they have developed. For instance, applicants are provided with information relating to a common population health problem and asked to summarize the information provided and develop an approach to the problem given.
Although this study was designed to be as rigorous and transparent as possible (Cote and Turgeon Citation2005), the nature of the qualitative method means that its findings are specific to these essays used at this school with these applicants in this particular year, and we encourage others to explore the essays used in the admission process. We intend to continue to study the role of the essay question in the selection process, and to further explore the factors that determine observed patterns of applicant response. We hope that further studies will help us gain more understanding of how applicants approach the selection process, and will, in time, help us develop better ways of selecting medical students.
Conclusions
By analysis of responses to essay questions on volunteerism, professionalism, and reflection utilized in the selection process at a single medical school in a single year, this study demonstrated that applicants appear to approach the essay question from a personal, subjective perspective, writing about their past experiences, their present state, and their anticipated futures. This approach was in contrast to the stated topics that the essay questions were intended to assess, and suggests a disconnect between the perspectives of applicants and those in charge of the selection process. These findings raise questions about how applicants represent themselves when applying for medical school, and suggest that it is important to understand the applicant's point of view when developing questions for selection processes. We anticipate that these findings will be of interest to those engaged in medical school admissions processes, and those charged with developing and assessing essay questions in this context.
Declaration of interest: The authors report no declaration of interest.
References
- Albanese MA, Snow MH, Skochelak SE, Huggett KN, Farrell PM. Assessing personal qualities in medical school admissions. Acad Med 2003; 78: 313–321
- Benbassat J, Baumal R. Uncertainties in the selection of applicants for medical school. Adv Health Sci Educ Theory Pract 2007; 12: 509–521
- Brownell K, Lockyer J, Collin T, Lemay JF. Introduction of the multiple mini interview into the admissions process at the University of Calgary: Acceptability and feasibility. Med Teach 2007; 29: 394–396
- Chur-Hansen A, Vernon-Roberts J. The evaluation of undergraduate students’ written English language skills. Med Educ 2000; 34: 642–647
- Cote L, Turgeon J. Appraising qualitative research articles in medicine and medical education. Med Teach 2005; 27: 71–75
- Gilbert GE, Basco WT, Jr, Blue AV, O'sullivan PS. Predictive validity of the Medical College Admissions Test Writing Sample for the United States medical licensing examination steps 1 and 2. Adv Health Sci Educ Theory Pract 2002; 7: 191–200
- Given LM, Olson HA. Knowledge organization in research: A conceptual model for organizing data. Libr Info Sci Res 2003; 25: 157–176
- Kennedy TJ, Lingard LA. Making sense of grounded theory in medical education. Med Educ 2006; 40: 101–108
- Klehe U-C, Latham G. What would you do-really or ideally? Constructs underlying the behavior description interview and the situational interview in predicting typical versus maximum performance. Hum Perform 2006; 19: 357–382
- Kulatunga-Moruzi C, Norman GR. Validity of admissions measures in predicting performance outcomes: A comparison of those who were and were not accepted at McMaster. Teach Learn Med 2002; 14: 43–48
- Kuper A, Lingard L, Levinson W. Critically appraising qualitative research. BMJ 2008; 337: a1035
- Lemay JF, Lockyer JM, Collin VT, Brownell AK. Assessment of non-cognitive traits through the admissions multiple mini-interview. Med Educ 2007; 41: 573–579
- Niemi PM, Vainiomaki PT, Murto-Kangas M. My future as a physician -professional representations and their background among first-day medical students. Teach Learn Med 2003; 15: 31–39
- Norman G. The morality of medical school admissions. Adv Health Sci Educ Theory Pract 2004; 9: 79–82
- Prideaux D, Roberts C, Eva K, Centeno A, Mccrorie P, Mcmanus C, Patterson F, Powis D, Tekian A, Wilkinson D. Assessment for selection for the health care professions and specialty training: Consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach 2011; 33: 215–223
- Salvatori P. Reliability and validity of admissions tools used to select students for the health professions. Adv Health Sci Educ Theory Pract 2001; 6: 159–175
- Showalter E, Griffin A. Teaching medical students how to write well. Med Educ 2000; 34: 165