Construction of educational proficiency in academia: peer review of educational merits in academic recruitment in Sweden

ABSTRACT

Little is known about the nature of educational proficiency in evaluation practices in academia. This is unfortunate, since excellence in teaching is increasingly seen as significant for the prosperity of contemporary higher education institutions. This study explores the meaning ascribed to educational proficiency in the recruitment of academic teachers, and draws on data from a comprehensive research-intensive university in Sweden offering educational programmes within a broad range of scientific domains. The data consists of 100 evaluation reports derived from 54 appointments. A thematic analysis and an analysis of reviewers’ strategies of intertextuality are carried out. The findings show that (1) although qualitatively distinct aspects are attributed to educational proficiency, the principal meaning ascribed to it is the mere act and experience of teaching; and (2) reviewers draw on the application files using distinct textual strategies that have significant consequences for their construction of educational proficiency. These strategies involve referring to existing information in the file in different ways (quoting/listing, paraphrasing/summarising, commenting/assessing), stressing the absence of information in the file (wanting), and disregarding information in the file (withdrawing). Reviewers’ dominant use of paraphrasing or summarising as a strategy of intertextuality is consistent with educational proficiency primarily being conceptualised as a matter of quantity.

KEYWORDS:

• Academic recruitment
• assessment
• educational proficiency
• intertextuality
• peer review
• teaching excellence
• tenure and promotion

Introduction

Successful recruitment of academic teachers has long been regarded as a crucial factor for the prosperity of higher education institutions (HEIs) (Henningsson, Jörnesten, & Geschwind, 2017). Academic teachers are one of the most essential resources for promoting student learning, and the recruitment of excellent teachers has been pointed out as an important part of the development of higher education (Schwartz & Westerheijden, 2004). In recent years, excellent university teaching has furthermore become successively embedded in a discourse of accountability and ranking (Fumasoli, Goastellec, & Kehm, 2015). The recruitment of outstanding and excellent teachers has consequently gained further importance, and many countries have developed a legal framework to rule these recruitment processes (Fumasoli & Goastellec, 2015).

Peer review of academic quality and excellence has been at the centre of meritocratic selection processes in academia for many years (Merton, 1973; Polanyi, 1962). Understood as part of an academic reward system, these evaluation practices indicate where academics should invest their time (Soudien & Gripper, 2016), as they reflect what kind of work is valued and meritorious (O’Meara, 2002). Since “every act of evaluation expresses a communal value-system, and every act of evaluation goes towards building up that value-system” (Thompson & Hunston, 2000, p. 6), peer review as an evaluative practice is expedient for exploring the beliefs and values of various forms of excellence in academia.

By drawing on empirical data from a Swedish case, this article develops knowledge about educational proficiency¹ as an object of evaluation in the recruitment of senior lecturers.² As such, the article aims to contribute to a wider understanding of educational proficiency in academic recruitment and promotion.

To explore the manifestation of educational proficiency in peer review, external peer reviewers’ written assessments (henceforth “evaluation reports”) are analysed. Reviewers’ evaluation reports reflect in written form the standard of the norms and values that organise academia. Furthermore, they have a profound impact on the outcome of the appointment process, and thus constitute an expedient source for exploring conceptions of educational proficiency. In this article, peer review is seen as an articulatory practice through which educational proficiency is discursively given meaning using different strategies of intertextuality. Such an approach implies that evaluation reports both reflect and shape perceptions of educational proficiency. The empirical work of this paper is guided by two research questions:

1. What is articulated as educational proficiency in peer review in the appointment of senior lecturers?

2. How are strategies of intertextuality used to articulate educational proficiency?

The first question explores what is articulated as educational proficiency, as manifested in the reports, while the second question refers to how educational proficiency is articulated. These two dimensions, what and how, are interdependent insofar as different strategies of intertextuality convey distinct conceptualisations of educational proficiency.

This introduction continues with some brief notes on the national context and a short review of the literature highlighting the conceptual and empirical contributions that inform and frame the current study, in addition to the discursive approach. Next, an account of the material and the methodological approach are provided, followed by a presentation and discussion of findings.

The recruitment of senior lecturers in Sweden

Swedish senior lectureship was established in the middle of the 20^th century in response to an increased flow of students. A senior lectureship is a faculty position with teaching as the primary work duty (Högskoleverket, 2006); although the division of labour varies across disciplines and universities, the emphasis on teaching usually remains. Based on general Swedish labour market legislation, in practice, all faculty members are permanently employed. Thus, recruiting a senior lecturer today basically implies tenure. Over the last 10 years, lectureships have increased by 41%. At present, senior lecturers constitute the largest group (29%) of all research and teaching staff in Sweden (UKÄ, 2019).

The use of peer review in academic recruitment in Sweden has been an institutionally established practice since 1876. Its original purpose was to maintain an unbiased evaluation and promote inter-academic cooperation (SOU 1973:54). Even though changes have occurred at some HEIs regarding the status of the peer review system, reviewers’ evaluation reports still have a significant gatekeeping function in the process.

The entire recruitment process includes several measures. Figure 1 provides a brief overview of the main stages in the appointment process at the university in question (henceforth the University), highlighting the stage under scrutiny in this study. The final decision of whom to hire lies with the head of the department.

Figure 1. The process of appointing senior lecturers at the University

A PhD degree and a demonstration of teaching expertise, including 10 weeks of mandatory teacher training for higher education, are required for eligibility as a senior lecturer at the University. When comparing and ranking candidates, consideration should be based on these two criteria. As much attention should be given to the assessment of educational expertise as to the assessment of research expertise; however, in the final ranking, the different assessment criteria may be weighted differently depending on the conditions defined in the announcement. As occurs elsewhere (see e.g. Weiser, 2012), the question of the qualification of peers is incessant. Normally, at least two external peers who are particularly knowledgeable in the relevant subject area are selected and appointed by the faculty board. The principle of selecting peers who are knowledgeable in the subject area goes back to the time when the national peer review system was first established. It rests on the idea that reviewers’ expertise is based on the relevant discipline or subject, and that the legitimacy of the evaluation is similarly based on the discipline (Askling, 2007). However, due to the expansion of academic positions and a still-growing number of review tasks for academics (Langfeldt & Kyvik, 2011), it is becoming increasingly common for reviewers to be expected to assess candidates who are, at best, in “overlapping”, “related”, or “adjacent” fields to that of the reviewer (Kaltenbrunner & de Rijcke, 2019, p. 873). Furthermore, consideration must always be given to the possibility of calling in a special reviewer to assess teaching expertise exclusively. These reviewers may belong to the same discipline as the candidate, but that is often not the case. However, many of them have something in common: they have completed a national course in the assessment of educational proficiency, which is offered to academics who want to develop their skills in this area.

All reviewers are commissioned to work independently, base their assessments on the candidates’ application files, and provide a written account of their assessment. The evaluation reports regularly include: 1) an assessment of each of the candidates’ eligibility; 2) the selection of a top group; and 3) a ranking of the top candidates. The first two sections may be co-authored at some faculties, but the ranking must be executed individually.

Research on the assessment of educational proficiency

Due to the extensive national and international literature on academic development, scholars in the field now have considerable knowledge about how educational proficiency can be perceived. The literature on this topic ranges from various conceptualisations of the phenomenon (e.g. Booth & Woollacott, 2018; Boshier, 2009; Boyer, 1990; Hutchings & Shulman, 1999; Kreber, 2002; Shulman, 1986, 1993) to imperatives of how to evaluate and reward excellent teaching performance (e.g. Braxton & Del Favero, 2002; Burkill, 2002; Colbeck, 2002; Fairweather, 2002; Gibbs, 1999; Glassick, Huber, & Maeroff, 1997; Huber, 2002; Paulsen, 2002; Ramsden, Margetson, Martin, & Clarke, 1995; Trigwell, 2001; Wahlén, 2002).

The literature on educational proficiency reflects a conceptual change from teaching skills to the Scholarship of Teaching and Learning (Levander, Forsberg, & Elmgren, 2019). This change comprises a shift in emphasis from action to interaction, from individual to collective teaching activities, from internal to external activities, and from an intuitive to a scholarly informed approach to teaching. That is, the focus has now moved away from the internal teaching activities that go on in the classroom. Instead, there is emphasis not only on how the teacher develops her or his own teaching practice through teacher training education, collaboration with colleagues, peers, and/or students, self-evaluation, and so forth, but also on how the teacher contributes to developing the teaching practice of colleagues and the teaching and learning environment in the department. This change stresses sharing one’s experiences and advanced knowledge of teaching and learning via national and international conferences, publications in scientific journals, and so forth. In terms of evaluation, this change has involved a shift towards relying on teaching portfolios that include documentation and reflection on teaching experience, a teaching philosophy, testimonials from peers and employers, teaching materials, and so forth, in addition to trial lectures and student course evaluations (Seldin, Miller, & Seldin, 2010). Thus, rather than merely assessing a teacher’s “demonstrated proficiency”, the teacher’s “documented achievement” is evaluated (Magin, 1998). Nevertheless, it should be noted that critical voices have addressed the challenges that lie in evaluating successful teaching by means of a teaching portfolio (Berk, 2005; Weiser, 2012).

Although various concepts, criteria, and measurement instruments for the assessment of educational proficiency have been developed over time, studies exploring how educational proficiency is assessed in academic recruitment peer review remain scarce, and there is little consensus on what criteria to use (Altbach, 2015; Lucal et al., 2003; Meizlish & Kaplan, 2008; Subbaye, 2018). Previous work focusing on the assessment of educational proficiency in academic recruitment has shown that less weight is given to educational proficiency than to research proficiency (Altbach, 2015; Chalmers, 2011; Parker, 2008; Schimanski & Alperin, 2018), except in certain HEIs with a specific academic profile (see e.g. Subbaye & Vithal, 2017). Others studies show that educational proficiency is mainly being assessed in terms of quantity (Levander et al., 2019), and that reviewers have a greater impact on the outcome of assessments than the assessment regulations and criteria do (Gustafsson, 2014; Levander, 2017). Prior research also indicates that reviewers who have been specially assigned to exclusively assess educational proficiency tend to articulate and assess a somewhat wider notion of educational proficiency than other reviewers (Bolander Laksov, 2018). This implies that reviewers’ qualifications and distinct objects of evaluation play a significant role in the gatekeeping practice.

In sum, while the academic development literature on educational proficiency is vast, research on how educational proficiency is manifested in academic recruitment and promotion is limited. This article aims to bridge this gap in the literature and contribute to the understanding of educational proficiency as an object of evaluation.

Material and method

This study is part of a research project studying the evaluation of academic competences in the recruitment of teachers in academia in Sweden. The empirical data are derived from an old comprehensive research-intensive university in Sweden, which offers educational programmes within a broad range of scientific domains. The data include a sample of 100 evaluation reports that correspond to 54 instances of recruitment of senior lecturers across the University during 2012. This sample was drawn from a total of 162 evaluation reports and 75 recruitments during that year. To obtain a fairly even sample distribution, disciplinary domain was used as the first division of the data. Within each domain (Humanities and Social Sciences, Medicine and Pharmacy, and Science and Technology), approximately 60% of the evaluation reports were then included in the sample based on a random selection. It should be noted that, despite this sampling rationale, this particular study does not focus on disciplinary differences.

Of the sample, 37 reports are written by reviewers affiliated with foreign universities, and 63 by reviewers affiliated with various Swedish universities, with seven of the latter reviewers being employed at the University. That is, 93% of all reviewers are external to the University.

In 11 cases, the first two parts of the reports were co-authored, while the final ranking was written individually. Of the 100 peer reviewers, 62 were assigned to evaluate both research and educational proficiency (henceforth Rev), 28 had a special assignment to pay extra attention to the assessment of educational proficiency (henceforth Spec_Edurev), and 10 were asked to assess nothing but educational proficiency (henceforth Only_Edurev).

Table 1 shows the distribution of the number of cases, evaluation reports, and applicants. The analysis is based on the sample.

Table 1. Number of cases, evaluation reports, and applicants (total population in parenthesis)

Recruitment cases	Evaluation reports	Sample reports %	Applicants	Applicants/case mean
54 (75)	100 (162)	62%	402	7.4

Although the data is derived from a Swedish context, since many of the reviewers are affiliated with international institutions, the evaluation reports reflect a broader cross-national, and science-dependent culture of evaluating educational proficiency. Thus, the comprehension of educational proficiency comes from academics representing a wide range of institutions within and external to the national context.

Ethical considerations

Unlike the practice in many other countries (see e.g. van den Brink, 2010), reviewers’ evaluation reports are subject to the principle of public access to official records in Sweden, and are therefore available for anyone to read. As a result, researchers do not need to apply for authorisation to study them, which provides a unique opportunity to study these processes first-hand. All data were collected and stored according to the Swedish Research Council (2017) national guidelines. None of the information in the reports is covered by secrecy regulations.

Although the reports are official documents, it is the researcher’s responsibility to protect the integrity of those involved and to secure the confidentiality of their personal information (Swedish Research Council, 2017). For this reason, any information that may expose the identity of reviewers or candidates has been excluded from the presentation of findings.

Importantly, the aim of this study is not to value candidates’ merits per se, or to evaluate or reconsider the reviewers’ assessments. Rather, it is to develop current knowledge about a certain object of evaluation and the evaluation practice at hand.

Data analysis

Variables were initially registered, including the number of reviewers and candidates, affiliation of the reviewers, and number of pages written. Next, in order to address the first research question, I undertook a thematic analysis (Braun & Clarke, 2006) of articulations of educational proficiency following a deductive approach. This step involved a cursory reading and coding of reviewers’ articulations in NVivo 10 software. As a framework for the coding, I used 10 qualitatively distinct themes representing different aspects of educational proficiency developed in previous work on the recruitment of full professors (Levander et al., 2019). These were: teaching, PhD supervision, management of education, development of education, commissions of trust, scholarly interaction, service, teaching philosophy, teaching training, and testimonials and recognition. Two of these (teaching and PhD supervision) refer to teaching and situations where the candidate may directly influence students’ learning. Two other themes (management of education and development of education) refer to the overall formation of education and interaction with close colleagues. As such, they refer to activities that go beyond the act of teaching and direct teaching-learning activities. Three themes (commissions of trust, scholarly interaction, and service) focus on activities involving communication with actors external to the candidates’ own department or institution. The last three themes refer to participation in various teacher training activities (teacher training); testimonials from students, colleagues or employers, along with different types of awards (testimonials and recognition); and reflection on one’s own teaching and students’ learning (teaching philosophy).

Using these themes as a basis for the coding served as a conceptual framework. However, I used an open approach to consider whether this specific data would alter the predefined themes in any way. This methodological approach also made it possible to test the durability of the framework and permitted comparisons across faculty positions.

To address the second research question – that is, how educational proficiency is articulated – I explored how reviewers draw upon application files when articulating educational proficiency, using the strategies of intertextuality first developed by Chen and Hyon (2005). This step involved an analysis of how the content of the themes was articulated by the reviewers. At first, I used Chen and Hyon’s (p. 161) three strategies: (a) quoting/listing, (b) paraphrasing/summarising, and (c) commenting/assessing. Quoting/listing is primarily used to present information from other sources again without explicit interpretation. This strategy establishes a clear linguistic link between the current text and previous texts, making it the most obvious intertextual strategy. Furthermore, it creates the impression that the reviewer is unbiased and objective, which in turn affects the reviewers’ credibility (Chen & Hyon, 2005). When reviewers are paraphrasing/summarising, they reorganise or rephrase the information in their own words. One of the advantages of this strategy is the concision by which a range of activities described in the application file can be reduced to a sentence or two (Chen & Hyon, 2005). Commenting/assessing refers to articulations in which the reviewer explicitly provides an opinion about a certain activity. Hence, this strategy involves more explicit values than the others. During the data analysis, I identified and developed two additional strategies: d) wanting and e) withdrawing. Wanting refers to reviewers commenting on, asking for, or expecting merits that are not accounted for in the application file. This may be viewed as the reviewers constructing educational proficiency based on their own expectations, and their perception of educational proficiency then being mirrored in their review of the merits that are present or not present in the applications. However, policy stipulations may be another reason for this strategy. Articulations such as the following would support the latter assumption: “The job description also lists teaching and examination in [subject] at doctoral level, but so far NN has neither teaching nor supervising experience at doctoral level” (ER7). In this sense, the wanting strategy illuminates a stronger link between evaluation reports and policy documents than the other strategies do. Withdrawing refers to reviewers omitting to provide an assessment of educational proficiency. This is an inverted strategy of intertextuality, in that reviewers relate to the information in the application file by not relating to it. Hence, the reviewer withdraws from assessing the candidate.

Findings

Articulations of educational proficiency

In this section I present each theme as they were articulated by the reviewers in this study.

Teaching is a broad and comprehensive theme that includes reviewers’ articulations regarding candidates’: amount of teaching in general; breadth, depth, and relevance of teaching experience; experience with various teaching methods; supervision of students; examination of students; and the integration of research into teaching (the research-teaching nexus). Within this theme, the reviewers place the most emphasis on the amount of teaching in general, supervision from undergraduate and master’s level, and experience with various teaching methods, and the least emphasis on the research-teaching nexus. Interestingly, reviewers who were appointed to assess nothing but educational proficiency (Only_Edurev) tend to articulate the research-teaching nexus in slightly different ways, depending on whether or not they are knowledgeable in the discipline at hand. Reviewers who are knowledgeable in the discipline comment on the candidates’ pedagogical-content knowledge, whereas those who are not comment on whether the candidates base their teaching on the Scholarship of Teaching and Learning literature.

The reviewers refer to PhD supervision interchangeably as an aspect of educational proficiency, of research, or of academic leadership/administration. The researchers commonly focus on the number of doctoral students the candidate has supervised. A fifth of the reviewers do not mention PhD supervision at all.

The themes management of education and development of education refers to activities that are more peripheral to teacher-student interaction, but are still related to the actual teaching situation. Management of education refers to course or programme responsibility, or to being the director of studies. The reviewers articulate management of education more frequently than development of education, which can be expected, since management of education (primarily course responsibility) is regarded a common and important task for a senior lecturer. In development of education, the reviewers place more emphasis on course development than programme development, which also seems plausible, since programme development may be seen as a task for more senior academics.

Commissions of trust refer to serving the scientific community, such as acting as a faculty opponent or dissertation examiner. The reviewers do not frequently articulate commissions of trust as an aspect of educational proficiency in this material; this may be related to the fact that a lectureship is a more junior position than, for example, a professorship, so the candidates may not have had that experience yet.

The theme Scholarly interaction is articulated by the reviewers in terms of sharing one’s experiences and knowledge, primarily within academia. It includes sharing one’s educational or scientific knowledge and one’s innovations through various activities (e.g. textbooks, publications on higher education teaching, and pedagogical or academic conferences).

Service to society comprises articulations about activities such as teaching in contract education or continuing professional development, the production and dissemination of popular science, and collaboration with society.

Teaching philosophy refer to the candidates’ own teaching philosophy and vision of teaching. This theme includes explicit valuations to a greater degree than the other themes. However, these are usually short and not fully elaborated. The reviewers seldom discuss how the philosophy is manifested in the candidates’ teaching.

Teacher training is primarily articulated with reference to participation in shorter or longer training activities. In many of their articulations of this theme, the reviewers display widespread compensatory thinking, in that they seem to consider other experiences (primarily extensive teaching experience) to compensate for a lack of mandatory teacher training.

The last theme, testimonials and recognition, include reviewers’ articulations about student course evaluations and testimonials from employers and/or peers, as well as different kinds of awards related to teaching. The articulations primarily refer to whether students’ voices are included or not, and whether these voices are positive or negative. The reviewers do not comment extensively on how the more critical student voices are dealt with, but seem to regard positive testimonials and awards per se as evidence of teaching quality.

Figure 2 illustrates how the emphasis between different themes varies, based on the number of reviewers articulating each theme.

Figure 2. Share of reviewers articulating the themes (N = 100)

Taken together, teaching, teacher training, management of education (primarily course responsibility), testimonials and recognition (chiefly student course evaluations), and teaching philosophy seem to be regarded as the prime evidence of educational proficiency in the recruitment of senior lecturers. The reviewers place emphasis on activities that are closely related to the act of teaching, rather than on interactions with the academic community or society at large.

Articulation through strategies of intertextuality

Policy documents, including the job description, may be seen as a guiding framework for what the reviewers should look for in the candidates’ application files, and the application files contain the alleged merits on which the assessment is to be based. The reviewers draw upon these files in different ways when assessing educational proficiency using different strategies of intertextuality.

Quoting and listing merits – a common strategy

Quoting/listing is a frequently used strategy. The reviewers use quoting less than listing, and use the former most often to cite either testimonials included in the application file (testimonials and recognition) or the candidate herself/himself, as in the two following quotes:

A Belgian colleague’s letter of recommendation states that NN is ”a highly competent and motivated teacher,” who ”makes effective use of the [local] learning platform.” (ER7)

It is not easy to speak of NN’s educational merits, based on the application file. He states that he has “attended a number of advanced pedagogical workshops to advance my teaching aptitude.” I mean to say that he thereby has acquired knowledge equivalent to those provided in teacher training. (ER20, author’s translation from Swedish)

The reviewers also use quoting to criticise (or praise) a specific argument. On occasion, it is used to justify a specific stance or judgement, such as when quoting requirements in the appointment regulations or a letter of recommendation.

The reviewers use listing in many themes, primarily when enumerating various activities the candidates have performed. It is most frequently used when listing courses taught (teaching), and teacher training courses (teaching training), as in these typical quotes:

She has taught courses in [subject], [subject] and [subject] (whatever that may be) – in [subject] and [subject] on several levels at the universities of [foreign city] and [foreign city]. (ER8)

He has attended ”Academic Teacher Training Course step 1 and 2” (four weeks) ”Supervising PhD Students” (three weeks) at [The University] (ER9-10, author’s translation from Swedish)

Paraphrasing or summarising merits – the most common strategy

The reviewers articulate most themes through summarising. To subsequent readers (e.g. recruitment committees), doing so appears less objective than quoting/listing, as it filters the information from primary sources through an extra lens. The following two quotes are typical examples of summarising:

NN has extensive teaching experience both at the graduate and undergraduate level. (ER19)

NN has teaching experience on different levels. He was teaching assistant for two different courses at the University of [city] and was responsible (meaning he taught the course) for a distance education undergraduate course. These courses were in [subject]. (ER63)

Although more elaborated articulations can be found, it is not uncommon for a number of pages in an application file to be reduced to a summary of a couple of lines in the evaluation report. The insertion of adjectives like “extensive” in the first quote is quite common and puts an evaluative spin on the summary. However, the reviewers seem to use such adjectives almost instrumentally when describing candidates’ experience of something as, for example, “broad”, “extensive”, or “limited”. Such articulations are considered to be paraphrasing/summarising because they appear to be matter-of-fact statements.

Commenting or assessing merits – a less frequently used strategy

Commenting/assessing is the most evaluative strategy; the reviewers use it to provide their opinion about something and to comment on a candidate’s abilities at a personal or professional level. This strategy is commonly used in relation to teaching philosophy. In their articulations, the reviewers predominantly refer to generic teaching qualities by describing the candidate as “reflective”, “competent”, “clear”, “structured”, or “devoted” to teaching, or as having demonstrated sufficient features of their ability to become an excellent teacher. Some reviewers also comment on the candidate’s teaching philosophy itself as being “well thought through” or “demonstrat[ing] educational interests”. Although all these aspects per se are seen as evidence of good quality, the reviewers seldom elaborate on how or in what way this is evidence of quality. Some reviewers comment on the teaching philosophy in a negative way – for example, when stressing that it is “too short” or “general”.

The special reviewers (Spec_Edurev or Only_Edurev) commonly write the most elaborated articulations, in terms of providing explicit value judgements. To a greater extent than other reviewers, the special reviewers also tend to comment on if the candidates’ teaching philosophy is based on academic development literature or if their teachings are subject-based.

Wanting: inquiry about absent or desired merits – a fairly common strategy

Unlike the strategies that were based on the work of Chen and Hyon, I identified this strategy during the process of analysis (see Method section). Fully a third of the reviewers employ wanting as a strategy, with the most wanted aspects being teacher training, PhD supervision, teaching, and testimonials and recognition (student course evaluations). On the whole, this finding corresponds to the emphasis between themes presented above.

Similar to quoting/listing and paraphrasing/summarising, the great majority of wanted aspects have a “what” character; that is, the reviewer focuses on what has been done or not rather than how it was done or not.:

He has not yet been opponent or served in a PhD examination committee. (ER9, author’s translation from Swedish)

However, in a few examples, the reviewers request more than just an enumeration of activities, and ask for more elaborated descriptions and reflection. The following quote, which refers to teacher training, illustrates this:

NN has joined pedagogical education, but she does not tell about the course content and she does not reflect about what she learned during the courses and how she has used the new knowledge in her pedagogical work. (ER79, Only_Edurev)

It is noteworthy that teacher training is the most wanted aspect. Teacher training is also the second most articulated theme, and this connection between the theme and this strategy makes it one of the most dominant aspects of educational proficiency on the whole.

Withdrawal from assessing educational merits – an inverted strategy

Omitting an assessment of educational merits is referred to as withdrawing. As stated above, withdrawing is an inverted strategy of intertextuality, in that the reviewers withdraw from the responsibility of providing an assessment. Consequently, these reviewers do not explicitly convey a conception of educational proficiency. These three examples, all written by reviewers affiliated with foreign universities, demonstrate distinct ways in which this strategy is employed:

I see that he is also teaching for [The University], so you are in a position to assess his capabilities directly. (ER82)

I consent to the statement and conclusion of [the other reviewer]. (ER38, author’s translation from Norwegian)

Pedagogical education

Candidate does not supply this information, but I assume that he has appropriate qualifications. (ER70)

The first reviewer clearly has read (at least parts of) the application file but does not seem to comply with the stipulations to assess educational proficiency. The second reviewer also seems to have read the application file but omits actually providing an assessment by simply agreeing with the other reviewer. The last quote comes across as a bit startling, as the reviewer – seemingly based on nothing – merely assumes that the candidate has the required qualifications – an assumption that would hardly ever be considered legitimate when assessing research qualifications.

Furthermore, withdrawing seems to be prompted by the designation of special reviewers to pay extra attention to the assessment of educational proficiency, as suggested by the following quote from another reviewer affiliated with an international institution:

I weighted the academic credentials more strongly, with the understanding that [the other reviewer] would be conducting a more thorough evaluation of the pedagogy. (ER86)

In this particular case, the announcement stipulated that educational and research proficiency should be given equal attention and equal weight in the final individual ranking.

The relationship between themes (what) and strategies (how) – a short summary

The analysis shows that paraphrasing/summarising is the most frequently used strategy, followed by quoting/listing (Table 2). Most themes are articulated by means of these two strategies; that is, the greater part of the candidates’ merits are summarised or listed. Commenting/assessing is used more seldom, and mainly in relation to teaching philosophy or testimonials and recognition. Wanting is used in relation to teacher training, PhD supervision, experience from teaching, and testimonials and recognition (primarily student course evaluations), while withdrawing does not relate to any specific theme.

Table 2. The relationship between themes and strategies

Theme Strategy	Quoting/listing	Summarising/paraphrasing	Commenting/assessing	Wanting	Withdrawing
Teaching	x	x		x
PhD supervision	x	x		x
Management of education		x
Development of education		x
Commissions of trust	x	x
Scholarly interaction	x	x
Service	x	x
Teaching philosophy	x	x	x
Teacher training	x	x		x
Testimonials and recognition	x	x	x	x

Taken together, the findings demonstrate that educational proficiency is articulated through a summary of teaching-related activities and credentials, and that teacher training education, extensive teaching experience, PhD supervision, and positive student course evaluations are perceived as the prime evidence of educational proficiency.

Discussion

A restricted notion of educational proficiency

At first sight, educational proficiency seems to be ascribed a rich and varied content through various qualitatively distinct themes. However, by the strategies of summarising and listing and an emphasis on the internal act of teaching, this articulatory practice produces a restricted notion of educational proficiency. Individual activities closely related to the act of teaching are brought to the fore, as are teacher training education (albeit without value for the final ranking) and positive student course evaluations. Since the articulations lack explicit value judgements, educational proficiency stands out as being equal to experience. These findings are similar to previous research (see e.g. Levander et al., 2019; Levander & Riis, 2016). Experience per se may indeed indicate evidence of quality; the rationale could be that “practice makes perfect”, or that a dysfunctional teacher would not repeatedly be assigned further teaching. However, the reviewers do not explicitly argue for this logic, which is why it is difficult to discern the line of argument for what (dis)qualifies a candidate when quoting/listing or paraphrasing/summarising are used. This in turn aggravates the work for subsequent readers, who often need reviewers’ explicit reasoning as guidance for their decision-making.

The focus on internal teaching activities may be explained in part by a perception of teaching as an institutionally and locally embedded practice. In particular, foreign reviewers convey such a perception through the withdrawing strategy, which implies that educational proficiency cannot be (solely) assessed by means of a teaching portfolio (cf. Berk, 2005; Weiser, 2012). This position does not align with the work of scholars who advocate the teaching portfolio as a viable method for assessing educational proficiency (see e.g. Magin, 1998; Seldin et al., 2010). It also indicates academic cultural differences across nations. This in turn raises questions about peer review in general and, more specifically, about how educational proficiency and teaching could or should be assessed, and by whom (see e.g. Weiser, 2012). The majority of the special reviewers in this study are affiliated with Swedish HEIs, and they seem to be more familiar with the national context and discourse on educational proficiency, and know what is expected of them. By extension, this points to the importance and influence of reviewers’ perception of and competence regarding the object of assessment – as does the observation that reviewers tend to stress different aspects of the research-teaching nexus depending on their own subject knowledge. All in all, these findings connect and give renewed fuel to the ever-present, ongoing discussion about what qualifies a “peer” (see e.g. Askling, 2007; Bolander Laksov, 2018; Weiser, 2012).

Furthermore, via the articulations in the reviewers evaluation reports, educational proficiency comes across as a set of generic skills detached from both discipline and educational philosophical issues, and teaching emerges as a technical endeavour rather than a professional activity permeated with professional values, beliefs, and judgements. This transpires not least through an absence of articulations about the research-teaching nexus. It is indeed an interesting finding, as well as a somewhat troublesome one, since the research-teaching nexus is the basis of all higher education. From a national perspective, it is particularly interesting considering that, by the time of the establishment of the senior lectureship position in Swedish universities, the PhD degree was set as a qualification criteria as a means of assuring that all teaching was carried out by highly qualified scholars who would uphold the link between teaching and research. However, as a fundamental aspect of academic work, the research-teaching nexus does not seem to have manifested itself in these high-stake processes in the recruitment of academic teachers. Here, it is worth noting that – despite noble intentions of expressing the espousal of educational proficiency in policy – having two distinct criteria of eligibility may still lead to a separated rather than integrated view of academic competence, at the expense of the research-teaching nexus. If so, this finding calls for an overhaul of the criteria of eligibility and assessment.

Similar tendencies can be observed when special reviewers are appointed to assess educational proficiency more thoroughly than other reviewers. Part of the reason to appoint special reviewers to assess educational proficiency more carefully is to increase the value of teaching in recruitment and promotion. However, this procedure seems to lead to an unexpected division of labour among reviewers, since the “ordinary” reviewers (who are expected to assess both research and educational proficiency) withdraw from assessing educational proficiency. Thus, if the reason for choosing two distinct reviewers is to receive two separate assessments of both criteria, it might be worth looking into this finding further.

It is important to note at this point that the analysis does not show anything about the reviewers’ personal perceptions or views on educational proficiency, or about the candidates’ “actual” level of proficiency. Nonetheless, by means of their evaluations, the reviewers reproduce a scientific discourse in which educational proficiency is being viewed as equal to experience, and in which some aspects emerge as more important than others. Ultimately, the articulatory practice studied here supports the idea that the best way for an academic to spend her or his time (O’Meara, 2002; Thompson & Hunston, 2000) in terms of teaching is basically not to educate themselves in educational philosophy or develop as teachers, but rather to invest their time in amassing as much teaching experience as possible.

(In)variability across faculty positions

In an exploration of a position that was originally established to teach, and thus commonly entails a higher teaching load than a professorship, it was plausible to expect that the specific evaluation practice studied here would render a different construction of educational proficiency than that constructed in the recruitment of full professors (see Levander et al., 2019). At first sight, the value and evidence of educational proficiency appear to be the same across faculty positions. However, smaller variations emerge upon closer examination, with different themes being given a distinct emphasis depending on the position. The emphasis for both lectureships and professorships is on activities closely connected to the act of teaching rather than interaction with the academic community. However, for professorships, comparatively more emphasis is put on engagement in activities more peripheral to this act – such as service to society or scholarly interaction. This may be regarded as a rather uncontroversial outcome. If the general perception of senior lecturers is that they are primarily expected to conduct classroom teaching, it seems plausible for generic competencies and potential for teaching excellence to be put forward for such positions. At many faculties, a senior lecturer is expected to teach more than a professor; therefore, it seems relevant to emphasise having various teaching experiences. However, these findings may also indicate that, for a professorship, mastery of, for example, various instructional forms is taken for granted based on the assumption that a candidate for a professorship has by definition amassed more extensive teaching experience; thus, it may be regarded as superfluous to explicitly articulate such forms.

Taken together, the articulatory practice studied here in relation to the recruitment of senior lecturers produces a somewhat more restricted notion in comparison with the recruitment of full professors, in that it focuses even more on activities closely related to the act of teaching. However, apart from the heavy emphasis on PhD supervision for a professorship position, educational proficiency in terms of content is still articulated similarly regardless of position. This is reinforced by the fact that no additional themes were prompted by the empirical data studied here.

Future research

Recruitment of academics is part of the formation of academia and the life of professionals; hence, the topic addressed in this article is of common academic interest and of high relevance for the scientific community. The analysed data provided an opportunity to not only explore appointment processes in general, but also scrutinise what is articulated as educational proficiency in peer review for a junior academic position.

Peer review in academic recruitment and promotion is a highly complex phenomenon. These findings suggest that there are cultural and national differences in the perception of educational proficiency, prompting further research on how educational proficiency is conceptualised in different HEIs, nations, academic cultures, and by distinct reviewers. The research design employed here facilitates further comparative studies. Also, since the findings imply that education and research are perceived and assessed as separate rather than integrated aspects of academic scholarship, further research on the conceptualisation of academic scholarship would preferably include analyses of both educational and research proficiency. This is crucial, not least in view of the emergence of parallel reward systems in academia to appoint so-called distinguished university teachers.

Moreover, to more fully comprehend the value of educational proficiency in the context of academic scholarship, complete cases would be a promising avenue to further explore the manifestation of educational proficiency and its value in academic recruitment and promotion.

Disclosure statement

No potential conflict of interest was reported by the author.

Additional information

Notes on contributors

Sara Levander

Sara Levander is a senior lecturer and researcher in Education. Her research interests are higher education, academic work, and faculty evaluation in academic recruitment and promotion. A focal point for her research is the assessment of educational proficiency in academia.

Notes

1. Similar and related terms in the international literature are “teaching skills”, “teaching expertise”, and “Scholarship of Teaching and Learning”. The term “educational proficiency” is used herein to point to a potential expansion of these terms, indicating the possibility of a competence that includes knowledge, know-how, and reflection.

2. Equivalent to assistant professors in the United States.