Accounting for economies of scope in performance evaluations of university professors

Abstract

Teaching and research are widely regarded as the two key activities of academics. We propose a tailored version of the popular Data Envelopment Analysis methodology to evaluate the overall performance of university faculty. The methodology enables accounting for the potential presence of economies of scope between the teaching and research activities. It is illustrated with a dataset of professors working at a Business and Administration department of a university college. The estimation results reveal that overall the performance scores of faculty decrease if we allow for spillovers from research to teaching and vice-versa.

Keywords:

• teaching-research nexus
• Data Envelopment Analysis
• conditional efficiency
• economies of scope
• higher education

Notes

1 Some of the sceptics even go further claiming that teaching and research are not complementing but rather substituting activities, with the professor being forced to choose one activity over the other (eg, Barnett, 1992; Massy and Zemsky, 1994 etc).

2 Typically, polytechnic universities in the UK and colleges in the US are comprehensive higher education institutions offering a broad range of (both professional and academic) education programmes where courses also contain a research component.

3 We believe that a 3-year period is sufficiently broad to (1) collect student questionnaire data on multiple courses taught by the evaluated professors in order to obtain an accurate SET-score and (2) account for a possible lag between the research effort (eg, the writing of a paper) and the impact of this effort on the RES-score (eg, the paper being published in an international journal).

4 For all academics, official teaching and research time are measured as percentages of a common base (ie, 358 official working hours per week or, equivalently, 1600 official working hours per year). Therefore, for example, an academic that is fully contracted at the university college with 60% teaching task and 40% research task is expected to spend approximately 23h on the teaching duties and 15h on research-related activities.

5 Note that the percentages can be adjusted according to the total contract time. As an example, for a junior professor who is only working half-time at the department, instead of having research and teaching percentages of 60% and 40%, percentages will be set equal to 30% and 20%.

6 It is important to note that this set of background variables is not exhaustive. The literature indicated also other background variables (ie research collaboration (eg, Maske et al, 2003; and Ramos et al, 2007), family situation (eg, children versus no children; see, for instance, Bellas and Toutkoushian, 1999, and Hunter and Leahey, 2010), etc) that have the potential to influence the teaching and research performances of academics.

7 This finding of heavily and negatively skewed distributions of research output among the professors should not be a surprise. Numerous studies reported similar findings at other institutions or in other fields of academic research (Daniel and Fisch, 1990; Ramsden, 1994). In the literature, one also refers to this finding as Lotka's Law (after Lotka, 1926).

8 In fact, Cherchye et al (2008) originally considered a general setting with multiple inputs and multiple outputs. It can be verified that their general method boils down to the method described below for our specific setting.

9 Cherchye et al (2008) only prove necessity of the condition. Cherchye et al (2010) prove sufficiency for a formally similar model that applies to efficient consumer behaviour. This sufficiency argument can be adapted to our production setting.

10 While some may argue that outlying observations are highly interesting, and require additional research and inspection, we suggest mitigating their impact in performance evaluations. The reason is that the use of outlying performances as the sole benchmarks in the performance evaluations might be perceived by the professors as too ambitious, and that this perception might have an adverse effect on professor morale (demotivating instead of motivating professors to improve their performances).

11 For instance, there are some studies (eg, Basow, 2000; Centra and Gaubatz, 2000; Wolfer and Johnson, 2003) that found a student gender—teacher gender interaction in the evaluation of teaching effectiveness, with students rating professors from the same gender slightly more favourably than professors from the opposite gender.

12 In what follows, we restrict ourselves to briefly sketching the basic idea of the robust and conditional measurement procedure. We refer to Cazals et al (2002) and Daraio and Simar (2005, 2007a, 2007b), and De Witte and Kortelainen (2009) for a more detailed explanation of the method; this also includes a discussion of attractive statistical properties (which carry over to our setting).

13 See Cherchye et al (2008) for a detailed discussion. For ease of exposition we do not repeat the technical argument here.

14 Measurement errors might be present in the data as there are examples of professors whose registered teaching/research performances are probably not in correspondence with the actual teaching/research performance.