Determinants of international university rankings scores

Abstract

This article analyses the relationship between a university’s score in international university rankings, its expenditure per student and other factors such as university mission, size and productive inefficiency. We develop an economic model of rankings and universities and estimate this model with data on universities classified in the top 200 by the Times Higher Education Supplement ranking of 2007. We find that the elasticity of a university’s ranking score for the expenditure per student is between 4% and 9%, and that there are no clear signs of inefficiency in production among the top 200 universities. University mission and size are also significant predictors of ranking score. These results are especially interesting given the relevance attributed to rankings by government officials, university directors and students.

Keywords:

• international university rankings
• higher education institutions
• times higher education supplement
• education expenditures
• production functions

JEL Classification:

• C51
• D20
• F23
• I21

Notes

¹ We include size in the ranker’s score function, but not in the university utility function. In reality, the university may care about relative measures of output as well. If the university size is exogenous, this is not a problem for our model, because size is not relevant in the university maximization process.

² The aggregation function in Equation 3 is similar to the one suggested by Fernández et al. (2000, 2002), although we choose a Cobb–Douglas instead of a Constant Elasticity of Substitution aggregation function.

³ The value of θ_T would be equal to zero if the ranker does not correct output for the size of the institution, and it would be equal to the sum of the parameters ϕ₁,…,ϕ_n if the ranker is only interested in relative measures of output (i.e. output divided by the size of the university). In our baseline specification the estimate of θ_T is 0.015 (equal to the difference between β₁ and β₂). However, we do not know the parameters ϕ₁,…,ϕ_n, so that it is not possible to give a useful interpretation for the size of θ_T.

⁴ In fact, our upper bound is not necessarily higher than the ‘true’ coefficient, because the noise in the measurement of expenditures is likely to produce a downward bias in the coefficient estimate. If this bias is large, it could more than compensate the upward bias induced by the omission of the fixed-effect.

⁵ The methodology of the THES-QS ranking has been subject to slight modifications between 2005 and 2007. For example, the way in which the opinions of the surveyed academics are aggregated into the peer review score has changed. However, we believe that the THES-QS score of 2005 still adequately captures those unobserved characteristics of a university that we wish to control for.

⁶ It would be very interesting to make a distinction by category of expenditure, and examine whether expenditure on different items or accounts have a differential impact on the ranking score. However, this is made extremely difficult by the different languages and accounting traditions on which the financial statements that we consulted are based (see Bonaccorsi et al., 2007, for a discussion of problems related with collecting measures of expenditures for an international sample of universities). Furthermore, as the ranking methodology remained approximately the same until 2009 and ranking scores are available, it would add value to collect expenditure data for different years. However, we rely on data collected several years ago in the context of an earlier research project (Ritzen, 2010). It is currently very difficult to find expenditure data older than 2010 for a large sample of universities.

⁷ The use of proxies for university mission differs from the approach followed by studies estimating cost functions of higher education institutions. This literature has dealt with mission diversity through parameter heterogeneity, either by estimating cost functions for different types of institutions separately (Johnes et al., 2004), or by allowing for different parameters for each institution (Johnes and Johnes, 2009).

⁸ O’Leary et al. (2008) rank universities in five fields of study, based on the peer review sub-score. These fields are technology, natural sciences, bio-medicine and life sciences, social sciences and arts and humanities. We classify the first three of them as ‘hard fields’. The index of specialization is generated by assigning one point for each possible couple comprising one ‘hard field’ and one ‘soft field’ where the hard field ranks better. Thus, the index reaches its maximum score (six) for a university ranking better in all three ‘hard fields’ than in any of the ‘soft’ ones. Conversely, a university ranked higher in social sciences and arts and humanities than in all the other three fields receives zero points.

⁹ Caltech, which researches for the NASA, includes in its accounts the expenditures for the Jet Propulsion Laboratory (which, in turn, represents more than 75% of the total expenses reported), and has also by far the lowest number of students and the highest level of relphd. However, after the logarithmic transformation the observation does not appear as an outlier according to our diagnostic measure (Wilks statistic). Our results are not substantially affected by excluding this institution from the analysis.

¹⁰ The number of observations is only 127 in this regression, because we observe only institutions for which we collected data in 2009 and which subsequently appeared in the 2012/2013 ranking. Notice that the 2012/2013 score is computed according to a very different methodology than the 2007 score. As a result, the coefficients from model 11 are not entirely comparable to the coefficients from other models.