Latent Variable Analysis: A New Approach to University Ranking

Abstract

This study applies a Bayesian latent variable analysis to the task of determining rankings of universities in the UK and US, on the basis of a set of quality‐related measures. It estimates the degree of uncertainty in the rankings and permits the assessment of statistically significant differences across universities. It also provides a methodology for determining the weighting of various measures that is based on the patterns embedded in the data and compares the latent variable rankings with traditional weight‐and‐sum rankings. Overall, the methodology contributes to a better understanding of ranking efforts and illustrates the need for caution in interpreting distinctions published in traditional ranking systems.

1. The authors performed this work in RAND Education, a unit of RAND Corporation. The research activity was funded from a contract from the State of Qatar Supreme Education Council. The authors wish to express their appreciation for this support. The present article derives from a research project aimed at identifying high‐quality institutions of higher education, for the attention of Qatari students with government‐sponsored scholarships. This paper does not represent the views of the State of Qatar or its Supreme Education Council.

Notes

1. The authors performed this work in RAND Education, a unit of RAND Corporation. The research activity was funded from a contract from the State of Qatar Supreme Education Council. The authors wish to express their appreciation for this support. The present article derives from a research project aimed at identifying high‐quality institutions of higher education, for the attention of Qatari students with government‐sponsored scholarships. This paper does not represent the views of the State of Qatar or its Supreme Education Council.

2. See U.S. News & World Report's America's Best Colleges at ⟨http://www.usnews.com/usnews/home.htm⟩; Barron's Profiles of American Colleges at ⟨http://barronseduc.com/⟩; The Top American Research Universities, published by TheCenter at the University of Florida at ⟨http://thecenter.ufl.edu/research2002.html⟩, and Avery et al. (2004).

3. See The Times Good University Guide at ⟨http://www.timesonline.co.uk/section/0,,716,00.html⟩; The Guardian University Guide at ⟨http://education.guardian.co.uk/universityguide/0,10085,488282,00.html⟩, and the Sunday Times University Guide at ⟨http://www.timesonline.co.uk/section/0,,8403,00.html⟩.

4. See, for example, Maclean's Guide to Canadian Universities at ⟨http://www.macleans.ca/universities/index.jsp⟩; Hobsons Good Universities Guide at ⟨http://www.thegoodguides.com.au/⟩, and the Melbourne Institute Index of the International Standing of Australian Universities at ⟨www.melbourneinstitute.com/ publications/reports/aus_uni/mainpaper.pdf, 22 May 2005⟩, for Australia.

5. Available at ⟨http://ed.sjtu.edu.cn/ranking.htm⟩.

6. Available at ⟨http://www.thes.co.uk/worldrankings/⟩.

7. For example, President Gerhard Casper of Stanford University. Details at ⟨http://news‐service.stanford.edu/news/1997/april23/usnstatement.html⟩.

8. Data used with permission from The Times.

9. In particular, see pp. 13–18.

10. This can be seen from both the range and the coefficients of variation, i.e., the standard deviation divided by the mean. For the facilities, library, and research variables, the coefficients of variation are equal to 0.41, 0.38, and 0.38, respectively.

11. Data retrieved November 30 2004, from ⟨http://www.usnews.com/usnews/home.htm⟩. Data used with permission of the U.S. News & World Report .

12. Details at ⟨http://www.usnews.com/usnews/edu/college/rankings/about/weight_brief. php#giving⟩.

13. The coefficients of variation for the percentage of students in the top 10 percent of their class and for the alumni giving rate are higher than 0.60.

14. See ⟨www.mathstat.Helsinki.fi/openbugs/⟩.

15. Recall that our analysis used non‐informative priors.