How to measure research efficiency in higher education? Research grants vs. publication output

References

• Abbott, M., & Doucouliagos, C. (2003). The Efficiency of Australian Universities: A data envelopment analysis. Economics of Education Review, 22(1), 89–97. doi:10.1016/S0272-7757(01)00068-1
   Web of Science ®Google Scholar
• Abramo, G., & D’Angelo, C.A. (2016). A farewell to the MNCS and like size-independent indicators. Journal of Informetrics, 10(2), 646–651. doi:10.1016/j.joi.2016.04.006
   Web of Science ®Google Scholar
• Agasisti, T., Catalano, G., Landoni, P., & Verganti, R. (2012). Evaluating the performance of academic departments: An analysis of research-related output efficiency. Research Evaluation, 21(1), 2–14. doi:10.1093/reseval/rvr001
   Web of Science ®Google Scholar
• Agasisti, T., & Gralka, S. 2017.  The transient and persistent efficiency of Italian and German universities: A stochastic frontier analysis. CEPIE Working Paper, 14/17, 1–32.
   Google Scholar
• Agasisti, T., & Haelermans, C. (2016). Comparing efficiency of public universities among European countries: Different incentives lead to different performances. Higher Education Quarterly, 70(1), 81–104. doi:10.1111/hequ.2016.70.issue-1
   Web of Science ®Google Scholar
• Agasisti, T., & Johnes, G. (2009). Beyond frontiers: Comparing the efficiency of higher education decision-making units across more than one country. Education Economics, 17(1), 59–79. doi:10.1080/09645290701523291
   Google Scholar
• Agasisti, T., & Johnes, G. (2010). Heterogeneity and the evaluation of efficiency: The case of Italian Universities. Applied Economics, 42(11), 1365–1375. doi:10.1080/00036840701721463
   Web of Science ®Google Scholar
• Aigner, D.C., Lovell, K., & Schmidt, P. (1977). Formulation and estimation of stochastic frontier production function models. Journal of Econometrics, 6(1), 21–37. doi:10.1016/0304-4076(77)90052-5
   Google Scholar
• Aksnes, D.W. (2003). A macro study of self-citation. Scientometrics, 56(2), 235–246. doi:10.1023/A:1021919228368
   Web of Science ®Google Scholar
• Aksnes, D.W., & Rip, A. (2009). Researchers’ perceptions of citations. Research Policy, 38(6), 895–905. doi:10.1016/j.respol.2009.02.001
   Web of Science ®Google Scholar
• Auranen, O., & Nieminen, M. (2010). University research funding and publication performance - an international comparison. Research Policy, 39(6), 822–834. doi:10.1016/j.respol.2010.03.003
   Web of Science ®Google Scholar
• Banker, R.D., Charnes, A., & Cooper, W.W. (1984). Some models for estimating technical and scale inefficiencies in data envelopment analysis. Management Science, 30(9), 1078–1092. doi:10.1287/mnsc.30.9.1078
   Web of Science ®Google Scholar
• Başkaya, S., & Klumpp, M. (2014). International data envelopment analysis in higher education: How do institutional factors influence university efficiency?. Journal of Business and Economics, 5(11), 2085–2090. doi:10.15341/jbe(2155-7950)
   Google Scholar
• Baumol, W.J., Panzar, J.C., Willig, R.D., Bailey, E.E., & Fischer, D. (1982). Contestable markets and the theory of industry structure. New York, NY: Harcourt Brace Jovanovich, Inc.
   Google Scholar
• Bayraktar, E., Tatoglu, E., & Zaim, S. (2013). Measuring the relative efficiency of quality management practices in Turkish public and private universities. Journal of the Operational Research Society, 64(12), 1810–1830. doi:10.1057/jors.2013.2
   Web of Science ®Google Scholar
• Bolli, T., Olivares, M., Bonaccorsi, A., Daraio, C., Aracil, A.G., & Lepori, B. (2016). The differential effects of competitive funding on the production frontier and the efficiency of universities. Economics Of Education Review, 52, 91–104. doi:10.1016/j.econedurev.2016.01.007
   Web of Science ®Google Scholar
• Bornmann, L. (2012). Measuring the societal impact of research: Research is less and less assessed on scientific impact alone—we should aim to quantify the increasingly important contributions of science to society. EMBO Reports, 13(8), 673–676. doi:10.1038/embor.2012.99
   PubMed Web of Science ®Google Scholar
• Bornmann, L. (2013). what is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society for Information Science and Technology, 64(2), 217–233. doi:10.1002/asi.22803
   Web of Science ®Google Scholar
• Bornmann, L., & Haunschild, R. (2016). Efficiency of research performance and the glass researcher. ArXiv Preprint, arXiv: 1605.09515, 1-8. doi: 10.1016/j.joi.2015.11.009
   Google Scholar
• Bornmann, L., Leydesdorff, L., & Mutz, R. (2013). The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits. Journal of Informetrics, 7(1), 158–165. doi:10.1016/j.joi.2012.10.001
   Web of Science ®Google Scholar
• Bozeman, B. (2000). Technology transfer and public policy: A review of research and theory. Research Policy, 29(4–5), 627–655. doi:10.1016/S0048-7333(99)00093-1
   Web of Science ®Google Scholar
• Butler, L. (2003). Explaining Australia’s Increased Share of ISI Publications—The Effects of a Funding Formula Based on Publication Counts. Research Policy, 32(1), 143–155. doi:10.1016/S0048-7333(02)00007-0
   Web of Science ®Google Scholar
• Coelli, T.J., Prasada Rao, D., O’Donnell, C.J., & Battese, G.E. (2005). An introduction to efficiency and productivity analysis. Springer Science & Business Media.
   Google Scholar
• De Groot, H., McMahon, W.W., & Volkwein, J.F. (1991). The cost structure of American research universities. The Review of Economics and Statistics, 424–431. doi:10.2307/2109566
   Web of Science ®Google Scholar
• De Witte, K., & López-Torres, L. (2017). Efficiency in education: A review of literature and a way forward. Journal of the Operational Research Society, 68(4), 339–363. doi:10.1057/jors.2015.92
   Web of Science ®Google Scholar
• Eagan, K., & Titus, M.A. (2016). examining production efficiency in higher education: the utility of stochastic frontier analysis. In Michael B. Paulsen (Ed.), Higher Education: Handbook of Theory and Research (pp. 441–512). Cham: Springer.
   Google Scholar
• Eck, A., Gralka, S., & Heller, J. (2015). Zur Effizienz Der hochschulen in den ost-und Westdeutschen flächenländern. Ifo Dresden Berichtet, 22(2), 33–41.
   Google Scholar
• Fandel, G. (2007). On the performance of universities in north rhine-westphalia, Germany: Government’s redistribution of funds judged using DEA efficiency measures. European Journal of Operational Research, 176(1), 521–533. doi:10.1016/j.ejor.2005.06.043
   Web of Science ®Google Scholar
• Gawellek, B., & Sunder, M. (2016). The German excellence initiative and efficiency change among universities, 2001–2011. Working Paper, Universität Leipzig, 142(1), 1–29.
   Google Scholar
• German Federal Statistical Office. (2015). Monetäre Hochschulstatistische Kennzahlen, fachserie 11 reihe 4.3.2 (pp. 1–480). Wiesbaden: Statistisches Bundesamt (Destatis).
   Google Scholar
• German Research Foundation. (2015). Funding atlas 2015: Key indicators for publicly funded research in Germany. Bonn: Author.
   Google Scholar
• Good, B., Vermeulen, N., Tiefenthaler, B., & Arnold, E. (2015). Counting quality? The czech performance-based research funding system. Research Evaluation, 24(2), 91–105. doi:10.1093/reseval/rvu035
   Web of Science ®Google Scholar
• Gralka, S. (2018a). Stochastic frontier analysis in higher education: A systematic review. CEPIE Working Paper, 5/18, 1–58.
   Google Scholar
• Gralka, S. (2018b). Persistent inefficiency in the higher education sector: Evidence from Germany. Education Economics, 26(4), 373–392. doi:10.1080/09645292.2017.1420754
   Web of Science ®Google Scholar
• Heinrich, C.J. (2002). Outcomes–based performance management in the public sector: Implications for government accountability and effectiveness. Public Administration Review, 62(6), 712–725. doi:10.1111/1540-6210.00253
   Web of Science ®Google Scholar
• Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The leiden manifesto for research metrics. Nature, 520, 429–431. doi:10.1038/520429a
   PubMed Web of Science ®Google Scholar
• Hornbostel, S. (2001). third party funding of German universities. An indicator of research activity?. Scientometrics, 50(3), 523–537. doi:10.1023/A:1010566916697
   Web of Science ®Google Scholar
• Izadi, H., Johnes, G., Oskrochi, R., & Crouchley, R. (2002). stochastic frontier estimation of a CES cost function: The case of higher education in Britain. Economics of Education Review, 21(1), 63–71. doi:10.1016/S0272-7757(00)00044-3
   Web of Science ®Google Scholar
• Jansen, D., Wald, A., Franke, K., Schmoch, U., & Schubert, T. (2007). drittmittel als performanzindikator der wissenschaftlichen forschung. KZfSS Kölner Zeitschrift Für Soziologie Und Sozialpsychologie, 59(1), 125–149. doi:10.1007/s11577-007-0006-1
   Web of Science ®Google Scholar
• Johnes, G. (1997). Costs and industrial structure in contemporary british higher education. The Economic Journal, 107(442), 727–737. doi:10.1111/ecoj.1997.107.issue-442
   Web of Science ®Google Scholar
• Johnes, G., & Johnes, J. (1993). Measuring the research performance of uk economics departments: An application of data envelopment analysis (Oxford Economic Papers), 332–347. doi:10.1093/oxfordjournals.oep.a042095
   Google Scholar
• Johnes, G., Johnes, J., Thanassoulis, E., Lenton, P., & Emrouznejad, A. (2005). An exploratory analysis of the cost structure of higher education in England. Nottingham: UK Department for Education and Skills.
   Google Scholar
• Johnes, G., & Schwarzenberger, A. (2011). differences in cost structure and the evaluation of efficiency: The case of German universities. Education Economics, 19(5), 487–499. doi:10.1080/09645291003726442
   Google Scholar
• Johnes, G., & Tone, K. (2017). The efficiency of higher education institutions in england revisited: Comparing alternative measures. Tertiary Education and Management, 23(3), 191–205. doi:10.1080/13583883.2016.1203457
   Web of Science ®Google Scholar
• Johnes, J. (2006). Data envelopment analysis and its application to the measurement of efficiency in higher education. Economics of Education Review, 25(3), 273–288. doi:10.1016/j.econedurev.2005.02.005
   Web of Science ®Google Scholar
• Kempkes, G., & Pohl, C. (2008). Do institutions matter for university cost efficiency? Evidence from Germany. CESifo Economic Studies, 54(2), 177–203. doi:10.1093/cesifo/ifn009
   Web of Science ®Google Scholar
• Kempkes, G., & Pohl, C. (2010). The efficiency of German universities–some evidence from nonparametric and parametric methods. Applied Economics, 42(16), 2063–2079. doi:10.1080/00036840701765361
   Web of Science ®Google Scholar
• Koshal, R.K., & Koshal, M. (1999). Economies of scale and scope in higher education: A case of comprehensive universities. Economics of Education Review, 18(2), 269–277. doi:10.1016/S0272-7757(98)00035-1
   Web of Science ®Google Scholar
• Kumbhakar, S.C., Lien, G., & Brian Hardaker, J. (2014). Technical efficiency in competing panel data models: A study of norwegian grain farming. Journal of Productivity Analysis, 41(2), 321–337. doi:10.1007/s11123-012-0303-1
   Web of Science ®Google Scholar
• Kumbhakar, S.C., & Lovell, C.K. (2003). Stochastic frontier analysis. Cambridge: Cambridge University Press.
   Google Scholar
• Lehmann, E.E., Meoli, M., Paleari, S., & Stockinger, S. (2017). approaching effects of the economic crisis on university efficiency: A comparative study of Germany and Italy. Eurasian Business Review, 1–18. doi:10.1007/s40821-017-0091-7
   Web of Science ®Google Scholar
• Lin, L.I.-K. (2000). A note on the concordance correlation coefficient. Biometrics, 56, 324–325.
   Google Scholar
• Nazarko, J., & Šaparauskas, J. (2014). Application of DEA method in efficiency evaluation of public higher education institutions. Technological and Economic Development of Economy, 20(1), 25–44. doi:10.3846/20294913.2014.837116
   Web of Science ®Google Scholar
• OECD. (2017). Education at a Glance 2017 (Vol. 2017). Paris: Author. Education at a Glance https://read.oecd-ilibrary.org/education/education-at-a-glance-2017_eag-2017-en
   Google Scholar
• Olivares, M., & Wetzel, H. (2014). competing in the higher education market: Empirical evidence for economies of scale and scope in German higher education institutions. CESifo Economic Studies, 60(4), 653–680. doi:10.1093/cesifo/ifu001
   Web of Science ®Google Scholar
• O’Mahony, M., & Stevens, P. (2004). international comparisons of performance in the provision of public services: Outcome based measures for education. In Royal Economic Society Annual Conference. Swansea.
   Google Scholar
• Orr, D., Jaeger, M., & Schwarzenberger, A. (2007). performance-based funding as an instrument of competition in German higher education. Journal of Higher Education Policy and Management, 29(1), 3–23. doi:10.1080/13600800600980072
   Google Scholar
• Rhaiem, M. (2017). Measurement and determinants of academic research efficiency: A systematic review of the evidence. Scientometrics, 110(2), 581–615. doi:10.1007/s11192-016-2173-1
   Web of Science ®Google Scholar
• Schmoch, U., Schubert, T., Jansen, D., Heidler, R., & Regina, V.G. (2010). How to use indicators to measure scientific performance: A balanced approach. Research Evaluation, 19(1), 2–18. doi:10.3152/095820210X492477
   Web of Science ®Google Scholar
• Schubert, T. (2009). Empirical observations on new public management to increase efficiency in public research—boon or bane?. Research Policy, 38(8), 1225–1234. doi:10.1016/j.respol.2009.06.007
   Web of Science ®Google Scholar
• Stevens, P.A. (2005). A stochastic frontier analysis of english and Welsh universities. Education Economics, 13(4), 355–374. doi:10.1080/09645290500251581
   Google Scholar
• Thanassoulis, E., Kortelainen, M., Johnes, G., & Johnes, J. (2011). Costs and efficiency of higher education institutions in England: A DEA analysis. Journal of the Operational Research Society, 62(7), 1282–1297. doi:10.1057/jors.2010.68
   Web of Science ®Google Scholar
• Warning, S. (2007). The economic analysis of universities: Strategic groups and positioning. Cheltenham UK: Elgar.
   Google Scholar
• Wissenschaftsrat. (2018). Basisdaten zu hochschulen und forschungseinrichtungen in deutschland. Wissenschaftsrat, 1–4.
   Google Scholar
• Woelert, P., & McKenzie., L. (2018). Follow the money? How Australian universities replicate national performance-based funding mechanisms. Research Evaluation, 27(3), 184–195. doi:10.1093/reseval/rvy018
   Web of Science ®Google Scholar
• Wohlrabe, K., Bornmann, L., Gralka, S., & Anegon, F.D.M. (2019). Wie effizient forschen universitäten in deutschland, deren zukunftskonzepte im rahmen der exzellenzinitiative ausgezeichnet wurden?. Zeitschrift Für Evaluation 1/2019.
   Google Scholar
• Wolszczak-Derlacz, J., & Parteka, A. (2011). Efficiency of European public higher education institutions: A two-stage multicountry approach. Scientometrics, 89(3), 887. doi:10.1007/s11192-011-0484-9
   PubMed Web of Science ®Google Scholar
• Worthington, A.C. (2001). An empirical survey of frontier efficiency measurement techniques in education. Education Economics, 9(3), 245–268. doi:10.1080/09645290110086126
   Google Scholar
• Worthington, A.C., & Higgs, H. (2011). Economies of scale and scope in Australian higher education. Higher Education, 61(4), 387–414. doi:10.1007/s10734-010-9337-3
   Web of Science ®Google Scholar
• Worthington, A.C., & Lee, B.L. (2008). Efficiency, technology and productivity change in Australian universities, 1998–2003. Economics of Education Review, 27(3), 285–298. doi:10.1016/j.econedurev.2006.09.012
   Web of Science ®Google Scholar