The performance based research fund in New Zealand: taking stock and looking forward

References

• Abramo, G., D’Angelo, C.A., & Di Costa, F. (2011). National research assessment exercises: a comparison of peer review and bibliometric rankings. Scientometrics, 89, 929–941. doi:10.1007/s11192-011-0459-x
   Web of Science ®Google Scholar
• Abramo, G., D’Angelo, C.A., & Di Costa, F. (2019). When research assessment exercises leave room for opportunistic behavior by the subjects under evaluation. Journal of Informetrics, 13, 830–840. doi:10.1016/j.joi.2019.07.006
   Web of Science ®Google Scholar
• Adams, J. (2008). Strategic review of the performance-based research fund: The assessment process. Report Prepared for the Tertiary Education Commission, Wellington.
   Google Scholar
• Aksnes, D.W., Langfedlt, L., & Wouters, P. (2019). Citations, citation indicators, and research quality: An overview of basic concepts and theories. Sage Open, 9. doi:10.1177/2158244019829575
   Web of Science ®Google Scholar
• Aksnes, D.W., & Taxt, R.E. (2004). Peer reviews and bibliometric indicators: a comparative study at a Norwegian university. Research Evaluation, 13, 33–41. doi:10.3152/147154404781776563
   Web of Science ®Google Scholar
• Anderson, D.L., Smart, W., & Tressler, J. (2013). Evaluating research – Peer review team assessment and journal based bibliographic measures: New Zealand PBRF research output scores in 2006. New Zealand Economic Papers, 47, 140–157. doi:10.1080/00779954.2013.772879
   Google Scholar
• Anderson, D.L., & Tressler, J. (2009). The ‘Excellence in Research for Australia’ scheme: a test drive of draft journal weights with New Zealand data. Agenda: A Journal of Policy Analysis and Reform, 16, 7–24. doi:10.22459/AG.16.04.2009.01
   Google Scholar
• Anderson, D.L., & Tressler, J. (2012). The impact of journal weighting scheme characteristics on research output measurement in economics: The New Zealand case. Review of Economics and Institutions, 3, 1–25. doi:10.5202/rei.v3i3.95
   Google Scholar
• Anderson, D.L., & Tressler, J. (2014). The New Zealand performance-based research fund and its impact on publication activity in economics. Research Evaluation, 23, 1–11. doi:10.1093/reseval/rvt017
   Web of Science ®Google Scholar
• Baccini, A., & De Nicolao, G. (2016). Do they agree? Bibliometric evaluation versus informed peer review in Italian research assessment exercise. Scientometrics, 108, 1651–1671. doi:10.1007/s11192-016-1929-y
   Web of Science ®Google Scholar
• Boston, J. (2006). Rationale for the performance-based research fund and its evaluation. In L. Bakker, J. Boston, L. Campbell, & R. Smyth (Eds.), Evaluating the performance-based research fund: Framing the debate. Wellington: Institute of Policy Studies.
   Google Scholar
• Buckle, R.A., & Creedy, J. (2018). The impact on research quality of performance-based funding: the case of New Zealand’s PBRF scheme. A Journal of Policy Analysis and Reform, 25, 25–48. doi:10.22459/AG.25.01.2018.02
   Google Scholar
• Buckle, R.A., & Creedy, J. (2019a). The evolution of research quality in New Zealand universities as measured by the performance-based research fund process. New Zealand Economic Papers, 53, 144–165. doi:10.1080/00779954.2018.1429486
   Google Scholar
• Buckle, R.A., & Creedy, J. (2019b). An evaluation of metrics used by the performance-based research fund process in New Zealand. New Zealand Economic Papers, 53, 270–287. doi:10.1080/00779954.2018.1480054
   Google Scholar
• Buckle, R.A., & Creedy, J. (2020). The ‘disciplinary effect’ of the performance-based research fund process in New Zealand. New Zealand Economic Papers, 54, 107–126. doi:10.1080/00779954.2019.1636122
   Google Scholar
• Buckle, R.A., & Creedy, J. (2022). The performance-based research fund in NZ: Taking stock and looking forward. Working Papers in Public Finance 02/2022. Wellington: Victoria University of Wellington.
   Google Scholar
• Buckle, R.A., Creedy, J., & Ball, A. (2021). Fifteen years of a PBRF in New Zealand: Incentives and outcomes. Australian Economic Review, 54, 208–230. doi:10.1111/1467-8462.12415
   Web of Science ®Google Scholar
• Buckle, R.A., Creedy, J., & Gemmell, N. (2020). Is external research income associated with convergence or divergence of research quality across universities and disciplines? Evidence from the PBRF process in New Zealand. Applied Economics, 52, 3919–3932. doi:10.1080/00036846.2020.1725235
   Web of Science ®Google Scholar
• Buckle, R.A., Creedy, J., & Gemmell, N. (2022). Sources of convergence and divergence in university research quality: Evidence from the New Zealand PBRFS. Scientometrics, 127, 3021–3047. doi:10.1007/s11192-022-04376-9
   Web of Science ®Google Scholar
• Butler, P., & Mulgan, R. (2013). Can academic freedom survive performance-based research funding? Victoria University of Wellington Law Review, 44, 487–520. doi:10.26686/vuwlr.v44i3/4.4989
   Google Scholar
• Chatterjee, B., Cordery, C.J., Loo, I.D., & Letiche, H. (2020). The spectacle of research assessment systems: Insights from New Zealand and the United Kingdom. Accounting, Auditing and Accountability Journal, 33, 1219–1246. doi:10.1108/AAAJ-01-2019-3865
   Web of Science ®Google Scholar
• Chavarro, D., Tang, P., & Rafols, I. (2016). Why researchers publish in non-mainstream journals: Training, knowledge bridging and gap fillings. In University of Sussex Social Policy Research Unit Working Paper Series, 2016–2022.
   Google Scholar
• Clear, T., & Clear, A. (2012). PBRF 2012 and the nature of degree teaching. Paper presented at the Third Annual Conference of Computing and Information Technology Research and Education New Zealand (CITRENZ2012). Christchurch, New Zealand.
   Google Scholar
• Curtis, B. (2004). The PBRF and its implications for academics (in the humanities and social sciences). Labour, Employment and Work in New Zealand. Wellington.
   Google Scholar
• Curtis, B., & Matthewman, S. (2005). The managed university: The PBRF, its impacts and staff attitudes. New Zealand Journal of Employment Relations, 30, 1–17.
   Google Scholar
• de Boer, H., Jongbloed, B., Benneworth, P., Cremonini, L., Kolster, R., Kottmann, A., … Vossenstey, H. (2015). Performance-based funding and performance agreements in fourteen higher education systems. Centre for Higher Education Policy Studies, C15HdB014. Enschede: Universiteit Twente.
   Google Scholar
• Gemmell, N., Nolan, P., & Scobie, G. (2017). Estimating quality-adjusted productivity in tertiary education: methods and evidence for New Zealand. Victoria University of Wellington Working Papers in Public Finance, No. WP17/2017.
   Google Scholar
• Geuna, A., & Piolatto, M. (2016). Research assessment in the UK and Italy: Costly and difficult, but probably worth it (at least for a while). Research Policy, 45, 260–271. doi:10.1016/j.respol.2015.09.004
   Web of Science ®Google Scholar
• Gibson, J., Anderson, D.L., & Tressler, J. (2014). Which journal rankings best explain academic salaries: Evidence from the University of California. Economic Inquiry, 52, 1322–1340. doi:10.1111/ecin.12107
   Web of Science ®Google Scholar
• Gibson, J., Anderson, D.L., & Tressler, J. (2017). Citations or journal quality: which is rewarded more in the academic labor market? Economic Inquiry, 55, 1945–1965. doi:10.1111/ecin.12455
   Web of Science ®Google Scholar
• Hazledine, T., & Kurniawan, C. (2005). Efficiency of New Zealand universities and the impact of the performance based research fund. New Zealand Economic Papers, 39, 153–179. doi:10.1080/00779950509558491
   Google Scholar
• Hicks, D. (2012). Performance-based university research funding schemes. Research Policy, 41, 251–261. doi:10.1016/j.respol.2011.09.007
   Web of Science ®Google Scholar
• Hirschberg, J.G., & Lye, J.N. (2020). Grading journals in economics: the ABCS of the ABDC. Journal of Economic Surveys, 34, 876–921. doi:10.1111/joes.12369
   Web of Science ®Google Scholar
• Khurana, S., Gunby, P., & Menclova, A.K. (2021). The relationship between research achievement of academics and student labour market outcomes. Paper presented to the NZ Association of Economists’ Annual Conference, Wellington. Retrieved from: 623971_8d8db8380044440e9cd6c73228c8667d.pdf (filesusr.com).
   Google Scholar
• Kolarz, P., Arnold, E., Dijkstal, F., Nielsen, K., & Farla, K. (2019). International landscape study of research and innovation systems. Brighton: Technopolis.
   Google Scholar
• Martin, B.R. (2011). The Research Excellence Framework and the impact agenda: Are we creating a Frankenstein monster? Research Evaluation, 20, 247–254. doi:10.3152/095820211X13118583635693
   Web of Science ®Google Scholar
• New Zealand Ministry of Education. (2018a). A history and overview of the PBRF. Wellington: Ministry of Education. Retrieved from: https://www.education.govt.nz/assets/Documents/Further-education/Policies-and-strategies/Performance-based-research-fund/PBRFHistoryAndOverview.pdf
   Google Scholar
• New Zealand Ministry of Education. (2018b). Terms of reference issue paper 1: Revisiting the objectives of the PBRF. Retrieved from: Terms-of-Reference-Issue-Paper-1-Revisiting-PBRF-Objectives.pdf (education.govt.nz).
   Google Scholar
• OECD. (2010). Performance-based funding for public research in tertiary education institutions: Workshop proceedings. Paris: OECD.
   Google Scholar
• PBRF Review Panel. (2020). Towards the tertiary research excellence evaluation (TREE): The report of the PBRF review panel. Wellington: Ministry of Education.
   Google Scholar
• Quacquarelli Symonds. (2021). QS World University Rankings. Retrieved from: QS World University Rankings 2022: Top Global Universities | Top Universities.
   Google Scholar
• Shockley, W. (1957). On the statistics of individual variations of productivity in research laboratories. Proceedings of the IRE, 45, 279–290. doi:10.1109/JRPROC.1957.278364
   Google Scholar
• Smart, W. (2009). The impact of the performance-based research fund on the research productivity of New Zealand universities. Social Policy Journal of New Zealand, 34, 136–151.
   Google Scholar
• Stern, N. (2016). Building on success and learning from experience: An independent review of the research excellence framework. London: Department for Business, Energy and Industrial Strategy.
   Google Scholar
• Tertiary Education Commission. (2019). Improving research quality: The results of the PBRF 2018 quality evaluation. Wellington: Tertiary Education Commission.
   Google Scholar
• Times Higher Education. (2022). World university rankings. Retrieved from: https://www.timeshighereducation.com/w%C3%B6rld-univ%C3%A9rsity-r%C3%A0nkings
   Google Scholar
• Tressler, J., & Anderson, D.L. (2012). Citations as a measure of the research outputs of New Zealand’s Economics Departments: The problem of ‘long and variable lags’. Agenda: A Journal of Policy Analysis and Reform, 19, 17–37. doi:10.22459/AG.19.01.2012.02
   Google Scholar
• Walsh, P. (2004). Transcript of plenary sessions, PBRF forum: Evaluation of the assessment framework. The Royal Society of New Zealand Social Sciences Committee. Retrieved from: http://www.rsnz.govt.nz/advisory/social_science/media/pbrf2004-plenary.php
   Google Scholar
• WEB Research. (2004). PBRF: Phase 1 evaluation – July 2004: Phase 1 evaluation of the implementation of the PBRF and the conduct of the 2003 quality evaluation. Wellington: Centre for Research on Work, Education, and Business.
   Google Scholar
• Whitley, R. (2008). Changing governance of the public sciences. In R. Whitley & J. Glaser (Eds.), The changing governance of the sciences: The advent of research evaluation systems (pp. 14–16). New York, NY: Springer Science and Business Media.
   Google Scholar
• Willmott, H. (2011). Journal list fetishism and the perversion of scholarship: Reactivity and the ABS list. Organisation, 18, 429–442.
   Web of Science ®Google Scholar
• Wilsdon, J. (2015). The metric tide: Independent review of the role of metrics in research assessment and management. New York: SAGE Publications.
   Google Scholar
• Woelert, P. (2015). The ‘logic of escalation’ in performance measurement: An analysis of the dynamics of a research evaluation system. Policy and Society, 34, 75–85. doi:10.1016/j.polsoc.2015.03.004
   Web of Science ®Google Scholar
• Woelert, P. (2021). Reactivity and the dialectics of performance measurement: Micropolitics between agency and compliance. Administration and Society, 53, 963–983. doi:10.1177/00953997211003841
   Web of Science ®Google Scholar
• Woelert, P., & McKenzie, L. (2018). Follow the money? How Australian universities replicate national performance-based funding mechanisms. Research Evaluation, 27, 184–195. doi:10.1093/reseval/rvy018
   Web of Science ®Google Scholar