Performance data governance and management of learning and teaching in higher education: the SQELT project

This special issue of Quality in Higher Education is dedicated to the theme of Performance Data Governance and Management (PDGM) of Learning and Teaching in Higher Education. The seven articles deal with the achievements, success and desiderata of PDGM at six European universities from Austria, Belgium, Italy, Poland, Portugal and the United Kingdom and beyond. The articles arose from work done in the Erasmus+ project SQELT (Sustainable Quality Enhancement in Higher Education Learning and Teaching) that gathered ten institutional partners from nine European countries in a Strategic Partnership (SQELT, 2020). More specifically, the authors of the special issue came from eight European universities, an evaluation and quality assurance agency and a policy research centre.

The SQELT project investigated PDGM in learning and teaching of higher education including its policy and core elements, for example a PDGM policy, an ethical code of practice for data analytics and a comprehensive performance indicator set. These and further relevant elements are not per se completely novel. However, at many higher education organisational places they are still often not fully known or only present in rudimentary ways related to pronounced island approaches in PDGM, which are not exactly a proof of systematic organisational effectivity and efficiency. Therefore, it was the intention of the SQELT project to produce, as far as possible with such limited resources, a systematic and integrated approach to PDGM matters that should support the universities of the partnership as well as any other interested university to improve their PDGM systems.

In the light of notorious success factors of quality management and evidence-informed organisational development in higher education institutions (Leiber, 2019b, 332ff.), benchlearning and strategic SWOT analyses related to the SQELT university project partners’ actual PDGM approaches exhibited the need of several initiatives of evidence-informed organisational development to further develop, improve and refine their PDGM models. Particularly, the sample PDGM models showed the following four main organisational transformation needs. First, procedures of data processing and communication, software platforms and responsible bodies for collecting and interpreting performance indicators must be further developed to improve quality as well as usability and accessibility of data and information. Particularly, there is often a need for better organising PDGM systems that avoid multiple island solutions and unnecessary resources’ consumption. Second, the empirical performance monitoring needs of higher education institutions must be balanced with various opposing policy demands originating from traditional disciplinary attitudes (for example, rejection on performance indicators, particularly quantitative performance indicators) as well as from ministerial education politics (for example, focus on quantitative and economic indicators and reduction to very small performance indicator sets that are easily manageable though less informative). Third, processes, organisational bodies and human resources for fostering participative responsibility for PDGM must be established including more efficient decision-making of collegial bodies. Fourth, educational strategies (mission, values, vision) must be established, including the prospects and ambiguities of PDGM and data analytics (for example, learning analytics).

Furthermore, based on the stocktaking and benchlearning insights of the SQELT project partners including stakeholder focus group surveys and discussions, the following critical success factors of PDGM could be identified that may be supportive guidance for other higher education institutions that engage in developing their PDGM (SQELT-GL, 2020).

• Provide justifiable belief in success promises of PDGM: surveyed stakeholders are often unsure about the possibility of fulfiling all promises of PDGM, particularly learning analytics.

• As always, leadership engagement is also a core driver of PDGM development and implementation: surveyed stakeholders sometimes diagnose insufficient engagement of leaders in PDGM.

• Reflected understanding and practice of PDGM based on adequate, sufficient and self-determined performance indicator sets is also of basic importance: surveyed stakeholders see various deficits in their institutions’ performance indicator sets.

• Reflected and applied PDGM ethics is indispensable: this is seen as an important issue by most surveyed stakeholders (while the willingness to practice this theoretical insight does not always seem to keep pace with the claimed importance).

• Finally, an adequate financial climate is necessary: underfinanced learning and teaching seems to be a widespread reality and is often experienced as one of the obstacles to implement appealing PDGM solutions.

Against this backdrop, this special issue serves five main goals. First, it embeds the themes of performance data management and performance indicators in learning and teaching into current debates and perspectives on university governance and management and higher education system policies. This includes consideration of the topics in a European and international perspective. Second, particularly, it gives a contextualised discussion of the complex and sought-after, though still contested (and often misunderstood) subject of performance indicators in learning and teaching, their nature, scope, number and applicability. Third, it further contextualises and justifies performance indicators of learning and teaching by investigating the role of contemporary theories of learning and concepts of academic teaching and their evaluation. Fourth, it serves as one of the core outcomes of the SQELT Erasmus+ project, thus contributing to the exploitation and dissemination of the project’s results. Fifth, it brings together authors from various institutions and fields of activity, thus also contributing to bridging the theory-practitioner gap.

Overview of the articles in this special issue

In the first article, Cláudia Sarrico discusses the role of performance indicators when high-participation higher education systems are confronted, for example, with challenges for the quality of student outcomes, equity of access, societal relevance and financial sustainability. Against this background, Sarrico weighs the strengths and weaknesses of current performance indicators and assesses the perennial tension between responsible accountability and the innovation possible from using performance indicators for organisational development.

Maarja Beerkens focuses on developments of performance data in higher education related to digitalisation that occurred in the last few decades. She states that performance indicators have become an integral part of governance and management of institutions as well as the higher education system, while simple input measures have been increasingly complemented by more sophisticated output measures. Since the digitalisation of learning and teaching, management and communication has created new options for the use of ‘big data’, Beerkens explores what this might mean for higher education governance, for example with respect to improved performance elucidation as well as technical, ethical and policy challenges.

Philipp Pohlenz starts from the assumptions that universities are supportive of innovation, professional approaches to teaching and evidence-informed management practice. With regard to evaluation practice in higher education, however, he states that often tensions exist, for example, between teaching evaluations based on predefined quality indicators and the need for more openness and error-tolerance if teaching should be innovative. Pohlenz thus discusses innovation, professionalisation and evaluation as interrelated concepts that should be integrative part of an adaptive, agile and trust-based quality management.

The basics of contemporary theories of learning and teaching and the learning and teaching process with a focus on higher education constitute the starting point of Theodor Leiber’s article. He analyses how far and in which ways performance indicators grasp aspects of theories and practice of learning and teaching. For a few exemplary performance indicators, the article shows that they are justified by theories and practice of learning and teaching by means of material inference. The analysis is meant to contribute to better understand the epistemology of performance indicators of learning and teaching.

Based on the benchlearning exercise among the six SQELT partnership universities (SQELT-GL, 2020, 8ff.), Maria J. Rosa, James Williams, Joke Claeys, David Kane, Sofia Bruckmann, Daniela Costa and José Alberto Rafael present one of the two articles which draw more directly on SQELT by exploring in which ways and to what extent learning analytics is implemented at the six project partner universities, including its multiple functions and ethical issues. The analysis reveals that learning analytics is present, in need to be further improved and seen to be useful in providing novel and more accurate data on the personalised learning process of students. This may, according to the authors, contribute to more sophisticated quality management systems. Opportunities and threats of learning analytics in the six SQELT partnership universities and beyond are also discussed.

The second article relying on the above-mentioned benchlearning exercise is presented by Giovanni Barbato, Justyna Bugaj, David F.J. Campbell, Roberto Cerbino, Piotr Ciesielski, Agnieszka Feliks-Długosz, Manuela Milani and Attila Pausits. They discuss exploratory findings on the assessment of a performance indicator set for learning and teaching developed in the SQELT project by members of four stakeholder groups (leadership, quality management staff, teachers, students). The authors argue that the quality of learning and teaching should be assessed through a holistic approach though it emerges that some domains of learning and teaching, namely teaching and learning competences and learning outcomes, prove particularly important for quality enhancement. Findings also show that different stakeholders perceive the usefulness of performance indicators quite differently. According to the authors, the insights of the research may be useful for diagnostic activities or strategic endeavours to complement existing quality management systems in learning and teaching.

In the seventh article, Jeroen Huisman and Bjørn Stensaker identify and discuss current trends and developments with respect to performance governance and management in higher education institutions. The authors argue that performance governance is changing through the ways performances are measured and documented and their assessments disseminated. In the view of Huisman and Stensaker meaning creation and higher education institutions’ achievements to deal with current societal challenges are coming to the fore. Finally, the article introduces recent policy initiatives and practices related to performance and accountability and showcases potential implications for institutional management and leadership.

In summary, the articles of this special issue aim at giving an, unavoidably selective, overview of the state of the art of PDGM in higher education, particularly in learning and teaching. In so doing, the different contributions provide information on the levels of development and options for future perspectives.

Acknowledgments

The author would like to thank the European Commission (EC) for co-funding the Erasmus+ project SQELT (grant no. 2017-1-DE01-KA203-003527). However, the EC support for producing this publication does not constitute an endorsement of the contents, which reflect the views only of the author, and neither the EC nor the project’s national funding agency DAAD can be held responsible for any use which may be made of the information contained therein.

Disclosure statement

No potential conflict of interest was reported by the author.