The role of performance measurement and management systems in changing public organizations: An exploratory study

IMPACT

The most recent administrative reforms aim not only to open up public organizations to citizens but also to encourage other organizations to take similar initiatives, namely in collaboration, transparency and the availability of information. So they need performance measurement and management systems (PMMS) that allow them to be closer to their stakeholders and dynamically promote effective organizational change. This article shows that a PMMS can be a mediation instrument in the effective implementation of administrative reforms. It will help managers and employees of public organizations understand how to improve performance measurement processes and mitigate dysfunctional behaviours that cause unintended effects on organizational performance.

ABSTRACT

In recent decades, governments worldwide have used administrative reforms to change public organizations. The initial concerns were to increase their efficiency but, more recently, it has been their openness to society. Performance measurement and management systems (PMMS) have been used on this changing path to support public managers’ decision-making, although not always successfully. This study contributes to the current debate on using PMMS to align public organizations with the needs of citizens and remaining stakeholders.

KEYWORDS:

• Administrative reforms
• effectiveness
• mediation
• organizational change
• organizational performance
• performance management
• performance measurement
• processes interaction

Introduction

Over the past few decades, public sector organizations have been facing challenges in responding to new technological, social and business realities. Governments have been implementing successive administrative reforms to respond to these challenges and improve the performance of public organizations, for example New Public Management (NPM) which streamlined public organizations through downsizing, decentralization and privatization (Hood, 1991). However, criticism of the principles that guided the creation of the NPM has grown (Reiter & Klenk, 2019), pointing to the fragmentation of organizational structures (Donadelli et al., 2020) and the complexity of relationships with stakeholders (Hwang, 2019). In response to these criticisms, the public administration literature has changed the focus of discussion from NPM to other management approaches, namely co-operation, networking, governance and stakeholder involvement in formulating and implementing public policies (Kim, 2021). Despite the changes introduced by successive administrative reforms, research evidence of their effects on organizational performance is still lacking (Hammerschmid et al., 2019).

As a key instrument for successful administrative reforms implementation, performance measurement and management systems (PMMS) seek to enhance the efficiency and effectiveness of public resources, helping governments to deliver on their promises of democratic responsibility and better service delivery (Smith et al., 2021). However, little is known about the real consequences of using PMMS (Yetano et al., 2021) and criticisms of their lack of effectiveness have been increasing (Dhillon, 2022; George et al., 2018). The evidence of their effects on organizational performance is inconclusive (Gerrish, 2016; Gigliotti & Sorensen, 2022) and managers continue to struggle to exploit the full potential of the information they provide (Deschamps, 2022). In addition, the investigation of pubkic management reforms globally has not been able to keep up with the same pace of the change in public organizations and their effects on organizational performance (Chang, 2021; Hammerschmid et al., 2019; Widianto et al., 2021). We sought to answer two research questions:

• How do PMMS components interact in public organizations?

• What is the contribution of PMMS to the effective implementation of administrative reforms?

To answer these questions, we explored the relationship between the implementation of administrative reforms and the performance of public organizations, analysing the mediation role of PMMS—see Figure 1.

Figure 1. Conceptual framework.

The data used to analyse this framework was obtained through a survey of public officials and managers working in several public sector organizations in the state of Ceará in Brazil, which has recently implemented administrative reforms.

This article makes important contributions to the public administration literature. It provides empirical research on the functioning of PMMS: identifying their role as mediation mechanisms for change in public organizations and how they contribute to increasing organizational performance. It also analyses the impact of administrative reforms on the performance of public organizations from a comprehensive perspective. This approach enables practitioners to see the effect of these reforms beyond the efficient use of public resources.

Literature review

Administrative reforms

Governments have periodically implemented administrative reforms to modernize their public administration and improve service delivery with lower costs and prices (Bianchi & Xavier, 2017). These reforms aim to improve organizational performance (Hameduddin & Fernandez, 2019) and implement government policy proposals (Lichtmannegger & Tobias, 2020). However, implementing administrative reforms has not always improved organizational performance, especially from a comprehensive perspective that encompasses more than mere efficiency (Reiter & Klenk, 2019).

The reform movement most mentioned in the literature on public administration is NPM. Supported by the OECD, it began in the 1970s in the UK and soon afterwards in Australia and New Zealand (Xiaolong & Christensen, 2019). The initial objectives of the NPM were to promote lower personnel expenditures, privatization, decentralization, increased automation of public services and international and intergovernmental co-operation (Hood, 1991). However, the first criticisms of NPM implementations were not long in coming.

Critics of NPM have argued that the differences between public and private organizations are so significant that business practices should not be transferred to the public sector (Boyne, 2002). A critical implication of introducing market practices into the public sector was that the interaction between various partners, who often enjoyed a great deal of autonomy from the state, became increasingly complex (Hwang, 2019). It was also found that downsizing would negatively influence the quality of services (Hammerschmid et al., 2019).

In the late 1990s, the post-NPM reform movement emerged due to NPM’s shortcomings—mainly the increased fragmentation and lack of control resulting from the initial NPM implementations (Donadelli et al., 2020). These new administrative reforms tried to overcome these shortcomings by advocating re-centralization and re-regulation and by strengthening governance capacity through the improvement of control and coordination mechanism (Hwang, 2019). Furthermore, the active participation of citizens based on their rights has become a concern of governments, moving away from the customer's view (Donadelli et al., 2020). New reforms have often been added to the old ones, producing hybrid reforms (Xiaolong & Christensen, 2019). Therefore, the post-NPM is understood as a model that succeeds the NPM period and, principally, as a movement to improve NPM (Reiter & Klenk, 2019).

The NPM and post-NPM paradigms, despite their intense scrutiny in the literature, remain poorly defined as explanations for the changes that have taken place in public administration (Laffin, 2019). Perhaps for these reasons, more recently, new reform movements have emerged, seeking to promote the openness of public organizations to society. One of these movements advocates network governance, aiming to build successful relationships between relevant actors that facilitate a culture of mutual respect and share learnings to serve the public interest (Kim, 2021). Another reform movement advocates government openness and has played a significant role in administrative reforms during the past decade, which has driven many countries to implement initiatives related to information availability, transparency, participation, collaboration and information technology (Gil-Garcia et al., 2020). This movement aims to change public organizations, promoting their openness to citizens and encouraging other organizations to take similar initiatives (Ingrams, 2020).

According to the recent literature, it appears that public organizations, unlike in the past, are continuously changing through administrative reforms that aim to improve the use of their resources and meet the needs of citizens. However, not all these reforms necessarily result in successful change that improves organizational performance (Lichtmannegger & Tobias, 2020), especially from a comprehensive performance perspective (Reiter & Klenk, 2019).

Bearing in mind that governments use administrative reforms to improve the performance of public organizations, our first hypothesis was:

H1: Implementing administrative reforms is directly and positively associated with improving organizational performance.

Performance measurement and management

Although performance measurement and performance management are sometimes used with similar meanings, the literature also identifies them as two distinct and interrelated processes integrated into systems (Henri, 2006; Koufteros et al., 2014; Smith & Bititci, 2017). These systems have become essential instruments in supporting the decision-making process of managers of business and public organizations.

In the public administration literature we reviewed, performance measurement was often associated with operational aspects such as the definition of a measurement object, the development/formulation of performance measures, data collection, data analysis, its transformation into information and communication to internal and external stakeholders (Eliuz et al., 2017; Van Dooren et al., 2015). Performance management is often associated with other aspects, such as improving people's motivation levels and engaging them in ongoing changes, participation in plan formulation and performance evaluation, performance counseling, the use of performance information in the decision-making process, improving the assessment of programme effectiveness, recognizing and celebrating successes and demonstrating value to stakeholders (Ateh et al., 2020; Behn, 2003; Brusca et al., 2017; Plaček et al., 2020; Rabovsky, 2014).

Therefore, performance measurement and performance management in public organizations, as in a business context, can be defined as two distinct and interrelated processes integrated into a system. The difference between PMMS in business and public organizations may be how these two processes interact.

The growing use of PMMS in the public sector was driven by administrative reforms (Johnsen, 2015). As the public sector environment has become more complex, and economic resources and financial budgets have tightened, the focus on performance measurement and management has become more critical (West & Blackman, 2015). Therefore, PMMS have become essential instruments of administrative reforms (Micheli & Pavlov, 2020). They have a significant potential to promote critical components of current administrative reforms, such as efficiency, transparency, accountability and governance in public organizations (Taylor, 2021). The use of these systems strengthens the accountability processes and the behaviour of managers (Gao, 2015), developing positive forces in the administrative process (Andrews & van de Walle, 2013) and leading to efficiency gains (Pollitt & Dan, 2013). As such, using performance information in decision-making processes on an ongoing basis to meet predefined goals will have positive effects on organizational performance (Hall et al., 2022; Nitzl et al., 2019), although the literature also presents non-significant and negative results for these effects (for example Christensen & Laegreid, 2007; Dobrolyubova, 2017).

Unlike in business organizations, the use of performance information in public organizations is not restricted to direct performance improvement. This information can also be used to gain legitimacy and support, promote organizational learning and negotiation, provide accountability, support to stakeholders and access to resources like money and political support (Eckerd et al., 2021). Based on the literature reviewed, public organizations use PMMS adapted to their organizational realities to enhance the implementation of administrative reforms and increase organizational performance. Hence our second hypothesis:

H2: PMMS mediate the positive relationship between the implementation of administrative reforms and the organizational performance

Methods and measurement

Sample and data collection

Following NPM ideas, the Brazilian government launched the ‘Master plan for public administration reform’ in 1995. Later, a significant political shift in the government marked the transition to post-NPM reforms through initiatives that promoted public participation and transparency (Donadelli et al., 2020), following the Open Government Declaration of 2019 (Schnell & Jo, 2019). Thus, administrative reforms in Brazil have attempted to modernize public organizations and increase stakeholders’ participation (Gomes & Lisboa, 2021). The different states of Brazil interpreted and implemented these reform movements at different paces. Ceará’s government began the 2000s with the main goals of reducing costs and increasing citizen satisfaction, seeking to increase the efficiency of processes and the effectiveness of public policies and promoting ethics and transparency. In this context, the government has implemented performance measurement and management initiatives to help achieve these goals. Currently, Ceará’s government uses a balanced PMMS that links processes and outcomes to align the efforts of all public organizations with the government's strategy.

To meet the objective of this study, managers and employees of public organizations in Ceará were sent questionnaires in 2019. Our questionnaires were sent to 1,213 participants and 260 valid responses were received—a response rate of 21.43%. Although this is not a high rate, it is comparable to response rates from similar studies (Nitzl et al., 2019; Verbeeten & Speklé, 2015). Table 1 presents the characteristics of the respondents and the organizations where they worked, including information about gender, age, education level and organizational function.

Table 1. Descriptive statistics of respondents.

		Frequency	Percentage
Gender	Male	130	50.0
	Female	130	50.0
	Total	260	100.0
Age	18– 25	1	0.4
	26– 35	37	14.2
	36– 45	96	36.9
	46– 55	77	29.6
	56–65	39	15.0
	> 65	10	3.8
	Total	260	100.0
Educational level	Technical level/high school	2	0.8
	Graduate	25	9.6
	Postgraduate	116	44.6
	Master’s degree	102	39.2
	PhD	15	5.8
	Total	260	100.0
Function	President/secretary	4	1.5
	Director/co-ordinator	65	25.0
	Manager/executive	80	30.8
	Employee	111	42.7
	Total	260	100.0
Type of organization	Secretaries	131	54.4
	Executive agencies	9	3.5
	Foundations	10	3.8
	Public companies	30	11.5
	Mixed economy society	11	4.2
	Legislative and courts of auditors	54	20.8
	Judicial and prosecutor's office	15	5.8
	Total	260	100.0

The sample includes various types of public organizations in the state of Ceará. It also includes respondents with different education levels and has a balanced gender distribution.

Measures

The measurement items included in the questionnaire were selected based on a comprehensive literature review. To measure the success of the administrative reform implementation, we used an adapted version of the construct created by Kellough and Nigro (2006) through a seven-point Likert scale (1 = strongly disagree to 7 = strongly agree).

The PMMS was measured by the constructs used by Henri (2006) through a seven-point Likert scale (1 = never to 7 = every time). The performance measurement construct included items related to monitoring results, their comparison with the goals and expected results and the review of the key measures. The performance management construct comprised items related to continuous improvement dynamics, including sharing and discussing the main organizational challenges and promoting a strong organizational culture driven by all the internal stakeholders.

Organizational performance was measured through a well-known comprehensive measure of public organizations’ performance developed by Van de Ven and Ferry (1980) and adopted and validated by other authors (for example Nitzl et al., 2019; Verbeeten, 2008; Verbeeten & Speklé, 2015). This includes items related to quantity, quality, efficiency, innovation, reputation and employee morale.

All questionnaire items, originally written in English, were translated into Brazilian Portuguese and later into English by experts with knowledge of both languages. The comparison between the two English versions showed no relevant differences. To validate the content of the questionnaire, we asked a panel of experts in the field of public administration, including professors, researchers and professionals, to verify that the terminology of the questions was consistent with the respondent's level of understanding (Forza, 2002).

To analyse the relationships presented in the conceptual model, we used structural equation modeling (SEM), through the two-step method. Thus, the relationships between the constructs were only estimated after the measurement model results had reached the standards defined in the literature (Byrne, 2010). For data analysis, we used the IBM-AMOS Version 24 software.

Results

The initial model was modified based on successive interactions, considering item reliability, standardized residuals’ covariances and modification indices (Byrne, 2010; Hair et al., 2014). The measurement items and related statistics are presented in Table 2. None of the variables showed skewness and kurtosis values—indicating no substantial violations of the normal distribution, thus fulfilling the assumptions of using structural equation models (Hair et al., 2014)

Table 2. Statistics of measurement items.

Measurement items	Mean	SD	Skew.	Kurt.
Administrative reform
The provisions and purposes of the CSRL have been clearly communicated to employees of my organization	3.60	1.753	0.031	-1.236
The CSRL has made the workforce of my organization more productive and responsive to the public	3.83	1.666	-0.035	-1.062
I believe my organization has made good use of the greater discretion it has under the CSRL	4.14	1.459	-0.301	-0.326
Under authority provided by the CSRL, my organization has established an effective human resources programme	3.73	1.554	-0.028	-0.666
It has been possible to terminate low performers without major procedural delays in my organization	3.65	1.490	-0.075	-0.721
The CSRL causes employees to be more responsive to the goals and priorities of administrators	4.03	1.631	-0.274	-0.824
Under authority provided by the CSRL, my organization can hire highly qualified people in a timely manner	3.54	1.668	0.178	-0.845
Performance measurement
Track progress toward goals	4.63	1.301	-0.471	0.218
Monitor results	4.74	1.372	-0.472	-0.119
Compare outcomes to expectations	4.47	1.402	-0.417	-0.293
Review key measures	4.33	1.355	-0.327	-0.331
Performance management
Enable discussion in meetings of superiors, subordinates and peers	4.42	1.493	-0.356	-0.661
Enable continual challenge and debate data, assumptions and action plans	4.22	1.506	-0.281	-0.657
Provide a common view of the organization	4.26	1.484	-0.362	-0.418
Tie the organization together	4.08	1.477	-0.334	-0.521
Enable the organization to focus on common issues	4.18	1.314	-0.282	-0.464
Enable the organization to focus on critical success factors	4.25	1.428	-0.303	-0.328
Develop a common vocabulary in the organization	4.25	1.465	-0.431	-0.242
Organizational performance
The quantity or amount of work produced	5.00	1.306	-0.635	-0.192
The quality or accuracy of work produced	4.86	1.309	-0.678	-0.001
The number of innovations or new ideas	4.41	1.421	-0.33	-0.605
Reputation of ‘work excellence’	4.78	1.413	-0.498	-0.371
Attainment of production or service goals	4.85	1.270	-0.457	0.001
Efficiency of operations	4.62	1.337	-0.407	-0.387
Morale of personnel	4.59	1.445	-0.258	-0.604

Notes: SD = Standard deviation; Skew. = Skewness; Kurt. = Kurtosis; SRF = civil service reform law.

The final measurement model presented an adequate fit to data (χ²= 737.28, df = 269, χ²/df = 2.741, GFI = 0.813, CFI = 0.921, TLI = 0.912, PCFI = 0.826, RMSEA = 0.083). Concerning item reliability, all factor loadings were statistically significant and reasonably strong. Table 3 shows construct validity, reliability and discriminant validity (Fornell & Larcker, 1981; Hair et al., 2014).

Table 3. Measures of construct validity and reliability.

	AR	OP	PMa	PMe	α	CR	AVE
Administrative reforms (AR)	0.753				0.899	0.901	0.567
Organizational performance (OP)	0.420	0.777			0.913	0.914	0.604
Performance management (Pma)	0.535	0.578	0.883		0.960	0.961	0.779
Performance measurement (PMe)	0.416	0.627	0.857	0.914	0.952	0.953	0.836

Notes: α = Cronbach's Alpha; CR = Composite Reliability; AVE = Average Extracted Variance. The diagonal present the square root of AVE value for each construct (in bold).

No significant concerns about common method bias were found, based on the results of Harman's single factor test and common latent factor approach (Podsakoff et al., 2003). Figure 2 presents the results of the estimated parameters for the structural research model.

Figure 2. Results of the structural model. *Significance level < 0.01.

The direct, indirect and total effects results for all tested relationships are presented in Table 4. According to these results, the direct effect between administrative reforms and organizational performance was positive (0.186) and significative (p < 0.01). As such, hypothesis H1 was supported. The indirect effect between these two variables was also positive (0.239) and significant (p < 0.01), supporting hypothesis H2. Therefore, PMMS play a decisive role as mediators between administrative reforms and organizational performance.

Table 4. Standardized estimates of the effects.

Relations	Direct effect		Indirect effect		Total effect
	Estim.	Sig.	Estim.	Sig.	Estim.	Sig.
Administrative reform → Organizational performance	0.186	*	0.239	*	0.426	*
Administrative reform → Performance management	0.533	*	–	–	0.533	*
Administrative reform → Performance measurement	-0.041	NS	0.469	*	0.428	*
Performance management → Organizational performance	0.026	NS	0.464	*	0.489	*
Performance management → Performance measurement	0.880	*	–	–	0.880	*
Performance measurement → Organizational performance	0.527	*	–	–	0.527	*

Notes: Estim. = estimate; Sig. = significance; *Significance level < 0.01; NS = not significant.

Table 4 shows how the PMMS components interact to ensure the mediation effect. On the one hand, the direct effect between administrative reforms implementation and performance measurement was not significant (p > 0.05) and the direct effect between performance management and organizational performance was also not significant (p > 0.05). These results mean that there were no independent mediation effects of each PMMS component.

On the other hand, we found that the direct effect between administrative reforms and performance management (0.533), the direct effect between performance management and performance measurement (0.880) and the direct effect between performance measurement and organizational performance (0.527) were all significant (p < 0.01). These results mean that the PMMS mediates the relationship between the implementation of administrative reforms and the improvement of organizational performance through a double mediation of its two components.

Discussion

The initial priority of the NPM reform movement was the implementation of PMMS, which would become an instrument for more efficient and effective use of public organizations’ resources. After almost three decades, it would be expected that these systems would play a decisive role in improving organizational performance—particularly in implementing administrative reforms. However, based on our results, these systems do not fully explain the positive effect that the implementation of administrative reforms has on increasing organizational performance. Our results may mean that, over successive administrative reforms, public organizations have incorporated a learning culture that allows them to react more effectively to change—regardless of the existence of PMMS. It seems that, in the case of Brazilian public organizations, PMMS have not only been used as a control mechanism but are playing a broader role that contributes to organizational learning (Andersen & Nielsen, 2020; Eckerd et al., 2021). Thus, the role of PMMS would be to increase organizational performance through the internal discussion of the processes for implementing administrative reforms and promoting a continuous learning culture.

The analysis of the results related to the mediating variables, which represent the processes of performance measurement and performance management and the interrelationship between them, provides a better understanding of the role of PMMS in increasing organizational performance. These results identified a double mediation in which the performance measurement process assumes a fully mediating role between the performance management process and the organizational performance of public organizations. They show that performance measurement plays a decisive role in promoting organizational performance in highly procedural organizations, such as public organizations.

The results of this study seem to explain how PMMS drives a dynamic process that keeps public organizations in continuous alignment with the changes occurring in their external environment. To this end, the performance measurement processes ensure that organizational resources remain aligned with the government policy proposals through administrative reforms. Performance management processes ensure the reconfiguration of performance measurement processes when organizational change is required. While not acting directly to improve organizational performance, performance management will help managers and employees understand how to improve performance measurement processes. Therefore, there seem to be distinct characteristics in the relationship between performance measurement and performance management processes not found in the public administration literature until now.

The results of this study may also explain why the use of performance measurement processes alone will hardly improve organizational performance levels (Smith et al., 2021). In this context, the literature reports dysfunctional behaviours of employees and managers that cause collateral and unintended effects on the organizational performance of public organizations (Bianchi & Rua, 2022; Lewis, 2015; Siverbo et al., 2019; Taylor, 2021). As such, these behaviours can be mitigated through performance management processes that encourage discussion of action plans among all internal stakeholders of the organization, provide a shared view and vocabulary and focus on common problems and critical success factors. The performance management process can define the focus of performance measurement in aligning stakeholders’ needs and objectives (West & Blackman, 2015) and can also strengthen citizens’ participation (Kroll et al., 2019). Therefore, the central role of performance management is to develop performance measurement processes that dynamically ensure organizational performance improvement.

These results also align with the literature that points to the critical role of managers of public organizations in performance management processes, promoting negotiation and seeking commitments between different stakeholders (Eckerd et al., 2021). As such, managers and users of PMMS play a crucial role in implementing administrative reforms and in how they influence organizational performance (Igalla et al., 2020; Pilonato & Monfardini, 2020).

Conclusions

The performance of public organizations should be viewed from a comprehensive perspective by considering a range of dimensions that go beyond production and financial efficiency, including the quality of services provided and the way they meet the needs of their internal and external stakeholders. Based on a recent literature review on public administration, we verified the existence of two important gaps. We found that, as expected in public organizations, even when administrative reforms are implemented efficiently, they do not always increase organizational performance, viewed from a comprehensive perspective. We also found that, although PMMS are already in the maturity stage in public organizations due to the NPM reforms initiated more than 40 years ago, there are few studies on how they are used and their consequences.

Limitations

Although this article addresses relevant theoretical and practical issues, it has the limitations of cross-section and survey-based research. Our sample used is specific to Ceará’s public sector. Any generalization of the results should be made with caution. Future research should conduct similar research at the whole government level and in other contexts that are not emerging economies like Brazil but are distinctly either developed or developing countries.

Although comparable to similar studies, the sample size prevented further analysis, for instance, testing some control variables, including the type of public organization effect on organizational performance. Future research should use other methods, like case studies and qualitative research approaches to analyse other important variables that can influence organizational performance, such as the public employees’ educational background and engagement with particular types of PMMS.

Contributions

The research findings make two significant contributions to the theory and practice of public administration:

• First, they provide empirical research on the interconnections between PMMS components, identifying their role as mediation mechanisms for change in public organizations. We found that performance measurement processes directly influence organizational performance, while performance management processes have a leadership role. Although performance management does not directly affect organizational performance, it contributes to the continuous improvement of performance measurement processes. From a practical point of view, identifying these two processes and how their interaction could work makes it possible to enhance their use and prevent PMMS from becoming merely symbolic systems, as reported in the literature, to justify its failures. Thus, the managers of these organizations will be able to extract more value from the information provided by the PMMS and contribute to an organizational learning culture.

• Second, the findings of this study enable researchers and practitioners to analyse organizational performance from a comprehensive perspective, avoiding the minimalist approaches of the efficient use of resources, extending this analysis to the dimensions of innovation, quality, reputation and the morale of the organization's members.

Overall, based on the findings of this study, it seems that Brazilian public organizations are using PMMS to follow an open system path and to get closer to their stakeholders.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Samuel Leite Castelo

Samuel Castelo is Assistant Professor of Administration and Public Accounting at Ceará State University, researcher at Center for Applied Social Studies and External Control Auditor of the Ceará State Court of Auditors, Brazil. His main research interests are public budgeting, administrative reforms, government performance and public governance.

Carlos F. Gomes

Carlos F. Gomes is Associate Professor at the Faculty of Economics at the University of Coimbra and researcher at the CeBER—Centre for Business and Economics Research. His main research interests are performance management, performance measurement and operations strategy.