Performance management and evaluation meets culture and politics: Australia’s experience

Abstract

An underlying tension exists between the fundamental elements of accountability, government decision-making, performance management, evidence-based policy and the measurement of results. This article considers this through three lenses: evaluation of government policies and programs; program structures and performance indicators; and the challenges of big data. While use of new technologies and big data opens up opportunities for strengthened evidence-based policy, this does not address the tension between a disciplined results-based approach to government decision-making and the inherently political nature of these decisions. There remain also important challenges to the effective development and use of big data in government.

KEYWORDS:

• Public administration
• performance management
• evidence-based policy
• big data

Introduction

Reflecting upon 30 years of experience in the Australian Public Service (APS), including as the Public Service Commissioner,¹ Andrew Podger identified a key tension in modern public policy formulation:

Performance management encourages a disciplined results-based approach but has never been (and can never be) the only way in which decision-making in government occurs; much government decision-making is inherently political, requiring judgments on priorities, and also often involves reaction to events where performance information is lacking. (Podger, 2018, p. 120)

This tension, between the measurement of results, performance management, accountability, evidence-based policy and government decision-making, is the focus of this article. We consider this through three lenses. The first is policy and programme evaluation, its role in accountability, and as a basis for evidence-based policy making.² The second considers the historical evolution of how one policy, income support for the unemployed, has been identified in programme structures and performance indicators. The third considers the implications of this tension for the role of big data and technology.

Evaluation

Bray et al. (2019) identify three aspects of evaluation central to the tension between a disciplined results-based approach and the political realities of government decision-making. The first concerns the scope of evaluation and whether policy and programmes should be evaluated solely against their own specified goals – or more widely. The second is the nature of accountability and programme evaluation. This is whether evaluation is primarily a tool for government to assess its own performance, and that of the public service, in delivering on their policies and programmes,³ or if there is a wider accountability framework, and if there is.how should this be constructed. The third is how the management of evaluation within the public service should be structured and organised.

The scope of evaluation

Evaluation is frequently framed as being concerned with the “efficiency, effectiveness and appropriateness of policies” (Volker, 1996). While the first two of these criteria can be considered uncontroversial, the third raises a more significant question. This is whether evaluation within the public service, or managed by the public service, has a role in forming a view which may relate to the political decision-making in the formulation of the policy or programme.

This question is posed by O’Faircheallaigh, Wanna and Weller as:

even where policy-makers or program managers do have clear objectives and pursue them vigorously, should these objectives be accepted as the basis for evaluation? It is quite clear that a range of individuals and interests (‘stakeholders’) will be affected by any policy or program, and that their goals might differ from those of the policy maker and/or organisation charged with program delivery (O’Faircheallaigh et al., 1999, p. 195)

Sharp (2003) provides some insight into how the Australian approach to this has evolved, especially the role played by a Senate report – the Baume Report, Through a Glass Darkly. While notionally this report was focused on the evaluation of health and welfare services, it addressed evaluation more generally, and Sharp concludes “it also provided successive governments with a recommended definition of program evaluation for the whole of government, as ‘the process of thoroughly and critically reviewing the efficiency, effectiveness and appropriateness of any program or group of programs’” (Sharp, 2003, p. 7). Sharp identifies that while “the Baume report suggested, inter alia, that appropriateness was ‘the extent to which the program is in accordance with the moral, ethical and social norms of the time’” (p. 7) that “however, with governments becoming more wary of the independence and authority of the public sector, the interpretation of appropriateness soon became firmly grounded in ‘Government policy’”. (p. 7)

This tension can also be seen in Volker’s⁴ (1996) discussion of the criterion of appropriateness which he defines as “how consistent are those objectives with the needs of the client group and wider government policy considerations” (p. 157). He suggests though “it may be difficult for evaluators to lend the same weight as the first two [efficiency and effectiveness]” (p. 157).

Other conceptualisations of the evaluation process omit the concept of appropriateness entirely. Rossi and Freeman (1989, p. 44) refer to the outcome evaluation phase as simply comprising “assessment of program effectiveness and efficiency”. In a similar fashion the UK Magenta book effectively puts the question of appropriateness outside of the direct evaluation task: “policy evaluation that shows a positive impact and good value for money does not mean that it was an appropriate policy, similar or better gains may have been realised by alternative policies that have not been evaluated” (Treasury (UK), 2011, p. 133).

Evaluation – accountability to whom?

While evaluation is recognised as a key part of the accountability framework, a central question remains as to whether it should be internal to government or have a wider role.

In 2014, Peter Shergold⁵ reviewed government processes for implementing large programmes and projects. This followed the Report of the Royal Commission into the Home Insulation Program.⁶ The Shergold report Learning from Failure: Why large government policy initiatives have gone so badly wrong in the past and how the chances of success in the future can be improved emphasised the importance of evaluation both directly and as central to public engagement and participation, recommending that:

the results of policy and program evaluations should be made publicly available as a matter of course. With the huge advances in digital technology, shared information can build a stronger relationship between government and citizens. Equally important, individuals can often find new methods of using public information in unexpected ways to create public benefit. (Shergold, 2015, p. 23)

A more explicit and expansive call for external accountability was made by Alexander and Thodey in their 2018 review of the operation of the Public Governance, Performance and Accountability Act 2013 (PGPA Act) and Rule:

Citizens have a right to know how their money is used and what difference that is making to their community and the nation – what outcomes are being achieved, how, and at what price. Insightful performance reporting goes beyond simply measuring activities. It talks in terms of outcomes and impacts (the value created by these activities). (Alexander & Thodey, 2018, p. 11)

The accountability role of evaluation has also been central in recent Australian debate around proposals to establish an independent statutory position of an “Evaluator General” within the federal government⁷ (Gruen, 2018; Leigh, 2018). These proposals envisage the role of the position, which would report, and be accountable to the parliament, as both directly undertaking a range of public policy and programme evaluations, and having responsibility for the oversight of evaluation across the APS.

Reflecting these issues Bray et al. (2019) identified two broad options for the organisation of evaluation within the APS – with the crucial difference between these being this focus of accountability. The first approach would create a centralised function within the existing public service structure with responsibility for creating and sustaining a strong evaluation focus across the APS. Such an approach would effectively maintain existing accountability links between departments and their ministers, although enhancing them across government including through the cabinet process. The second model would see the establishment of an evaluation agency as an independent statutory function reporting to the parliament, rather than the government, with this agency both undertaking some evaluations and seeking to promote the evaluation function within departments.

With regard to these strategies they conclude:

Each approach has strengths and weaknesses. It is our view that the internal approach would be a stronger mechanism for building an evaluation culture within the APS, although potentially with a primary focus on efficiency and effectiveness within the program or policy framework. In contrast, the second approach may be more effective in increasing the external focus on the evaluation of some government programs and extend the ambit of evaluation to encompass more aspects such as appropriateness. (Bray et al., 2019, p. 17)

Central to this question is the boundary between performance management as an independent function of good administration, and the role of government with its inherent political goals and interpretations.

Effective management of evaluation

The third issue drawn from Bray et al. (2019) concerns the management of evaluation. Central to this was the question of whether the challenges to improving evaluation in the public service were primarily structural and organisational, rather than more pragmatic operational concerns, such as the skills of the public service to effectively undertake evaluations, or the absence of information, or means to collect information to effectively enable evaluation to be conducted.

As background to this question they noted a strong divergence between what was recognised as being good public management and actual practice:

It is almost universally acknowledged that a systematic approach to high-quality evaluation of policies and programs embedded in APS agencies, as part of a broader approach that reflects on and learns from their own and others’ successes and failures, is pivotal to producing evidence to guide appropriate policy settings and effective and efficient public sector management.

However, it is equally widely acknowledged that such an approach is not reflected in current APS practice. (Bray et al., 2019, p. 8)

This focus is not new, and the authors cited the conclusion of David Shand, some twenty years earlier, that: “the major issues of evaluation are management rather than methodological ones; they are about how and where evaluation should be organized, located, planned and managed to best affect decision-making” (Shand, 1998, p. 15).

While in the first instance this concern can be seen as being quite different to those relating to the scope and accountability of evaluation, there are grounds to question this perspective. This can be seen when the question of “why has this problem developed, and why has it persisted?” is considered. The paper sought to summarise the possible reasons as:

In their day-to-day activities, departments and agencies tend to focus on achieving the ‘here and now’ priorities of program and policy implementation and of ministers. In examining their past performance, departments and agencies are often more concerned with reputational risk, seeking to pre-empt or divert criticism rather than learning from experience and feedback. In this context, evaluation is often also seen as yesterday’s news or a second-order and lower-priority issue. Evaluation processes and findings then become just one more thing that needs to be defensively managed, or an opportunity for quick grabs to justify current and future decisions and activities. (Bray et al., 2019, p. 9)

While on one hand the defensiveness alluded to in this statement can simply be attributed to a desire of individuals and organisations to seek to protect themselves, an alternative explanation concerns the programmes themselves. That is, notwithstanding their nominal objectives, policies and programmes are also shaped by unstated political criteria and priorities, and the public service recognises that evaluation can, or indeed will, reveal this.

Another was whether the evaluation function is decentralised to the responsible agency and closely aligned with policy and programme development and delivery, or is more centralised. The centralised approach has the potential to attach a higher priority to evaluation and to create the opportunity for a more strategic and coordinated approach to this function across government and departments. It also has the potential to reduce the variability in quality which can be associated with a dispersed function, and promote the independence of evaluations. The discussion in Australia is happening at a time when the APS evaluation function is highly decentralised.

This has however not always been the case. Rather the degree of centralisation has ebbed and flowed. For example, in 1988 Cabinet adopted a formal strategy for evaluation. This required, amongst other things: periodical evaluation of all programmes; inclusion of evaluation plans in all new policy proposals; and the reporting of major evaluation findings in budget documentation. While departments remained primarily responsible for determining evaluation priorities and conducting evaluations, the Department of Finance had a central role in providing advice, support and training (Mackay, 2011). From 1996, in contrast, a series of changes were made to public sector management which included deregulation and devolution of evaluation, with Finance no longer being involved in quality control.

There are several reasons as to why the more formal and partially centralised approach was not sustained, including a lack of support at the most senior levels of the public service, and ministerial and other opposition to rigorous evaluation. It can also be considered that the quality control role was not well effected. On the other hand, the prominence the approach brought to the evaluation role resulted in some departments investing heavily in this function, and in doing so they built a strong evidence base and became much better placed in policy contestation with central agencies. This in turn may have led to central agencies being less than enthusiastic about this approach.

Programme objectives and performance indicators – the case of unemployment benefit

Performance frameworks which link budgets, programmes and outcomes have been a key element of the “New Public Administration” changes in Australia. The history of this has been recounted in Podger (2018), but it is worth noting it has developed over a long period, with, for example, Cutt emphasising in the early 1970s:

The fundamental premise of program budgeting is that policy and budgets are inseparable and that the relationship between the structure and implementation of budgets and the determination and achievement of policy objectives should be made explicit. (Cutt, 1973, p. 4)

In 2011 the Australian National Audit Office (ANAO, 2011, pp. 15–16) articulated these, as guidance for the development of Key Performance Indicators, in terms of the SMART criteria (Specific, Measurable, Achievable, Relevant, Timed).⁸

Here, in our second lens, we consider how these approaches have been applied to the articulation of the role and objectives of the Australian unemployment benefit payment. In doing so we note that the Department of Social Services (DSS), which has traditionally been responsible for this programme, was one of those that responded to the 1996 reforms, welcoming the increased emphasis on evaluations (and agreement to necessary resources), and successfully integrating the new approach with their strategic planning and programme management arrangements.

The Australian Government has provided unemployment benefits to persons who are out of work, capable and willing to take suitable employment and have taken reasonable steps to find work, since 1945.⁹

In considering this our focus is on some of the more recent developments in how this programme’s objectives and performance criteria are articulated. This takes the form of two snapshots of the articulation of these: the first in 1997–98; and the second, 20 years later, in 2017–18.

The 1997–1998 the DSS Annual Report, which was published just after the 1996 reforms, listed the “Objectives of the Income Security Program for the Unemployed” as (inter alia) “to ensure that unemployed people receive adequate levels of income to support themselves” (Department of Social Security [DSS], 1998, p. 85). It further detailed that performance was measured by monitoring trends in customer numbers and outlays, and a number of other key indicators, including “Adequacy: the relativity of basic payment rates to Male Total Average Weekly Earnings (MTAWE) and increases in rates to movements in the Consumer Price Index” (p. 99). Other indicators were cited in terms such as: “self-help”; “financial independence and incentives to work”; “Take-up”; “Administrative efficiency and customer service”; “Protection of human rights”; and “Control of incorrect payment and fraud” (p. 99). The rationale for these indicators, and the performance of the programme was then discussed in detail in some 24 pages of the Annual Report focused exclusively on the unemployed.

The 2017–2018 DSS Annual Report contains an “Annual Performance Statement”. The entire Social Security system is covered in just 29 pages. In this the over-arching performance objective of the social security system is stated as to “encourage self-reliance and support people who cannot fully support themselves by providing sustainable social security payments and assistance” (DSS, 2018a, p. 17). The performance criteria are: “sustainability of the payment system”; “Extent to which payment recipients have improved financial self-reliance”; “Extent to which payments are made to, or with respect to, people unable to fully support themselves”; and “Extent to which delivery meets program objective” (p. 21). The indicators detailed encompass concepts such as the “average future lifetime cost (in current year dollars) of total welfare payments to individuals”; and expected percentages relating to the population who will not receive income support, or will receive less income support.

The associated Portfolio Budget Statement for the Social Services Portfolio for 2018–19 contains one page for the “Performance Criteria for Program 1.10 Working Age Payments”. This posits the outcome as: “to assist people who are temporarily unable to support themselves through work or have a limited capacity to work due to disability or caring responsibilities for young children” (DSS, 2018b, p. 59). The Performance Criteria is that an “Agreement is in place with the Department of Human Services to provide assurance that payments … are made in accordance with relevant legislation, policy and guidelines” (p. 59).

This comparison shows several remarkable changes. The first is that while in 1997–1998 a key concern of the policy objective and the indicators were the “adequacy” of the payment, this entirely disappeared in 2017–18. The extent of this transformation is reflected in a more remarkable form in the context of the recent Senate Committee Inquiry into the adequacy of Newstart (the main unemployment benefit payment). For this the Department prepared the 69 pages, whole of government, submission (DSS, 2019) without mentioning the word adequacy other than in the title. Further, this submission mentions living standards only with respect to why employment outcomes are important, only mentions wellbeing in terms of psychological wellbeing, and only mentions poverty as a consequence of low educational attainment.¹⁰

The second change that can be seen in these approaches is a shift in the focus of the delivery-linked outcomes from goals such as “customer service” and the “protection of human rights” to a much more legalistic statement of the objective of the delivery of assistance as just being “in accordance with relevant legislation, policy and guidelines”.

While these first two changes can be considered as a narrowing of the programme objectives, the 2017–2018 goals also feature some additional criteria. These include: “sustainability of the payment system … extent to which payment recipients have improved financial self-reliance [and] … Average future lifetime cost (in current year dollars) of total welfare payments to individuals”. The inclusion of these criteria raise a number of issues. The first is the criteria go well beyond the direct operation of the programme to incorporate policy objectives that can be considered to be difficult to directly relate to the delivery of this programme by the department. The second is they do not appear to relate to the nature of the programme which is demand driven, in large part as a consequence of macroeconomic conditions, industry restructuring and climatic and other factors. Thirdly is a real question as to whether they can be effectively and contemporaneously measured.

From the perspective of performance management, this example raises some important questions. Firstly should these changes to the objectives of the programme, and hence the basis on which there are evaluated, be seen merely as appropriately reflecting differing political perspectives of governments, or are there some core public policy objectives which can be seen as continuing and should be a feature of the performance standards? Related to this is a second question of whether programme performance indicators are too readily open to variation and manipulation? Thirdly there are questions as to what extent should performance targets reflect broader agendas that can be considered to go beyond the actual policy and programmes?

Finally we need to ask whether the shift, as seen in the policy objectives for this programme, from a public management perspective away from considerations such as customer service and human rights, to that of efficiently delivering the programme in accordance with legislation, policy and guidelines, should be seen as a natural extension of a narrow managerial performance-based culture?

Data, control, access and management

The third lens through which we consider this question is that of data, and in particular the question of “big data” and performance management.

Here Australia has strong aspirations. The Government’s 2015 Public Data Policy Statement (Turnbull, 2015, p. 1) presented this as: “the data held by the Australian Government is a strategic national resource that holds considerable value for growing the economy, improving service delivery and transforming policy outcomes for the nation.”

In the 2018 Review of Australian Government Data Activities the current circumstances and opportunities were described as:

The Australian Government holds vast amounts of data collected from individuals and businesses, and generated by different government agencies.

In the past two decades, developments in computing power have opened up opportunities to use big data more efficiently, by linking information across different datasets and identifying previously indiscernible patterns and preferences …

Better use of data can improve the efficiency of government spending through more effective and better targeted – evidence-based – government policies, programs and services. It can also help solve some of the most complex policy problems and fuel future innovation (Department of Prime Minister and Cabinet, 2018, p. 3).

Reflecting these objectives, some significant recent initiatives in Australia have involved the development of a number of integrated data environments. These include:

• Multi-Agency Data Integration Data Project (MADIP). (Australian Bureau of Statistics [ABS], 2019a) This combines data from the Australian Census with administrative data on education, government payments, health, income and taxation. These data have to date almost exclusively been used within government, but steps are being taken to broaden access to include university-based researchers.

• The Business Longitudinal Analysis Data Environment (BLADE). (ABS, 2019b) This is an environment that allows a range of different business datasets to be linked including: Australian Bureau of Statistics (ABS) data such as the Business Characteristic Survey; the Economic Activity Survey, and data on Business Expenditure on Research Development; along with government administrative data including Australian Tax Office data on Business Activity Statements, Business Income Tax filings and Pay as You Go summaries. The BLADE is available for research purposes and has been used to assess agency performance, and of employment placement and wage subsidy programmes.

• There are also smaller, but important initiatives. An example is the Northern Territory Early Childhood Data Linkage Project. This links health, child protection, education and juvenile justice datasets for a 60,000 child cohort. The data from this project has allowed for the precise identification of the impact of Income Management in the Northern Territory¹¹ on key wellbeing outcomes such as low birthweight births and school attendance.¹²

• Additionally, a number of Australian Government agencies make extensive use of administrative data, including the linking of data sets to create new data assets. As well as line departments which use these data for internal purposes, a stronger public focus can be seen in the work of other agencies including: the Australian Institute of Health and Welfare, which reports on health and welfare services and outcomes; the Productivity Commission, which reports on a wide range of economic and social issues; and in statistical products of ABS, the national statistical agency.

While these and other activities have demonstrated the value of this data, a number of issues remain unresolved, or as challenges for further development. These encompass both ethical and organisational issues. We focus here on four: privacy and ethical use of this type of data; issues when programmes are implemented by third parties; legislative and other provisions which limit use of data collected for specific purposes; and data quality and utility.

The right of individuals to privacy and the ethical uses of the data are perhaps the most contentious. Concerns around these questions were raised extensively in the recent Australian development of digital health records–“My Health Record”. This sought to link and hold in an integrated, accessible, database health data on individuals from sources including hospitals, pharmacies, and medical practitioners. Its key objectives were to facilitate information exchange and ensure that professionals could access comprehensive health data, including in emergencies. While the potential value of these data for research and related evaluation purposes was recognised in the development of the system, implementation of this aspect was initially deferred.¹³

The proposed introduction of this record provoked considerable public debate, ultimately leading to an extensive revision of privacy and related provisions. This included: excluding access by insurers and employers; requiring judicial approval of any release of data to law enforcement or other government agencies; and increased penalties for data misuse. Even with these measures, and otherwise automatic enrolment, 9.9% of individuals actively opted out of the scheme (Australian Digital Health Agency, 2019), while an undisclosed number have imposed limits on what access is permitted to their information.

The debate on this, and more generally on the use of linked data across government, has become further complicated, and been made more sensitive, by another recent Australian experience. To identify possible debt from overpayment of income support the government used automated data-matching across social security and taxation records, and income averaging, to issue debt notices (a process which has been popularly termed “robodebt”). This process has been mired in controversy, culminating in a recent court decision that the process used by the Australian Government to calculate an alleged debt was unlawful,¹⁴ and a commitment by the government to refund $A721 million to 373,000 individuals (Robert, 2020).

This experience highlights not just some of the real difficulties in attempting to use this data, but also the consequences of poor application. While “robodebt” must be considered, at the minimum, as a crude and clumsy use of big data, from a programme management perspective it also illustrates the actual limitations of using joined up data which has been collected for quite different purposes, with different definitions and on different time scales. More significantly, it illustrates the corrosive effects of such uses, or misuses, on the relationship between those citizens affected and government, and on the reputation of the agencies involved. This is reflected in the results of public opinion polling which found that more than 60% of Australians were very concerned or concerned about data related to individuals being used by the Australian Government to make unfair decisions (Biddle et al., 2018).

The other three challenges concern more technical aspects of data collection and use. The first arises where programmes or policies are delivered by third parties. The delivery of a range of programmes in Australia is devolved to third parties, sometimes to other levels of government, and in other cases to community-based organisations, or the private sector, through outsourcing. Under these arrangements, frequently on the basis of minimising administrative overheads, or allowing flexibility, or in line with a focus on “results and not process”, there may be minimal data collected, or it may be collected inconsistently across providers. In such cases, many elements of effective programme evaluation, and the development of comprehensive and usable big data, can be thwarted.

The second constraint concerns legislative limitations. For example, in Australia the information that can be collected from persons seeking income support is largely prescribed in terms of that which is required for the purposes of determining a claim for a payment. Consequently some variables, such as whether a person is an Aboriginal Australian, or a Torres Strait Islander, which are very important from an evaluation perspective, and for the wider use of the data for social and related analysis, are only recorded voluntarily. Similarly other valuable socio-economic variables, including level of education, or health status, are only collected if directly needed for the purposes of determination of eligibility for a specific payment for which the person has applied. A related set of issues arise within data collection by the ABS where confidentiality is seen as being an important factor in seeking participation. Hence while Census data are used to build datasets such as MADIP, and a longitudinal Census file, this relies upon a mix of deterministic linkage which involves assigning record pairs across two datasets that match exactly or closely on common variables, and probabilistic linkage based on the level of overall agreement on a set of variables common to the two datasets.

Addressing these challenges is however not simple. Not only are there substantial issues garnering sufficient support within government for change but the changes themselves raise the set of privacy and ethical issues identified above, and in areas such as data collection by the ABS, can create new challenges in achieving compliance.

Thirdly even when quality data is available in a number of administrative datasets, bringing this together in a meaningful fashion poses challenges. This was clearly highlighted in the case of robodebt. Other concerns are the extent to which specific data items may reflect particular, and quite idiosyncratic, programme concepts and definitions, and that datasets are frequently poorly documented other than for processing purposes.

Making effective use of administrative data is however dependent upon more than addressing these particular problems, either in addressing the technical issues, or through appropriate legislation and privacy protections. It also requires change in the culture of the public sector, and a willingness to make more data publicly available. This last is an important challenge. The power of big data, and the willingness of the community to contribute to it, needs to be seen not just in terms of delivering improvements within government, but equally as providing the community more widely, including researchers and interest groups, with the tools to better monitor and evaluate the performance of governments. To enable this the data also needs to be available in easily accessible forms, including via comprehensive, regular and frequent publications.

Conclusion

New technologies and big data can significantly boost the capacity of policy to be evidence-based and programmes to be better managed and evaluated. They also can significantly enhance the scope for interactions with citizens, in particular with respect to the openness of government, through enabling the public to more easily access information on what government is doing, and to information on policy and programme evaluations. They provide an opportunity to put into the hands of the community the information which would enable them to make their own evaluations of the performance of government.

They do not however address the long-standing challenges with regard to the scope of evaluation and its role in accountability and the tension between a disciplined results-based approach to government decision-making and the inherently political nature of these decisions. Additionally consideration of their application brings into sharper relief some fundamental questions around the relationship between governments and the population, and the willingness of people to trust government with their personal information.

Finally, and perhaps most importantly, is the question of the implications of these opportunities for the structure and culture of public administration. Clearly “business as usual” is not an option.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. The Public Service Commissioner has overarching functions relating to the standards of integrity and conduct in the APS, and oversighting its capacity, accountability, effectiveness and performance.

2. This draws on Bray et al. (2019). This was one of six background papers commissioned by the recent Independent Review of the Australian Public Service chaired by David Thodey from the Australian and New Zealand School of Government. The papers comprised a literature review and key issues which needed to be addressed in the Inquiry’s task “to review the APS to ensure it is fit-for-purpose for the coming decades” (Thodey, 2019, p. 6).

3. A separate issue addressed by Bray et al. (2019), but not considered in this paper, is how the public service could best learn from its successes and failures. Amongst other issues, this pointed to a tension between the internal and reflective role of learning and the judgemental role, often accompanied by the allocation of responsibility and blame, which can be involved where evaluation has an accountability focus.

4. Derek Volker was a long serving senior public servant including being Secretary of the Departments of Veterans Affairs, Social Security, and Employment, Education and Training.

5. Peter Shergold is a former Secretary of the Department of Prime Minister and Cabinet.

6. This was an initiative to provide a fiscal stimulus following the Global Financial Crisis and improve the energy efficiency of Australia’s housing stock. The programme was very rapidly implemented and a number of issues were identified in its implementation including whether it led to unsafe work practices which resulted in injury and death.

7. Gruen proposes that this position be established on a similar basis to the Australian Auditor-General, as an independent officer of the Parliament who reports to the Parliament. While the appointment of the Auditor-General (for a 10 year term) is based on a recommendation by the Prime Minister, the nomination requires the approval of a Parliamentary committee. The independence of the Auditor-General is enshrined in the legislation which provides, subject to Commonwealth laws “the Auditor-General has complete discretion in the performance or exercise of his or her functions or power” Auditor-General Act 1997 (Cth) s8(4).

8. The “SMART” criteria were developed by Doran (1981).

9. In Australia, support for the unemployed and those unable to work, along with the aged, is through budget funded flat rate, non-time limited, means tested income support payments, rather than through social insurance.

10. This contrasts with an earlier joint inter-agency submission to a 2012 Senate Inquiry which considered similar issues. This previous submission devoted 20 pages to providing data and analysis on the standard of living of recipients of income support payments, including the incidence of financial disadvantage and stress. It further emphasised these other outcomes by outlining the core values and principles of the social security system including community acceptance; adequacy; equity and fairness; maintaining incentives and encouraging self-provision; consistency and financial sustainability (Australian Government, 2012).

11. This programme quarantined a proportion of individuals’ income support payments and restricted how this could be spent.

12. In both cases, the use of this data indicated that the programme had been unsuccessful in achieving its goals. See Cobb-Clark et al. (2017) and Doyle et al. (2017). Further details on the project are at Charles Darwin University (2019).

13. Additionally, people who have a My Health Record are able to restrict access to their data, even when confidentialised, for the purposes of research.

14. Deanna Amato v The Commonwealth of Australia, Federal Court of Australia – Vic (VID/2019/611-0).