Constraints on evidence-based policy: insights from government practices

Abstract

Insights are offered into UK government built environment policy-making processes through an insider's perspective (based on experience of being the chief executive of a public body, the Commission for Architecture and the Built Environment – CABE) on three empirical examples. The government's mandate was for policies to be evidence based. There was no shortage of demand for evidence, but it was fed into political and bureaucratic domains where less- or non-evidence-based influences were also at work. The questions considered are how much the evidence really influenced the content of policy; and whether making a policy ‘evidence based' led to its acceptance across government, causing departments to commit to its delivery. It is found that evidence (1) is powerful for defining issues to which policy should attend, (2) captures the attention of policy and decision-makers, but only if presented succinctly, and (3) is essential for testing outcomes. Supposedly evidence-based policy is not always truly evidence based. Many subjective forces counterbalance objectivity. The most significant reasons for this are mooted. Advice is offered on how to make evidence a more effective part of a process that will always be partly technical and objective, but also political and subjective.

Keywords:

• built environment
• evidence-based policy
• governance
• outcomes
• policy formation
• public policy

Introduction: design policy

This article is directed principally at the question of what lessons can inform better policy-making and help produce more successful initiatives. It also bears on the questions of whether the processes and assumptions underpinning the selection of options can be made more transparent. Consideration of public policy affecting design in the built environment leads directly into epistemologically challenging territory.¹

Whereas Schweber, Lees, and Torrito (2015) aim to focus on the ways in which policy problems are framed (as opposed to how policy is used), the purpose of the current article is to examine how British government ministers and officials used the advice of a public body that the government had created and funded. Its purpose was to gather and share the best available evidence on design quality, to enable government to make ‘better’ design decisions.

While acknowledging that design policy is controversial both philosophically and in the real world,² government behaviours have been shown to be similar even when less contentious, or more easily scientifically definable matters than design are involved. As Warwick (2015) observes, built environment policy-making can be ‘messy'. Expediency, emotion or ideology play a role as often as the systematic and rational application of evidence. Design policy is thus an appropriate area to probe for lessons on the application of evidence.³

This is a commentary, from an insider's point of view, based on lived experiences of policy-making. As Chief Executive of the Commission for Architecture and the Built Environment (CABE) from September 2004 to March 2011, the author was personally involved in a wide range of government built-environment policy initiatives, including those used as examples here. To offer a wider perspective on these examples, the author interviewed three former colleagues. The interviews were structured around questions about process and perceptions of interactions between key actors. Interviews occurred between May and September 2014.⁴ In this methodology both the author and the interviewees were participants in the action, offering advice and evidence for policy formulation or validation.

Although principally an empirical commentary on practice, the experiences described can be considered in light of academic theories of policy-making. Schweber et al. (2015), and to a lesser extent Warwick (2015), offer a useful review of the relevant literature, so it would be redundant to repeat it here. It will be apparent that the examples given do not fall into the ‘rationalist, linear depiction of policy-making as a technocratic exercise' which Schweber et al. describe as the ‘touchstone of most work on evidence and policy'. Whether the current author was experiencing evidence influencing policy decisions, or being ‘mapped onto the policy process' will be discussed. The current author also draws on two, more empirical, sources in the traditions of political economy and modern history which illuminate the context for the described experiences particularly clearly (Hildreth & Bailey, 2013; King & Crewe, 2013).

CABE antecedents and formation

From the beginning of statutory town planning in England there were attempts to ensure that development should ‘preserve to the utmost extent every object of natural beauty' and ‘produce a pleasant and harmonious result, locality preserved, designed and built in accordance with the best conceptions of architectural and artistic beauty' (Bentley & Pointon Taylor, 1911, p. 2). Over subsequent years the balance of policy (or at least its enforcement) fluctuated between the state controlling design and allowing developers the freedom to build as they wished. This fluctuation derived from a number of influences: most obviously the philosophy of the government in power and the level of expressed public concern about what was being built. On the whole, Conservative governments were probably more inclined to relax design control, though that was not an absolute given. In 1970, Ted Heath's Conservative government (1970–74) instructed that ‘freedom should be given to developers in matters of design' (MHLG, 1970). In 1990 a Conservative minister in Margaret Thatcher's government (1979–90), commenting on the government's policy to allow developers aesthetic freedom, observed in Parliament that:

Judgments about external design are essentially subjective. [ … ] I have seen no evidence that a more interventionist approach [ … ] would result in improved standards. [ … ] Indeed, there is a risk that attempts to compromise between differing aesthetic judgments may produce bland buildings which satisfy no one. (Chris Patten⁵, quoted in Hansard, 6 March 1990)

Running in parallel with these policy fluctuations, the Royal Fine Art Commission (RFAC) continued to pass judgment on the design quality of significant development projects. It had been established by a Conservative administration in 1924 and was chaired in the 1990s by Lord St John of Fawsley (Royal Fine Art Commission, 1992). By the time that Tony Blair's New Labour government was elected in 1997 (the New Labour government lasted from 1997 to 2010: Blair was Prime Minister from 1997 to 2007 and then succeeded by Gordon Brown, 2007–10), the RFAC was seen by many as elitist, subject to cronyism and ripe for reform (Fisher, 1998).

As Fisher (1998) says, the RFAC was in the Labour Party's sights ahead of the 1997 General Election. Even before then, though, under Conservative Environment Secretary of State John Gummer, the balance had been shifting back towards greater regulation of design. This was a response to what were seen as the poor results of the relaxed controls of the 1970s, 1980s and early 1990s (Carmona, 2011).

New Labour's concern with design quality had to do not only with the performance of the town planning system but also with that government's intention to embark on a substantial programme of public building. It wanted the new civic buildings to be ‘the pride of our towns and cities' (Falconer, 2002).⁶ Like all new governments, this one undertook a fundamental review of public expenditure and the RFAC came under scrutiny. Simultaneously, in 1998, Deputy Prime Minister John Prescott commissioned the architect Richard Rogers to chair an Urban Task Force to identify the causes of urban decline, recommend solutions and establish a vision for urban regeneration in England. Its report emphasized the need to improve design quality. It recommended the creation of a new advisory public body to pursue this goal, with a wider remit than the RFAC (Urban Task Force, 1999).

As a result, in 1999 the RFAC was reformed to become CABE, the government's adviser on architecture, urban design and public space in England.⁷ One way that CABE differed from the RFAC was in placing a greater emphasis on conducting research to inform evidence-based advice about built environment design (Carmona, 2011). CABE had its own small team of researchers who did some in-house research, but their main task was to commission studies and recommendations from academia and the private sector. A variety of methods were used. Readers are referred to the extensive list of publications in the references for more information about how CABE and its agents went about investigating the complex areas of design quality and value.

CABE and the government's demand for evidence

A formal obligation was placed on public servants in the UK by the New Labour government to practise evidence-based policy-making (Schweber et al., 2015). This was established as a principle in the White Paper⁸ Modernising Government (HM Government, 1999) and given form by the Cabinet Office⁹ (Bullock, Mountford, & Stanley, 2001). Due to its research remit, from CABE's inception it was repeatedly asked by government¹⁰ to produce evidence on the value of good design.¹¹ This was reflected in its research priorities (e.g. CABE, 2002, pp. 7, 10, 14; 2003a, pp. 3–4, 7–8; 2003b, p. 9; 2004, pp. 11–12; 2007a, pp. 9–10). More CABE publications presented evidence and commentary on value than any other subject (CABE & DETR, 2001; Carmona et al., 2001; FPD Savills Research & Davis, Langdon, and Everest, 2003; Macmillan, 2004; The Bartlett School of Planning, 2002; Wooley et al., 2004).¹² Over time, CABE amassed a considerable array of evidence about different dimensions of the subject, yet governmental demand did not abate. In 2007 the author described the problem:

health minister Andy Burnham made an appeal to CABE [in May 2006]. He needed more evidence to justify to the accountants the value of investing in good design. In spite of the mandatory common minimum standards [OGC, 2006] demanding good design and whole-life valuation [e.g. HM Treasury, 2006, pp. 6–8, 30–1; OGC, 2007b], the case still has to be made to prevent short-term cost savings and bad design choices. [ … ] He knew about the evidence that good design creates value. Yet [ … ] it wasn't enough on its own to convince the sceptics who see good design as just an expensive add-on [ … ].

(CABE, 2007a, pp. 6–7)

The author could have added at the time: this problem exists in spite of the Department of Health (DoH)¹³ running its own highly regarded design review panel, with transparent and thorough criteria for evaluating quality (Department of Health, 2007), CABE's guide for designing neighbourhood healthcare buildings (Mason, 2006), and other efforts to support good design.

It was not that CABE's evidence was unconvincing. It was produced by rigorous academic researchers and experienced industry experts. It had been highly influential on the Office of Government Commerce's (OGC)¹⁴ Achieving Excellence in Construction programme (OGC, 2002), which said that public sector clients

need to consider the cost and impact of design over the whole life of their projects. They should also understand the importance of their own role in ensuring that good design is achieved and best whole-life value for money is delivered.

(OGC, 2007a, p. 4)

It was also integral to the National Audit Office's (NAO)¹⁵ guidelines on construction projects (Davis, Langdon, & Everest, 2005). The Cost of Bad Design (Simmons, 2006) was given an almost rapturous reception by the Treasury¹⁶ and welcomed by the media and CABE's other audiences as an innovative way to evidence value.¹⁷

Nor was CABE alone in these efforts. Constructing Excellence¹⁸ published Be Valuable (Saxon, 2005). The Royal Institute of British Architects (RIBA) has continued to present evidence (RIBA, 2011) and in December 2014 issued a new call for evidence about design quality and performance (RIBA, 2014). Others continue to try to convince decision-makers about the evidence for good design and better environmental quality (e.g. Penny, 2014). So why was Andy Burnham asking for yet more evidence? Why did this item remain high on the Department for Culture Media and Sport (DCMS)¹⁹ and Department for Communities and Local Governments' (DCLG)²⁰ agendas until they decided to end CABE's statutory existence? Was collecting evidence useful in forming policy, or was it just a displacement activity, or a smokescreen used to justify political decisions based on more subjective foundations?

Design policy and the National Health Service (NHS)

An example from CABE's relationship with Burnham's DoH and National Health Service (NHS)²¹ illustrates why CABE was so interested in the limits on the utility of evidence in forging and applying policy. In 2000 the government introduced Local Improvement Finance Trusts (LIFTs). These were public–private partnerships (PPPs)²² with NHS Primary Care Trusts (PCTs).²³ They procured private finance and expertise to build and maintain primary care centres (PCCs).²⁴

By the mid-2000s the design quality of LIFT PCCs had begun to cause concern (Building Design, 2005). In 2007 CABE audited the design of LIFT buildings (CABE, 2008a). The audit showed mixed results. Many PCCs were well designed in the terms of policy expectations. Other buildings were good but only in parts. Enough were judged poor to demonstrate that the programme was not achieving consistent value for money or quality of outcomes.²⁵ The results were shared with Ben Bradshaw, Minister of State for Health,²⁶ in October 2007. He was shown photographs of comparable schemes, including two sinks in brand new PCC consulting rooms.²⁷ One was obviously hygienic and fit for purpose. The other was visibly a trap for pathogens and clearly unfit. Yet both had been funded using the same budget parameters within the same policy programme. The minister felt that this was highly unsatisfactory. He asked his officials and CABE to look again at how to secure consistently high design quality.²⁸

To investigate further, CABE's Chief Executive and a DoH board member visited another new LIFT PCC in Kent. An example of one design flaw was that the phlebotomists' room had no window. This was a problem when nervous patients fainted and needed fresh air. The clinicians had not been consulted about the design. The PCT's client explained that her managers had assumed that the private sector partner had all the expertise to design a PCC. Consequently she had not been allocated enough time to act as fulltime client, so had not been able to pay sufficient attention to the layout. Nor was time found for staff and user consultation. CABE's Chief Executive and the DoH official agreed that steps should be taken to reduce the risk of this problem recurring.²⁹

Evidence was used to define problems not mainly of arguable aesthetics, but of unarguable functional design, and trigger ministerial commitment to action. There is evidence that action did follow. In 2014, design appeared to remain a sensitive issue for Community Health Partnerships (CHP),³⁰ which has oversight of LIFT. It headlined the following quotation on its webpages, distancing the programme from earlier criticism:

We were given every opportunity to be part of the design process and create the environment we now work in. With the project team we have achieved exactly what we want and need, enabling us to enhance our service delivery in a clinical environment that meets our patients [sic] expectations. All members of staff feel that the building promotes and inspires a professional approach to healthcare.

(Alan Buckley, Practice Manager, Shelton Primary Care Centre)³¹

This seems to be an indicator that awareness and application of good practice became, and remained, embedded in LIFT.

There were, however, limits to the power of evidence. CABE and DoH officials continued to meet bilaterally and as part of interdepartmental design initiatives until CABE's future came under review in 2010. They considered how better to raise the design quality across DoH capital programmes. Meetings soon after the LIFT report was published explored why the impact of design policy was sometimes inconsistent in spite of strong ministerial commitment and the evidence for the value of good design.³² The difficulties discussed included:

• Organizational complexity and accountability

  The devolution from the DoH (central government) to the NHS nationally, and then further to local trusts within the NHS, created complex administrative chains and uncertainty about accountability. Command and control were difficult, even when intended. Some bodies (e.g. Foundation Trusts, introduced in 2004) had been deliberately freed from central direction and scrutiny in order to act autonomously. At the same time, the private sector had a strong voice in PPP projects. All this made it hard to enforce centrally defined design standards and get messages about best practice to the right people.

• Client skills

  There was a shortage of clienting skills, especially in project management, design and construction. Clients (e.g. the mentioned PCT manager in Kent) were trained to do other sorts of job, then asked to lead construction projects.

• Alternative priorities

  There was strong contention for resources. As clinicians dominate the NHS, they demanded investment in clinical equipment, medications and staff. The medical worldview was focused on scientific interventions by clinicians. Investment in better buildings to improve patient and staff experiences came lower down the priority list.

• Dismissive attitudes

  Most significantly, the evidence for the value of good design in improving clinical outcomes was either not sufficiently believed or not given weight because it did not fit with this medical worldview. Evidence gathered by Roger Ulrich (Ulrich, 1984, 2001) and others highlighted that patient stress can be reduced, and recovery times accelerated, by well-designed surroundings and views of green space. However, when presented with this evidence, clients declined to give it credence because it was conducted in the United States. Exactly what differences between British and American patients validated this scepticism were not elucidated.

Limits of evidence in context

The study of policy formation and its consequences, however, suggests that the above example was not isolated. In the 1980s Hall (1981) pointed to the difficulties of policy creation and collective decision-making in the built environment, suggesting that public choice theory might explain them. Yet the theory's assumptions, derived from neo-classical economics, do not altogether fit the reality of UK public administration. Notably, to assume that all decision-makers are self-interested, rational and pursuing maximizing strategies (Ostrom & Ostrom, 1971) does not explain the altruism, ideological idées fixes or willingness to settle for satisficing compromises common in the British polity.

Hildreth and Bailey (2013) suggest the analogy of baking a pizza to describe policy-making. The topping is political rhetoric and assertion (‘PCCs will be well designed'). The cheese is the policy itself; but the fundamental of the pizza is its base – its ideological and cultural underpinnings. Bureaucratic cultures, political constraints, risk aversion and ingrained Treasury ideologies (the usual base ingredients) ensure that many new policies struggle to overcome inertia and be delivered, whatever the rhetoric and rationality behind their prescriptions.

This analogy resonates with CABE's experience with the DoH but does not fully explain it. Design policy was based on evidence and was successful to some extent. The ‘cheese' did soften the ‘base' in places. To grasp the limits of the power of evidence it is necessary to understand more precisely how the subjective ingredients of policy-making change the flavour of the policies which are then presented to the public. King and Crewe (2013), investigating policy ‘blunders', identified a specific set of obstacles that policy-makers must overcome. Not all are susceptible to evidence. Those most relevant to the use of evidence-based policy are:

• Cultural disconnect

  Projecting the policy-makers' lifestyles, preferences and attitudes onto the subject. An instinctive human response to making difficult subjects comprehensible but not the ideal platform for applying objective evidence.

• Group-think

  Faced with the need to maintain group (e.g. political party, civil service department, professional) cohesion, minority or dissenting voices are suppressed, even when they may have a better analysis than the majority.

• Prejudice and pragmatism

  An inbuilt, unquestioning belief that some kinds of policies and institutions will work better than others. Pre-judgements and unquestioned assumptions are hard to sway with evidence.

• Panic, symbols and spin

  Hasty, often knee-jerk reactions to events, frequently responding to media pressure. Accompanied by political rhetoric, the supporting solutions are not adequately tested against evidence.

• The centre cannot hold

  The authors describe the Prime Minister's limited power to exert control over a complex system of competing departmental fiefdoms. The same was true within the DoH and NHS in the illustration above.

• Musical chairs

  Ministers seldom last long enough in their role to implement policy successfully before another minister comes into post with a different agenda.

• Ministers as activists

  The relationship between civil servants and ministers changed under the Thatcher government, 1979–90. Ministers now make their reputations by passionate commitment to delivering a policy agenda that they have adopted or seek to champion. Civil servants who try to moderate policy with evidence of risk may be deemed obstructive, and therefore are sidelined. In parallel, the role of political special policy advisers (SPADs)³³ has grown (and continues to grow) at the expense of apolitical civil servants.

• Asymmetries of knowledge and expertise

  Government performance in procurement and project management has been hampered by a lack of the right skills and knowledge.

• A deficit of deliberation

  Policy is no longer subjected to sufficient deliberation before being decided. It is often created too quickly to give full consideration to diligent assessment and risk mitigation.

A disconnect between those who write policy and those who have to deliver it, a lack of accountability amongst policy-makers, and insufficient power in Parliament to influence and scrutinize government policy are the other obstacles which King and Crewe observed. For completeness one should add corruption and doing favours for political and/or business allies as hidden subjective obstacles,³⁴ and experimenter bias in the design and conduct of research (though much of the above is what leads to bias). King and Crewe, then, satisfy our appetite for a detailed recipe for the subjective constituents of the policy pizza. They also help to inform the understanding of the following short case study.

World Class Places policy initiative

In August/September 2008, CABE was asked by Ed Miliband's speechwriter (at that time, Miliband was Minister for the Cabinet Office) to compile a list of speakers and topics for an October 2008 seminar at No. 10 Downing Street, the Prime Minister's Office. Its purpose was to explore radical ideas to improve public buildings and spaces. It was one of a series to help the Cabinet Office Strategy Unit (COSU)³⁵ look for ‘what could be on the “agenda after next”'.³⁶ Speakers included Lord Richard Rogers (architect and formerly chair of the government's Urban Task Force), Ricky Burdett (London School of Economics' Cities Programme), leading urbanist Jan Gehl and myself.

In a reshuffle, Miliband left the Cabinet Office on 3 October 2008 to become Secretary of State at the Department for Energy and Climate Change (DECC).³⁷ His successor, Liam Byrne, inherited the responsibility for the seminar, having played no part in its gestation. The original impetus for examining this topic seems to have come from Gordon Brown saying, at one of his first Cabinet meetings as Prime Minister, that the quality of new housing was not good enough and something must be done about it.³⁸ On this or another occasion he expressed the same opinion about school design.³⁹ The COSU seminar was, therefore, to focus on the physical environment. It went ahead in October 2008 as planned, the speakers having been booked before Byrne had had the opportunity to review the project's objectives.

In spite of these origins, Byrne had other priorities once the seminar had taken place. Like DCLG Secretary of State Hazel Blears,⁴⁰ he was more interested in people's behaviour than the physical environment.⁴¹ The project was redefined to develop a policy called Total Place. It focused on how local communities could be enabled to tackle questions of crime, health and social care, alcohol and drugs, children's life chances, and high-cost communities.⁴²

This left the development of new policies for quality of place in limbo. Internal lobbying, mainly by one of Brown's SPADs who was personally interested in design, eventually led the Prime Minister's Office, in December 2008, to back a freestanding, whole-of-government strategy.⁴³ The timing was bad. Unfortunately, this was when the banking crisis was deepening. COSU staff who had worked on quality of place were redirected to look for solutions to the banking crisis, leaving one senior policy official and an assistant to research and write the strategy for quality of place. CABE seconded its Head of Policy to assist in December 2008.⁴⁴ By this time only CABE, DCMS and DCLG were active stakeholders. The government became much more departmentally fragmented (2008–10) than it had been under Blair's leadership. There was no real buy-in elsewhere, not even within the Cabinet Office and the Prime Minister's Office.⁴⁵

COSU followed the rigorous research procedures in its Strategy Survival Guide.⁴⁶ This required evidence to be presented for each claim, to explain why it mattered and what interventions might work. This analysis of issues and opportunities was distilled into a PowerPoint presentation, a very powerful way to focus information to be grasped quickly by busy officials and ministers.⁴⁷

The research took seven months. It looked at why place is important to outcomes for people and the economy (itself a challenging question when the Treasury had tended to see things aspatially, at least until Alistair Darling became Chancellor of the Exchequer⁴⁸). Research evidence such as CABE's housing audits⁴⁹ was complemented by case study visits, expert seminars, and other inputs from leading academics and professionals. The findings were presented internally in an interim analytical report. After review and further refinement, a final analytical report was prepared to inform policy writing. It was published in June 2009 (COSU, 2009).

As policy options began to emerge in early 2009 a problem arose because a collateral effect of the financial crisis was a collapse in new house building (Rhodes, 2014). A revival in the housing market was promoted as an important way to drive economic recovery (Timms, 2010). Anything that put quality pressures on the planning system now became unwelcome to the government's leadership. The Prime Minister's Office and the Treasury were demanding that obstacles to house building should be swept aside. Perceived obstacles included the possibility of engendering delays to the grant of planning permission and/or increasing building costs as a consequence of enforcing policies to improve housing design (Gardiner, 2010a; Hurst & Hopkirk, 2010).⁵⁰

It soon became apparent that there was little interest in World Class Places across the rest of government because many of the options had costs attached and new calls on public expenditure had to be excluded due to a need for austerity resulting from policies to tackle the financial crisis. The World Class Places strategy had, however, been conceived to bind all departments. The ‘HM Government' tag was intended to give the policies reach and clout. Intensive negotiations were held with eight government departments. In most cases the outcome was that COSU was offered support only for what ministers had already planned as part of their own departmental policy initiatives. There was little support for COSU's own, new, centrally generated evidence-based proposals.⁵¹

COSU asked the Prime Minister's Office to direct departments to adopt more new ideas. It was told that the built environment was ‘not a vote winning issue and does not matter to the C1 [lower middle class] demographic' (a key election target), so there was no real prospect of active support from the top leadership for a raft of new policies. Because the Cabinet and other minsters had become deeply divided over the effectiveness of the Prime Minister's leadership (e.g. Stratton & Wintour, 2009), COSU was also told that he only had sufficient political capital to deliver one significant cross-governmental policy change for the built environment.⁵² COSU and the stakeholders were asked to decide which proposition was most important.

In parallel with the quality of place project, CABE had been working with the Prime Minister's Office and others⁵³ on a design threshold for the Department for Education's (DfE)⁵⁴ Building Schools for the Future programme.⁵⁵ This minimum design standard (MDS) was announced at short notice (by Schools Minister Jim Knight in September 2008) in response to media criticism (Booth & Curtis, 2008). It was founded on evidence from CABE that designs could be improved by structured reviews, and the principle that public money should not pay for poor design.⁵⁶ In 2008, it was still in the early stages of development and was not introduced until May 2009 (CABE, 2010, p. 4). The quality of place project proposed adopting this idea across all government procurement. This was the one cross-government policy that the Prime Minister's Office obliged all departments to adopt. When the strategy was published as World Class Places in May 2009 (HM Government, 2009b) the design threshold featured but there was little else that was new.

World Class Places had minimal obvious lasting effect on the built environment. DCLG, DCMS, COSU and CABE wrote and monitored an Action Plan (HM Government, 2009a). Blears resigned from the Cabinet during a reshuffle in June 2009. Her replacement at DCLG, John Denham, showed little interest in design.⁵⁷ It was difficult to trace the impact of World Class Places on DCLG's activities and its design policy team was seen internally as increasingly marginalized. Notably, the new strategy was not even mentioned in the department's annual review.⁵⁸ Burnham, Secretary of State at DCMS during the writing of World Class Places, moved to be Secretary of State for the DoH in the same reshuffle. His successor at DCMS, Bradshaw, had little time to stamp his mark on delivery, except through the London 2012 Olympics. The Olympics had strong policy backing of, and active ministerial demand for, good design.⁵⁹

The election of the new Coalition government⁶⁰ in May 2010 led to almost complete abandonment of World Class Places policies before implementation could occur. The government set out to reform the planning system, regeneration, housing and public procurement on different lines. COSU was abolished. Two of COSU's staff went to work at DCLG, where briefings for the new ministerial team used parts of the analysis from World Class Places.⁶¹ Some of the Coalition's rhetoric echoed aspects of World Class Places (though the Conservative Party had said similar things before the election; Conservative Party, 2010). CABE advised DCLG on the design elements of the National Planning Policy Framework (DCLG, 2012, ch. 7) and good design continued to be a core objective for London 2012 Olympic developments, so all may not have been lost. Secretary of State for Education, Michael Gove, however, moved rapidly in 2010 to end the Building Schools for the Future programme, the MDS and CABE's involvement in school design (Building Design, 5 July 2010). No other department adopted an MDS, so penetration of World Class Places-type thinking into Coalition policy remained narrow.

The intention here is not to illuminate the product of policy-making so much as the manufacturing process. The lesson from examining the fate of World Class Places is that, even where reasonably robust methods of assembling and assessing evidence are applied, the factors identified by Hildreth and Bailey; King and Crewe; and discussed above are always at work to modify policy content and intent. The case study of World Class Places illustrates:

• The challenge of overcoming the crusty base of government inertia, as individual departments resisted new evidence-based policies and expenditure on subjects which they did not see as germane to their interests.

• Ministers on a mission of their own were able to divert the course of collective policy-making.

• The centre (the Prime Minister's Office) was, in King and Crewe's phrase, ‘unable to hold', i.e. to enforce compliance with its leadership in the face of strong ministers and senior civil servants seeking to maintain and extend their power and prospects through their own departments' policy agendas.

• The eroding effect of ministerial ‘musical chairs'.

• As the MDS was still being developed and tested when it was included in World Class Places, it could be argued that there was a deficit of deliberation in making this the stand-out policy innovation.

• Reactions to short-term events threw off course the development of policies intended to have long-term impacts.

Conclusions

Against the models of policy-making sketched by Schweber et al. (2015), one may, superficially, draw different conclusions about the DoH and COSU examples. At the DoH the production of evidence about design clearly was overlaid and mapped onto other policies. By contrast, within COSU evidence did influence and, indeed, drive the framing of policies in World Class Places. In both cases, though, one can see indications of both Advocacy Coalition Models (where ‘political actors selectively engage in evidence to support their core beliefs') and a processual model which describes ‘ongoing negotiations across a range of policy actors'. In both examples there are also teleological issues that clearly inhibited the application of design policy. In the NHS, the dominant medical paradigm meant that the ends sought by medical practitioners (cure by scientific means) appeared to be at odds with espousing the idea that the patient's environment could impact significantly on recovery. With World Class Places, the goals of ministers and civil servants pursuing departmental interests undermined those advanced by the centre of government. At the same time, the overlay of other policy imperatives (the need to tackle the financial crisis and the means chosen to do it) had an impact on key policy-makers' perceptions of the advantages of promoting design policy. As such, both examples presented here probably lend weight to Schweber et al.'s argument that there are ‘policy frames' which ‘limit the range of options which different actors perceive and entertain'.

One conclusion is that it is very difficult to make the processes and assumptions underpinning the selection of policy options in central government more transparent. Once evidence is injected into the political realm it becomes subject to issues (such as those illustrated here and by King and Crewe), which can appear arbitrary (e.g. a new minister who wants to pursue a different agenda) or are very hard to make explicit (dimensions of organizational culture), or public (e.g. the unintended consequences of otherwise intentional changes of accountability in the NHS).

What lessons emerge, then, to inform better policy-making? Evidence is powerful when defining issues to which public policy should attend, so it should remain a firm pillar of the system. Evidence captures the attention of policy- and decision-makers if it is presented simply, clearly and proves the relevance of the subject to their audiences or interests. Two photographs can be enough, as CABE showed in the example discussed above of the PCC consulting room sinks shown to Bradshaw. As a means to test policy outcomes at the delivery stage, evidence is essential, and depends upon comparable data collected before the policy is implemented. It is particularly helpful in highlighting the differences between policy expectations and results (the LIFT example shows this) but also in enabling different policies to be compared.

It is highly improbable that policy in the British political system could ever be made to be fully evidence based. With the change of government in May 2010, Prime Minister Cameron reiterated the call for evidence-based decision-making in the public sector (Schweber et al., 2015), yet a former SPAD observed in March 2014 that:

Environmental issues were viewed ‘sceptically' in Whitehall [i.e. by government] and the [construction] industry need to ‘tread carefully' [ … ] on initiatives to increase energy efficiency to existing buildings [ …  The construction industry] are treading on eggshells politically.⁶²

If policy-making is not an objective, scientific activity – and there can be no surprise at this finding amongst those who give the subject deep consideration – the examples given here show that there can be science to it; but that there are limits.

What needs to be more widely understood by those directly and indirectly involved with policy formation is that there is also an art to constructing and delivering policy successfully. The art consists in knowing when and how best to deploy evidence and in understanding its limits as a means to steer policy formation. It also requires accommodating the political and subjective modifications which are the frequent and often inevitable consequence of creating policy in a democracy. Frustrating though it can be, especially for professionals in the built environment sector whose milieu is often evidence-based decision-making, it is necessary to be both scientific and artful to make an impact in the policy world. This may seem a pessimistic analysis but it is not out of step with, for example, the Institute for Government's critique of policy-making (Hallsworth, Parker, & Rutter, 2011; Hallsworth & Rutter, 2011) which issued recommendations for the civil service and Parliament.

How can those outside government who wish to influence policy do better? In the built environment sector, a number of key points diminish the force of evidence. There is a perceived lack of interest amongst voters compared with other emotive issues (the NHS, education, immigration etc.) and a widespread belief within government that the built environment is mainly a local issue.⁶³ With strong vested interests (NIMBYism⁶⁴ as well as business), there has been the fragmentation of the industry's attempts to influence policy and at the same time the portfolio is divided amongst government departments (Farrell & the Farrell Review Team., 2014). In the case of design policy, there are deeper epistemological questions at play.⁶⁵ Politics is organized around short-term election cycles. In contrast, the timescale for changing the built environment can take decades. The typical educational background of the political and administrative elite does not provide a good acquaintance with or understanding of design, or spatial matters pertaining to the built environment.⁶⁶ Appreciation of these factors can help to inform ideas about how to address the problem.

When the author was Chief Executive of CABE he encountered all too many built environment professionals who found it difficult to understand why what appeared to be obvious from the evidence was, nonetheless, never translated into policy; or if it did make it into a policy document, did not generate the action within government which the evidence demanded. The following lessons are aimed mainly at such practitioners. They derive from both examples presented in this article and the author's wider experiential learning:

• For politicians, evidence is clearly about more than quantitative data and the logical flow of analytical reasoning. They are attuned to other influences as well, particularly what they perceive to be the opinions of their electors and other influencers such as business and the media. It follows that what people think and want is also a form of evidence (Schweber et al., 2015). Policy-makers and their advisers do not rely on technical arguments alone. They should find out what matters to citizens and communicate it.

• Understanding who the stakeholders and influencers are in any given policy formation process and addressing their positions is critical. Analysing the stakeholders' interests, ascertaining where vested interests lie and how either to gain their agreement or produce evidence strong enough to overcome their influence over the decision-makers, is a valid part of the process of deploying evidence.

• CABE's experience shows that it is important to know which bits of government matter to your subject, and to which your subject matters. Identifying the important decision-makers and communicating directly with them can enable one to understand their appetite for evidence and how best to supply it to them. With the NHS, for example, it was often more effective for CABE to work with its design review panel than with officials on the policy side.

• Evidence can be used to identify issues that government needs to address, and to help to devise ways to resolve those issues. The expectation, particularly amongst politicians, is that evidence-based policies will offer solutions rather than engender criticism or problems.⁶⁷ When devising research and modelling evidence-based solutions it is important to give equal weight to both dimensions.

• A key limit of evidence is reached when it points to solutions that go outside decision-makers' comfort zones. In the case of World Class Places, the reach of the Prime Minister's Office was limited by a weakening government facing a financial crisis. The appetite for evidence-led policy change in the built environment sphere was almost non-existent by the time the research phase was completed. What flows from this is, firstly, the necessity to be alive to the context in which evidence may need to be presented and, secondly, that more effort and persuasion may be needed to convince decision-makers to commit.

• The question of timing and timescales is critical to all policy-making. The deferral of World Class Places led to it running out of time. Evidence needs to be assembled as rapidly as possible if it is to be useful in the increasingly frenetic government policy world. A corollary of this is that evidence should be sought that will allow policy-makers to test whether quick wins are practicable. Even if a policy will take a long time to reach full impact, quick results will appeal to those working within election cycles.

• Many professionals and academics struggle to deliver evidence in ways which are digestible to busy decision-makers. It is vital to present data and solutions with brevity and clarity. For people whose culture and education do not prepare them to excel at built environment policy, keeping it simple but smart is ideal. The brilliance of COSU's use of PowerPoint presentations was that it was ‘graphics heavy, text light', easily understood and quickly digested. In this author's experience long, technical, wordy reports will not be read by most top decision-makers.

These techniques may help to get evidence noticed and believed. It remains a truism, though, that politics is the art of the possible. Democratic government is never going to be a purely technocratic exercise. Evidence can be a forceful influence on policy, but the territory in which it is applied is not neutral. Understanding this is vital for those who wish to change outcomes.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

1 The purpose is not to debate in detail principles such as whether there should, in fact, be public design policy; the extent to which the state (national or local) should regulate designs; how to define ‘good design'; and what the parameters for design policy judgements and their applications should be.

2 Contention about design policy derives from fundamental and reasonable questions from aesthetic and political philosophy. Is designing building and places a purely private aesthetic exercise or has it other, more public, dimensions? Should designers' artistic expression be questioned? Who should make design judgments if design is to be regulated? Is the mediation by the state and the professions of what is and is not acceptable in built environment design justified, and how does this relate to democratic governance? Is beauty only in the eye of the beholder, or can objective, or structured and reasonably consensual subjective judgments be made about it? Further, and crucially for this article, what evidence, if any, about the nature and impact of design quality can be agreed upon, assembled and applied? These questions are legitimate and interesting, but they have been considered in depth elsewhere (e.g. Carmona, 2009; Carmona & de Magalhães, 2009; Cuthbert, 2006, Ch. 10; De Botton, 2006; Lubbock, 1995, ch. 4; Punter, 1985, 1990, pp. 7–8 and 359–365; Royal Fine Art Commission, 1992; Rykwert, 2000). They are not the focus of this article.

3 While it would be interesting to explore further the nature of evidence as it relates to design, that is not possible here due to constraints of space.

4 The interviewees were Ben Rogers, project leader for World Class Places in the Cabinet Office Strategy Unit (COSU); Adrian Harvey, Head of Policy for CABE and seconded to COSU to work on World Class Places; and Paul Hildreth, who worked in policy advice and research roles in the Office of the Deputy Prime Minister (ODPM)/Department of Communities and Local Government (DCLG) from 2003 to 2009 and who stood in for Harvey at CABE during the latter's secondment.

5 The minister in question was Secretary of State for the Environment, though stating the value he placed on good design, was nevertheless reinforcing the government's commitment to curbing aesthetic control by local authorities through the town planning system, a policy set out in Circular 22/80, Circular 31/85 and the then current version of Planning Policy Guidance Note No. 1: General Policy and Principles.

6 Charles, Baron Falconer of Thoroton was Minister of State for Housing and Planning from June 2001 to May 2002 and, in that and various other ministerial roles in the Blair New Labour government, was (in the author's personal experience) an advocate for good design and for CABE.

7 CABE was a non-departmental public body, established on 1 September 1999, which ceased operating as a public body on 31 March 2011 as a result of the David Cameron's Conservative/Liberal Democrat coalition government's ‘Bonfire of the QUANGOs'. CABE was initially sponsored by the Department for Culture, Media and Sport (DCMS), but also received substantial core funding from the ODPM (renamed DCLG in 2006). CABE was made into a statutory body in April 2005. Its functions were ‘the promotion of education and high standards in, and understanding of, architecture, and the design, management and maintenance of the built environment' in England (Clean Neighbourhoods and Environment Act 2005, 88 (1)).

8 A White Paper is a statement of UK government policy.

9 The Cabinet Office is the government department that supports the Prime Minister and Cabinet. It is composed of various units that administer Cabinet committees and coordinate the delivery of government objectives through other departments. Staff working in the Prime Minister's Office are part of the Cabinet Office. The Cabinet itself is the ultimate decision-making body of the executive of the UK government. It is led by the Prime Minister and comprises the most senior departmental ministers.

10 All references to ‘government' refer to the British government. Because of devolution to Scotland, Wales and Northern Ireland, however, the article is, de facto, about policy-making in England.

11 CABE defined good design, using the Vitruvian principles of firmness, commodity and delight, as being fit for purpose, sustainable, efficient, coherent, flexible, responsive to context, good looking and a clear expression of the brief (CABE, 2006). Subsequently, the definition was extended to include inclusiveness and accessibility.

12 Macmillan (2004) presented work being done by the Construction Industry Council and CABE and included articles by a number of CABE's Commissioners and its then Chief Executive, Jon Rouse. Other examples of CABE publications on the value of design are PricewaterhouseCoopers in association with Queen Margaret University College Edinburgh (2004), MORI (2005), CABE (2005a, 2005b, 2005c, 2007b, 2007b, 2009), Mulgan et al. (2005), Simmons (2006), Macmillan and Eclipse Research Consultants (2006), and The Young Foundation et al. (2006).

13 The DoH is the ministry (governmental department) responsible for government policy on health and social care in England.

14 The OGC was, when it published Achieving Excellence in Construction, a branch of the UK Treasury responsible for improving value for money in public procurement and the use of government property. At the time of writing, it has been transferred to be part of the Cabinet Office's Efficiency and Reform Group.

15 The NAO is an independent body, reporting to Parliament, that audits central government departments, agencies and non-departmental public bodies, and assesses the value for money of the administration of public policy.

16 Her Majesty's Treasury (HM Treasury), informally known as the Treasury, is the department responsible for developing and executing the UK government's public finance and economic policies. Its most senior minister is Chancellor of the Exchequer.

17 Author's personal experience. The publication aroused considerable media interest, with items on, for example, BBC television and radio news programmes, and in The Times, Daily Mail, The Guardian, Evening Standard, Sunday Telegraph and local newspapers on 18–20 June 2006 and subsequently.

18 Constructing Excellence is a cross-sector, cross-supply chain membership organization which states that it exists to improve construction industry performance in order to produce a better built environment.

19 DCMS is the ministry with responsibility for policy on culture (including architecture), sport and aspects of broadcasting in England.

20 DCLG is the ministry with responsibility for policy on local government, town planning and building regulations, urban regeneration, and a variety of related functions in England.

21 The NHS is the publicly funded healthcare system for the UK. Primarily paid for through general taxation, it provides healthcare to every legal resident, mostly free at the point of use. This article deals with the NHS as it operates in England: much of its administration in the other nations of the UK has been devolved to their national assemblies.

22 A PPP is a government service or private business venture funded and operated through a partnership of government and one or more private sector companies.

23 PCTs were parts of the NHS in England from 2001 to 2013. PCTs were largely administrative bodies responsible for commissioning primary, community and secondary health services from providers. Until 31 May 2011 they also provided community health services directly.

24 See http://www.communityhealthpartnerships.co.uk/about-the-lift-programme/.

25 A Minister of State is a second-tier minister in a UK government department, the most senior minister usually being known as the Secretary of State, except where traditional titles such as Chancellor of the Exchequer apply.

26 Author's personal knowledge.

27 Photographs are also published in CABE (2008b, p. 16).

28 Author's personal experience and contemporary notes.

29 Author's personal experience.

30 A CHP was set up as a joint venture between the DoH and Partnerships UK (PUK), the latter being itself a joint venture between the UK Treasury and the private sector to promote PPPs. PUK ceased activity in 2011 but, as of December 2014, CHP remains head tenant for the LIFT estate and is responsible for the overall management of 300 LIFT buildings in England.

31 See http://www.communityhealthpartnerships.co.uk/about-the-lift-programme/.

32 Author's personal experience.

33 Special advisers are paid by central government and are styled as ‘temporary civil servants' appointed under Article 3 of the Civil Service Order in Council 1995. Unlike ‘permanent' civil servants, who must be politically impartial, special advisers are political appointees who owe their loyalty to the governing party and, often, particular ministers with whom they have a close relationship.

34 The extent of corruption in local government is difficult to discover (Barrington & Maxwell, 2013), but there have been well-documented alleged incidences in the town planning system (e.g. Report by the Local Ombudsman on an Investigation into Complaint 88/A/0006 against Rochester upon Medway City Council [re Alleged Maladministration in the Way a Planning Application was Dealt with]; Commission for Local Administration in England, 1990; The West Briton, 2009). Examples of influence in return for political party funding or through influential connections are probably more common, or at least more commonly alleged (e.g. Gillespie, 2011; Mathiason & Fitzgibbon, 2014).

35 COSU was made up of an elite staff working as part of a streamlined centre of government set up in the Cabinet Office to provide advice to the Prime Minister and Cabinet about the future direction of policy.

36 Author's personal experience and contemporary notes.

37 DECC is the ministry responsible for the government's policies on those topics in England.

38 Reported by an interviewee.

39 Reported to the author at the time by one of the Prime Minister's staff.

40 In Transforming Places, Changing Lives (2008, pp. 50–52), Blears promulgated a significant shift in government regeneration strategy away from physical regeneration towards economic and social measures and community and personal empowerment (DCLG, 2008, pp. 11, 23). This was partly a response to evidence collected in HM Treasury et al. (2007) that, while past funding for physical regeneration had led to positive urban change, the benefits to the most deprived people had been limited. The shift was, however, also very much in line with Blears' own political beliefs, e.g. ‘The public expects us to [ … ] target our efforts firmly on people as well as places' (speech to the British Urban Regeneration Association, quoted in Local Government Chronicle, 12 May 2009) (Blears, 2004, 2008). The shift was, to some extent, shifted back in the final form of the framework, published in 2009 following consultation and the impact of the Credit Crunch. The Crunch had been referred to in the 2008 document but, by 2009, government policy aimed to encourage a much larger investment in property projects to stimulate the housing and construction industries (DCLG, 2009).

41 Reported by an interviewee.

42 See http://webarchive.nationalarchives.gov.uk/20091005102954/http://localleadership.gov.uk/totalplace/themes/theme-groups/.

43 Reported by interviewees.

44 Author's contemporary note; reported by interviewees.

45 Reported by an interviewee.

46 See http://webarchive.nationalarchives.gov.uk/20100125070726/http://interactive.cabinetoffice.gov.uk/strategy/survivalguide/.

47 Author's experience of this and another COSU project.

48 Reported by an interviewee.

49 See http://webarchive.nationalarchives.gov.uk/20110118095356/http:/www.cabe.org.uk/housing/audit/.

50 During the drafting of the World Class Places, the author noted that it might not chime well with the Homes and Communities Agency (HCA) – the government agency responsible for targeting public investment in new housing and other regeneration projects in England. HCA was responsible for funding the Kickstart programme, which financed new housing starts. The contention between CABE, which reviewed Kickstart projects and found their design quality wanting in many cases, and the HCA was highlighted by the media later in 2009 and early 2010 (e.g. Gardiner, 2010b; and references in the main text above).

51 Reported by an interviewee.

52 Reported by interviewees.

53 Partnerships for Schools (PfS), the Department for Children, Families and Schools (DCFS) and RIBA.

54 DfE is the ministry responsible for policies affecting people in England up to the age of 19, including education and schools.

55 Building Schools for the Future was a Labour government programme of massive capital investment in building new and refurbished schools in England. It was intended to operate mainly through PPPs. It ran from 2005 until it was abandoned by the newly elected Coalition government in 2010.

56 The evidence on this subject became disputed. CABE had shown that, against a set of criteria agreed with PfS, external design experts and educationalists, the design of schools improved between first and final CABE design reviews (e.g. Hansard, 2009). As experience of the MDS grew, CABE and PfS simplified the scoring criteria. The James Review commissioned in July 2010 by Michael Gove, the Coalition's Secretary of State for Education, used figures from CABE and PfS to argue that the MDS had not worked (James, 2011, p. 25). Because James was comparing pre- and post-simplification scores (which were not strictly comparable), and arguing from the outset for doing away with individually designed schools in favour of standardized designs, the question of experimenter bias remains open. The principle that public money should not pay for poor design was at the heart of the OGC's Achieving Excellence in Construction programme, so it was a longstanding government policy.

57 Author's personal experience and reported by interviewees.

58 Reported by interviewees.

59 Author's personal experience.

60 A coalition between the Conservative and Liberal Democratic Parties led by Conservative Prime Minister David Cameron.

61 Reported by an interviewee.

62 Pitt (2014), reporting a speech to the Ecobuild Conference 2014 by Paul Hackett, formerly a SPAD to New Labour Deputy Prime Minister Prescott.

63 See the Coalition government's various policies to encourage localism, though this attitude is not, in the author's experience, unique to the coalition parties.

64 An acronym meaning ‘Not In My Back Yard' is used to refer to objectors to new development which may affect their personal interests.

65 See note 2.

66 At the time of writing, seven members of the Cabinet had degrees in philosophy, politics and economics (PPE), four were lawyers, three were economists, two were historians, and one each read classics, politics, agriculture, law, social anthropology and geography. One trained as an army officer. The Labour Shadow Cabinet comprised five PPEs, five with politics degrees, three historians, two lawyers, two linguists, two social scientists, one with an English degree, one with a Russian and Eastern European studies degree, one economist, two whose degrees were not stated online, and one who studied at a further education college whose qualification is not stated. This is not to say that these political leaders are unable to comprehend or appreciate design in the built environment, just that very few of their degrees would prepare them to understand it technically or see it as a priority (various online sources).

67 Ben Bradshaw, for example, was expecting to be presented with a solution to the difficulty in achieving consistent performance from LIFT partnerships.