Applying Social Return on Investment (SROI) to the built environment

ABSTRACT

The effective measurement and dissemination of the impact of design on building users requires an evaluative shift away from measuring building performance towards measuring the outcomes experienced by people. This agenda shares considerable overlap with the emerging concept of ‘social value’ and it is proposed that social impact valuation methods could fill the post-occupancy quantification gap. A review of the social impact sector identifies Social Return on Investment (SROI) as the most developed method with a robust framework for implementation. SROI generates monetized results, anticipated to enhance transferability compared with typical post-occupancy evaluation summaries and facilitate the dissemination and usefulness of findings within the design and construction industry. An in-depth account is presented of a study that trials the six stage SROI method in three non-clinical healthcare buildings; interviews, focus groups, user surveys and financial valuation techniques are used. The SROI results are summarized and the discussion provides a critical reflection on the application of SROI to the built environment: the technical challenges faced, the modifications made as a result, and the lessons learnt from this process. The conclusion offers practical recommendations for future applications of SROI to the built environment.

KEYWORDS:

• building design
• design evaluation
• post-occupancy evaluation
• social value
• Social Return on Investment
• valuation methods

Introduction

There has been a growing interest in the design quality of the built environment in the UK since the late 1990s (Macmillan, 2006). A long line of built environment tools have been designed and promoted in order to capture post-occupancy feedback from users of building design and public space. According to Mulgan et al.’s (2006) comprehensive review, these methods have tended to be based on multi-criteria analysis (MCA) and the weighting and scoring of pre-identified criteria, for example, the Design Quality Indicator (DQI), as well as environmental economics-based approaches such as stated and revealed preference studies, and impacts-based approaches. Value mapping in the built environment is currently dominated by MCA approaches. For example, the UK healthcare sector employs a number of MCA-based assessment tools including the Achieving Excellence Design Evaluation Toolkit (AEDET), a Staff and Patient Environment Calibration Tool (ASPECT) and Patient-Led Assessment of the Care Environment (PLACE).

There are three distinct problems with the MCA approach. The first is a lack of transferability in the results of MCA-based methods, as they tend to be qualitative in nature (Vischer, 2009). In order for best practice to be disseminated effectively, there is a requirement for robust quantification of post-occupancy feedback beyond MCA-based scores. Secondly, the results lack relatability for decision-making that is necessarily based on economic valuation. The design and construction industry is profit-based, yet user feedback is not presented in a way that can be easily factored into project budgets. Third, post-occupancy methods often emphasize building performance rather than user experience. A shift in evaluative focus is required to understand the impact of design for building users (Watson, Evans, Karvonen, & Whitley, 2016a). The subject of analysis should be the outcomes experienced by people rather than predetermined criteria, for example, the outputs of design like build quality or function, specific physical aspects like indoor air quality or acoustics, or fixed user outcomes like satisfaction and productivity, which are typically measured.

The need to overcome the limitations of existing post-occupancy methods by capturing the user experience of building design dovetails with the increasingly popular concept of ‘social value’ (Watson, Evans, Karvonen, & Whitley, 2016b). The Public Services (Social Value) Act 2012 has placed social value firmly on UK public agendas and accelerated the impact-evidencing activities of mission-led organizations and programmes. A wide range of social impact valuation methods have been developed for this end, coinciding with requirements to capture post-occupancy feedback from building users in a more meaningful and transferable way. This paper considers the potential of Social Return on Investment (SROI), a well-developed social impact valuation tool used in the UK, Europe, North America and beyond, to fill the quantification gap of current post-occupancy tools and effectively capture and disseminate the intangible impact of design for building users. The aim here is to trial the SROI method in an exploratory manner in a set of case buildings and to provide critical methodological reflection on the challenges faced. This represents applied research as part of the emerging ‘social value agenda within buildings’ (Watson et al., 2016b, p. 520). The following section reviews the range of social impact assessment methods that offer new opportunities for capturing social value in the built environment, and discusses the choice of SROI for this project. The stage-by-stage application of SROI to three case buildings is outlined in the methods section, followed by a summary of the SROI results. The discussion outlines the methodological challenges faced, the modifications required, and shares the lessons learnt from this process. The conclusion offers recommendations for the future application of SROI to the built environment.

Literature review

This paper is situated within a broader academic discussion about the need to improve post-occupancy evaluation (POE) methods to generate results which are both transferable and relatable to the development process, in order to facilitate the translation of knowledge in the learning loops of design (Preiser & Vischer, 2005; Vischer, 2009; Whyte & Gann, 2001). The surge of interest in design quality over the last few decades (Macmillan, 2006) has coincided with the development of a variety of post-occupancy tools to evaluate buildings from a user perspective. These include the renowned Probe studies (Bordass, Cohen, Standeven, & Leaman, 2001) in the 1990s which employed the building-use studies (BUS) method, Soft Landings (Way & Bordass, 2005), the AMA Workware nexus (Alexi–Marmot Associates, 2008), and the DQI (Gann, Salter, & Whyte, 2003; Gann & Whyte, 2003). POE was developed specifically to evaluate building performance from a user point of view to generate feedback for design professionals and building managers and facilitate the improvement of building design and functioning over time (Deuble & de Dear, 2014; Preiser & Nasar, 2007). However, these various tools have tended to be based on MCA techniques involving the weighting and scoring of pre-established criteria (Mulgan et al., 2006; Vischer, 2009) to produce qualitative summaries and value maps, which has inhibited the take-up of feedback in the learning loops of future projects.

Recent academic work has sought to emphasize the lack of attention given to the internal dynamics of the building user community and discusses the role that sociality plays in mediating the interactions between buildings and users (Vischer, 2008; Watson et al., 2016a, 2016b). The intangible impact of design on building users cannot be understood without consideration of the social context that mediates user experience, yet existing post-occupancy methods measure predetermined criteria about building performance. A shift in evaluative focus is required, away from measuring building performance from a user perspective towards measuring the outcomes experienced by building users as a result of the dynamic interactions between buildings, users and the social context that mediates them.

The need to capture post-occupancy feedback from building users in a more meaningful way shares considerable overlap with the concept of social value and the impact-evidencing activities of mission-led organizations and programmes (Watson et al., 2016a). Recognizing the subjective, malleable and variable nature of social value is key to the development of metrics suited to its capture and measurement (Mulgan, 2010). A wide range of social impact valuation methods now exist and Mulgan (2010) presents a concise summary of the main approaches currently used across social enterprise and public sector social value measurement. A comprehensive review of the various methods that fall within the social impact assessment category is provided in Table 1 with a glossary of key terms shown in Table 2. These tools measure social impact via the efficiency of funding decisions, namely, outcomes relative to inputs (Cugelman & Otero, 2010). There are a range of tools, including the process-oriented, e.g ., the social enterprise balanced scorecard (BSC) and the ongoing assessment of social impacts (OASIS), impact assessment to facilitate organizational decision-making, e.g ., benefit–cost ratio, best available charitable option (BACO) ratio and social accounting and auditing (SAA), and those with an external focus, e.g ., expected return, some of which also take a stakeholder engagement focus, e.g ., basic efficiency resource (BER) analysis and SROI. Not all methods are able to quantify social impacts, and only a few specifically aim to place a value on impact.

Table 1. Summary of social impact assessment methods.

Method/tool	Developer	Description	Benefits	Limitations
Social enterprise balanced scorecard (BSC)	Robert Kaplan and David Norton’s BSC model (1992) modified by Social Enterprise London for the charitable sector	Internal management tool. Process focused. Visual representation of strategic objectives for multiple bottom lines, including social impact. Identification, achievement and measurement of two to four key goals (Clark, Rosenzweig, Long, & Olsen, 2004; Sanfilippo, Cooper, Murray, & Neitzert, 2009)	Useful for strategy development. Key goals communicated in a focused way. Encourages their deeper measurement via another social impact assessment method	Limited in scope to the essentials, not in depth. No quantitative results. No external validation or certification
The third sector performance dashboard	Social Firms UK – developed from an internal performance management tool	CD-ROM tool based on the BSC (see above). Software to monitor performance against typical objectives in the sector using template measures or build your own. Can be used for projects, programmes or a whole organization (Sanfilippo et al., 2009)	Simple, practical, user-friendly – provides templates and samples for busy organizations to use/adapt as required. Clear self-assessment of performance against set objectives. Can generate reports for external audiences	A dashboard of quantitative indicators – no exploration of long-term outcomes or impacts. No real external use. No external validation or certification
Ongoing assessment of social impacts (OASIS)	Roberts Enterprise Development Fund (REDF), 1999	Social management information system to assess organizational outcomes. Integrates tracking practices with mission goals (Maas & Liket, 2011)	Outcomes measurement based on credible research methods	Tracking process limited to outcomes of two years (Clark et al., 2004)
Social return assessment (SRA)	Pacific Community Ventures, 2000	Social investment portfolio system to assess the social return of each investor. Distinct from financial performance assessment. Retrospective. Process focused (Maas & Liket, 2011)	Useful to target and improve services. Social returns in financial form	No impact measurement
Social accounting and auditing (SAA)	Social Audit Network; also Jed Emerson’s blended value accounting (Emerson, 2003)	Organizational framework for monitoring, evaluation and accountability. Evidence base for societal impact of economic activities for both internal and external stakeholders. Also used for internal decision-making. Does not require monetization of outcomes (Gibbon & Dey, 2011; Sanfilippo et al., 2009)	Flexible and holistic method for evaluating organizational performance and impact. Stakeholder engagement focus. Useful for strategic planning, CSR (corporate social responsibility) activities. External validation of social accounts	Time intensive, not yet recognized by funders. Flexibility of process reduces comparability of results. No benchmarking
Social impact measurement for local economies (SIMPLE)	Social Enterprise London and University of Brighton	Organizational framework to understand, measure and communicate impact. Internal strategic review combined with outcomes-based assessment. Five stages: scope, map, track, tell and embed (McLoughlin et al., 2009; Sanfilippo et al., 2009)	Quantifiable data collection. Strategic perspective to analysis. Holistic use across various levels of an organization	Time intensive at the beginning. No external validation or certification
Benefit–cost ratio	Robin Hood Foundation	Framework for making philanthropic investment decisions via predictive cost–benefit analysis. Underpinned by the principle of ‘relentless monetization’: outcomes are assigned monetary value relentlessly, even when they are hard to measure or evidence is slim. Can direct spending to programmes that do the most good per dollar of costs (Weinstein & Bradburd, 2013)	Offers a common yardstick to measure the success of philanthropic outcomes. Flexible seven-step framework (Clark et al., 2004; Weinstein & Bradburd, 2013)	Time intensive and costly. Requires a wide skill set. Broad-brush approach, no replicable method
Social Return on Investment (SROI)	Roberts Enterprise Development Fund (REDF) in the mid-1990s in the US; brought to the UK in 2003 by the New Economics Foundation (nef); The SROI Network was established in UK in 2008 and renamed Social Value UK in 2014	Outcomes-based measurement tool, related to cost–benefit analysis. Project/activity focus. Mixed methods: qualitative stakeholder engagement, quantitative outcomes measurement, valuation via financial proxies, to produce the SROI ratio of costs to social returns. Also produces a narrative of the organization’s value creation (Clark et al., 2004; Sanfilippo et al., 2009)	Produces a transferable, financial metric in ‘ROI’ (return on investment) language understood by investors and commissioners. Credible results based on actual data and proxy research. nef’s version promotes consistency in approach for robust results. Can be applied as predictive or evaluative. External validation through Social Value UK and others. One of the most developed social value tools	Time intensive and costly. Requires a range of skills, from stakeholder engagement to spreadsheet analysis. The ratio is often used out of context of the accompanying narrative report
Social e-valuator	d.o.b. Foundation, Noaber Foundation and Scholten Franssen (Dutch consultancy)	Web-based software acting as a guide to the SROI method. Provides information, expert knowledge and on-line training (Anyetei, 2012)	Provides a comprehensive guide to the complex SROI process which requires a range of skills (as above)	Software is purchased under licence
Basic efficiency resource (BER) analysis	Dr Brian Cugelman and Eva Otero for the Oxfam GB global climate change campaign	Cost-effectiveness-based framework for the evaluation of complex programmes by enabling a comparison between operational ‘units’, e.g., teams, departments, functions. Shares underlying the theoretical foundation with SROI, but no financial valuation. Over- or underperformance visualized in two-dimensional matrix of quadrants (Cugelman & Otero, 2010; Eurodiacona, 2012)	Simple framework for the evaluation of complex activities. Provides results that are easy to understand. Initiates a deeper discussion to understand why units fall into specific quadrants. Can be used to compare internal and external perspectives	Time intensive and costly (less so than SROI). Lack of quantitative results. Not to be used as the only evaluation approach. No monetization. No external validation or certification
Best available charitable option (BACO) ratio	Acumen Fund	Prospective project-focused tool to identify how best to allocate philanthropic resources. Compares cost per output of intended project with similar project. No impact measurement or valuation (Lee & London, 2008; Weinstein & Bradburd, 2013)	Cost per output focus means output units common to both projects are used, e.g., number of end users. Avoids the need for aggregating across various types of outcomes which is complex	Cannot direct funding decisions across different types of projects. Impacts are not valued
Cost per impact	Center for High Impact Philanthropy	Metric to estimate the cost of achieving a single, primary outcome. Like an inverse cost–benefit analysis. Retrospective. Empirical definition of success and the costs associated with achieving that success (Maas & Liket, 2011; Weinstein & Bradburd, 2013)	Useful starting point from which to evaluate an opportunity	Cannot aggregate across different categories of outcomes
Expected return	William and Flora Hewlett Foundation	Measure for the expected impact of a grant. Benefit monetized and multiplied by the probability of success. A prospective cost–benefit analysis (Maas & Liket, 2011; Weinstein & Bradburd, 2013)	Useful for application to large-scale projects	No stakeholder engagement focus. Simplified divide between success and failure

Table 2. Key terms from the social impact assessment sector.

Inputs	The resources that go into the intervention
Outputs	The product of the intervention, i.e. the number of end users reached
Outcomes	The difference that the intervention makes, i.e. the changes that end users experience
Counterfactual evidence	The outputs and outcomes that would have happened regardless of the intervention
Impact	The overall effect of the outcomes minus counterfactual evidence

According to Weinstein and Bradburd (2013), there are four main elements to consider when comparing social impact valuation techniques. This framework was used to compare the existing social impact methods, identifying SROI as the most developed:

• Measuring outcomes rather than tracking outputs (i.e., the number of end users)

  Reviewing the variety of social impact tools that currently exist showed them all to differentiate between measuring outcomes rather than tracking outputs, with the exception of BACO.

• The ability to compare the value of different types of benefits

  Whilst the cost of creating social impact is considered by most of the methods, only a small number place a value on social returns via monetization techniques, thereby facilitating comparison across different types of benefits and consideration of value returned. These methods are social return assessment (SRA), benefit–cost ratio, SROI and expected return. SRA disseminates the social returns of investors in financial form, but this equates to the returns of their processes, rather than impact measurement. Benefit–cost ratio, SROI and expected return all apply financial proxies to social returns in order to evidence their value in a more transferable way.

• The consideration of counterfactual evidence (other factors) in impact creation

  Unlike benefit–cost ratio and expected return, SROI has a robust and replicable method with detailed guidelines, including comprehensive impact establishment activities to prevent claiming for counterfactual evidence. Furthermore, expected return does not emphasize stakeholder engagement whereas this is a key aspect of the SROI method to enhance the authenticity of identifying, measuring and establishing the impact of social outcomes.

• Usefulness to effective and coherent funding decisions

  SROI can be applied prospectively and retrospectively compared with the prospective benefit–cost ratio application and expected return, boosting its usefulness to investment decisions and learning from completed projects.

SROI therefore emerges as the most developed social impact method, being the only tool that satisfies all four aspects of Weinstein and Bradburd’s (2013) framework. It is designed to measure the outcomes of an intervention, rather than solely tracking outputs, and its monetization technique facilitates the comparison of otherwise incommensurable benefits across different activities. The results are disseminated using ‘return-on-investment’ language that is familiar with investors and commissioners, and are based on real data collected through qualitative stakeholder engagement to ensure that what is being measured is what matters to end users. This is realized through a comprehensive method that is robust and replicable due to recent standardization work (Aeron-Thomas, Nicholls, Forster, & Westall, 2004). As a result, SROI has achieved a significant deal of traction within the social enterprise sector (Heady & Keen, 2010) and, increasingly, public policy (Musto, 2015; Shaheen, 2011) and commercial industry (Battle, 2014).

The main point of difference that SROI delivers beyond existing methods in the built environment arena is the monetization of outcomes identified through qualitative stakeholder engagement, producing a transferable evidence base that can be communicated to a wide range of audiences. It therefore has potential as a novel post-occupancy tool to capture the impact of design for building users and disseminate the findings in a more powerful way across the variety of actors in the design and construction sector.

Social Return on Investment (SROI)

SROI is a method that measures and accounts for a wider concept of value for the attainment of multi-bottom lines in the not-for-profit sector. It was originally developed in the US by the Roberts Enterprise Development Fund (REDF) in the mid-1990s. It has been further developed by the New Economics Foundation (nef) in the UK since the late 1990s with a widely applicable stakeholder emphasis and a standardized method. It is based on traditional cost–benefit analysis and assigns a monetary value to social returns using financial proxies, which are compared against the level of investment to produce an SROI ratio of costs to social outcomes. An SROI analysis can be retrospective or prospective, encompass the value generated by an entire organization or focus on specific programmes or projects. It is widely used in the UK social enterprise sector to evidence organizational impact and is endorsed by the Cabinet Office.

The SROI method has six stages as set out in A Guide to Social Return on Investment (Nicholls, Lawlor, Neitzert, & Goodspeed, 2012):

1. establishing scope and identifying key stakeholders

2. mapping outcomes

3. evidencing outcomes and giving them a value

4. establishing impact

5. calculating the SROI

6. reporting, using and embedding

SROI in practice

A number of critiques have been raised against SROI, both conceptual and practical, across academic and applied literatures. The growing trend towards monetizing social outcomes is considered a symptom of the not-for-profit sector’s marketization, harming its ability to create and maintain a strong civil society (Eikenberry & Kluver, 2004). Reducing social outcomes to monetary measures is considered to be problematic and often unwarranted, oversimplifying and underestimating the full impact of an intervention (McLoughlin et al., 2009; Wright, Nelson, Cooper, & Murphy, 2009). However, proponents of SROI argue that intangible outcomes are brought into the conversation by presenting them in a common unit of value. Another critique is that the SROI ratio is often disseminated as the sole finding of the analysis (Wright et al., 2009), yet is meaningless out of context. It requires appraisal with a predicted ratio, a preceding ratio or a corresponding ratio from a similar organization to evaluate the impact created (Olsen & Lingane, 2003). The SROI method (Nicholls et al., 2012) promotes contextualising the monetized results with qualitative stakeholder narratives and quantitative evidence of reported outcomes to communicate the full story of creating value.

An applied critique levelled at the SROI model developed by REDF was its lack of standards, making results unreliable and incomparable (Olsen & Lingane, 2003). nef therefore developed the six stage framework to ensure consistent application by a wide range of organizations. Undertaking an SROI analysis is a resource intensive process which can be problematic for charities, where access to human and financial resources may be limited. It also requires a significant degree of input from the individuals responsible for the intervention to drive the social impact parameters under investigation and avoid a ‘top-down’ evaluation (Cresswell, 2004). As a research activity, a process of mutual learning is recommended between the SROI practitioner and the organization under study, with continual communication and dialogue (Rotheroe & Richards, 2007). Identification of a central point of contact is therefore critical to access the contextual knowledge, information and support required (Rotheroe & Richards, 2007). Data quality and availability issues are common and can lead to a narrowed focus on internal activities and achievable objectives, rather than a full range of impacts both inside and outside the organization (Aeron-Thomas et al., 2004; Olsen & Lingane, 2003).

Another provocative aspect of SROI is the projection of social impact into the future. The guidelines suggest a discounting rate of 3.5%, as recommended in the Green Book (Nicholls et al., 2012). However, Emerson, Wachowicz, and Chun (2000) argue that a range of discounting rates are used as appropriate, due to the lack of comparative market rates available, observing that higher discount rates drive down the projected rate of return on investment. Olsen and Lingane (2003) also believe a rigorous selection process is required rather than accepting a standard discounting rate, based on the uncertainty of achieving social impacts over several years. Yet the SROI process involves numerous estimates including the discounting rate, financial proxies and impact establishment measures used. Sensitivity analysis is undertaken to identify which estimates have the greatest impact on social returns, informing resource allocation to ensure continual improvement. Ongoing monitoring of social impact is therefore required for the effective integration of SROI into organizational operations (Olsen & Lingane, 2003).

Methods

The SROI method was critically trialled in an exploratory manner in the built environment to investigate its applicability as a form of POE, taking a case-based approach to provide distinct test sites for the framework. This is the first time SROI has been applied to buildings in this way, as opposed to the wider impact of a building-related project within the local community (Aspden, Ball, Roberts, & Whitley, 2012). The six stages of the SROI method, which represent current prevailing practice in the UK social impact sector (Nicholls et al., 2012), were developed into a framework of analysis suitable for the built environment, summarized in Table 3 and detailed below.

Table 3. Application of the six stages of the Social Return on Investment (SROI) to the case buildings.

Stages of SROI	Application to case buildings
1. Establishing scope and identifying stakeholders	Establish scope
	Identify case buildings Establish what is being studied – internal impacts to building users, not external impacts to local economy/community, etc. Semi-structured interviews with project teams (n = 17)
	Identify stakeholders
	Semi-structured interviews with centre managers (n = 3) – identify building user groups
2. Mapping outcomes	Focus groups with each building user group in each case building (n = 12)
3. Evidencing outcomes and giving them a value	Evidence outcomes
	User survey for each building user group in each case building (n = 12) Give them a value Desk-based research to identify and apply financial proxies to outcomes
4. Establishing impact	Desk-based analysis of user survey data to calculate deadweight and attribution measures
5. Calculating the SROI	Inputting the building costs
	Calculating the ‘added cost’ using an NHS benchmark
	Inputting the social returns
	Sum of the total impact of outcomes
	Calculating the SROI ratio
	Social returns divided by ‘added cost’ of building
	Projecting into the future
	Application of discounting rate across useful time points, e.g., 10 years before a refit is required, 60 years typically the lifetime of design
	Payback period
	Calculating the number of months before the social returns equal the original investment
6. Reporting, using and embedding	Further research to assess whether the SROI has effectively captured the social value of the case buildings Report findings to case organizations Further research to understand whether the SROI is useful to designers, commissioners and other decision-makers Further research to assess whether it could be embedded in the design or commissioning process

Establishing scope and identifying stakeholders

Establishing scope involved the selection of case buildings and defining what is under study. Three cancer support centres with varying levels of user-centred design and different funding, governance and organizational set-ups were selected from the non-clinical healthcare sector in the UK (Table 4) to provide a range of environments to trial SROI (Watson et al., 2016a). An analytical barrier was set around each case building. The impacts under investigation were those accruing to the users due to the building, rather than the services being offered, nor external societal impacts such as local economic benefits or cultural value. Semi-structured interviews were conducted with the project teams of each case building, including client, project manager, cost manager, architect, engineer (structural, mechanical and electrical), interior designer and lighting consultant professionals, to gain a thorough understanding of the building design and the designers’ intentions. In total, 17 interviews were carried out, nine with the project team of case building 1, five for case building 2 and three for case building 3, based on the number of professionals involved in each project. The interviews were voice recorded, transcribed and coded using the qualitative analysis software Atlas.ti.

Table 4. Summary of cancer support centre case buildings.

Case building	Organization	Build type	Size (m^2)	Built environment	Clinical/non-clinical	Rooms provided	Services offered
1	Charitable	New build	360	Domestic scale, standalone building within an NHS hospital campus	Non-clinical design (homely)	Kitchen, library, private rooms, activity room, office	Psychosocial support, benefits advice, nutritional advice, low-impact exercise sessions, arts and crafts sessions
2	Charitable	Fit out	300	Ground-floor unit in a mixed-use commercial development in an inner-city context	Non-clinical design (spa-like)	Reception, library, therapy rooms, activity room, several offices	Psychosocial support, complementary therapies, nutritional advice, low impact exercise sessions
3	NHS	Structural retrofit	106	Non-clinical space within an NHS hospital building (between a ward and an external, fully glazed access corridor overlooking a walled garden)	Clinical feel (NHS environment)	Library with a reception desk, private consultation booths, several offices	Psychosocial support, benefits advice, signposting service to relevant information

Source: Watson et al. (2016a, p. 294).

Stakeholder identification was undertaken via a semi-structured interview with the centre manager of each case building to learn how the buildings operate on a day-to-day basis, who uses the space, and identify ‘building user groups’ (Watson et al., 2016b) to represent the key stakeholders. Several identifiable categories of building users were summarized into four user groups: visitors, centre staff, session leaders/therapists and volunteers (Table 5), forming the basis for the SROI analyses.

Table 5. Number of building users in each case building.

	Case building 1	Case building 2	Case building 3
Visits	10,517	4913	11,860
Centre staff	6	6	4
Session leaders/therapists	19	12	4
Volunteers	17	15	3

Mapping outcomes

Qualitative stakeholder engagement was conducted with each user group. A set of four focus groups were run at each building, one per user group, with approximately five participants in each session. The participants talked about their experiences of the buildings including their needs and preferences, how they use the space, how the design impacts them, and their opinions on improving the space. The focus groups were voice recorded, transcribed and coded using Atlas.ti for outcomes reported, such as well-being, care performance and revisit intention, and design features mentioned, such as homely environment, colourful furniture and large windows. The data analysis focused on identifying outcomes experienced by the user groups, and the design features they reported had impacted those experiences. The outcomes were inputted into a spreadsheet per building, according to the total number of individuals per user group (Table 5).

Evidencing outcomes and giving them a value

This is the most complex and time-consuming stage of SROI involving prolonged fieldwork followed by extensive desk-based analysis. The outcomes were evidenced through quantitative user surveys, one for each building user group in each building. A core principle of SROI is measuring what the stakeholders reported rather than predetermined outcomes, requiring a separate survey for each user group in each building. A total of 12 surveys were designed.¹

The survey design was initially based on the BUS questionnaires (Cohen, Standeven, Bordass, & Leaman, 2001) in terms of layout and item scales.² However, the purpose of the SROI user surveys is to measure outcomes in the building users, requiring the creation of new items with more similarity to social impact methods than typical post-occupancy surveys. Whilst the outcomes under study were different on each survey, the consistent sections are outlined in Table 6. The surveys were distributed in paper form in each building for several weeks up to several months, according to the speed of response rates. The data were collated electronically and analysed for descriptive statistics (frequencies and averages) to produce quantitative measurement of the outcomes reported by each user group. This was inputted as the proportion of each user group in agreement with the reported outcomes to calculate total incidence. It should be noted that the user surveys were designed and analysed in an exploratory manner and do not represent validated tools.

Table 6. Main sections in the user surveys.

Introductory paragraph: survey purpose and research team details Background information: user group status, age, time spent in building, time spent in specific spaces Agreement to reported outcomes Cross reference reported outcomes and reported design features Rate the impact of reported design features Information required for Stage 4 (see next Methods section) Rate percentage change to alternative outcomes identified during the transcript analysis

The second half of stage 3 places a value on the quantified outcomes. The SROI method uses financial proxies to assign a value to the outcomes, based on valuation traditions in environmental and health economics fields. Suitable financial proxies were identified via primary data from the occupying organizations, for example, average wages and cost-per-hire data, and through an exploratory desk-based literature search of academic, public sector and social enterprise publications relating to the measurement of social impact. Increasing interest in social value has led to a growing body of work relating to the ‘subjective wellbeing approach’ (Fujiwara & Campbell, 2011), described as an ‘established alternative to traditional preference-based valuation methods typically used in cost benefit analysis’ (Fujiwara & Dolan, 2014, p. 3). National-scale datasets containing people’s lived experiences of specific interventions are statistically analysed to attach value to the outcomes. This is considered the latest development in social impact measurement, beyond preference-based methods with unavoidable bias (Fujiwara & Campbell, 2011). The subjective well-being approach has spawned an increasing number of publications containing financial proxies, e.g ., the value of improved well-being, having a hobby or being able to seek advice locally. A particularly useful source of financial proxies was the recently published Measuring the Social Impact of Community Investment: A Guide to Using the Wellbeing Valuation Approach from housing innovator HACT (Trotter, Vine, Leach, & Fujiwara, 2014). These proxies represent the value to an individual for a full year being intended for application to the long-term outcomes of social interventions, rather than the impact of design on building users for a specific time period. Where applicable, adjustment was made from yearly to hourly values and from hourly to the average number of hours spent in the building per user group, calculated from the survey data.³ Table 7 presents a summary of the financial proxies identified for the current study. The financial proxies were inputted alongside the relevant outcomes reported by each user group. The total incidence multiplied by the financial proxy produces the value created by that outcome for that user group in a year.

Table 7. Financial proxies identified for SROI analysis of case buildings.

Financial proxy	Value	Source
Cost of suffering from depression or anxiety per individual p.a.	–£44,237	Valuing Mental Health: How a Subjective Wellbeing Approach Can Show Just How Much It Matters (Fujiwara & Dolan, 2014)
Value of being relieved from depression or anxiety per individual p.a.	£36,766	Measuring the Social Impact of Community Investment: A Guide to Using the Wellbeing Valuation Approach (Trotter et al., 2014)
Value of regular attendance at a local organization per individual p.a.	£1773	As above
Value of being able to obtain advice locally per individual p.a.	£2457	As above
Value of being a member of a social group per individual p.a.	£1850	As above
Value of having a hobby per individual p.a.	£1515	As above
Unit cost of a general practitioner visit in the UK	£49	Unit Costs of Health & Social Care 2013 (Curtis, 2013)
Average salary for centre staff in each case building	–	Primary data
Average wage per hour for session leaders/therapists in case buildings	£36	Primary data
UK minimum wage per hour for over 21s (2013/14)	£6.31	https://www.gov.uk/national-minimum-wage-rates
Cost per hire for employees in the not-for-profit sector (not senior managers or directors)	£1500	Resourcing and Talent Planning 2012 (Sinclair, 2012)

Note: p.a. = per annum.

The inherent subjectivity of the monetization process requires that a full account of how each valuation has been calculated is made available, especially where multiple financial proxies can be identified for any one outcome. Transparency is considered a core principle of the SROI method (Nicholls et al., 2012).

Establishing impact

The values placed on the outcomes are interrogated to establish their impact, prevent over-claiming and enhance credibility (Nicholls et al., 2012). Impact relates to four counterfactual aspects: deadweight, displacement, attribution and drop-off.

Deadweight

Deadweight is a measure of the proportion of the outcome that would have happened anyway, regardless of the intervention. The deadweight principle was applied to the proportion of the outcomes not reported as improved in the survey data, e.g ., if visitors reported well-being improved by 35%, then 65% of the value placed on well-being cannot be claimed for and is removed as deadweight. This process was carried out across the outcomes for all user groups in each case building.⁴

Displacement

Displacement is a measure of whether a positive outcome has simply displaced other negative outcomes elsewhere. It is not always relevant to every analysis (Nicholls et al., 2012) and is not considered applicable to post-occupancy analysis of the built environment.

Attribution

Attribution is a measure of how much of the outcome can be attributed to the intervention, rather than other factors, e.g ., the way the building is managed, the services being offered or the interaction between building users. These data were collected through the user surveys and the average attribution percentage applied to each outcome. For example, if visitors considered 85% of improved quality of care to be attributable to the building design, 15% of the value was removed. This process was carried out across the outcomes for all user groups in each case building.

Drop-off

Drop-off is a measure of how long outcomes last for over time. It is usually calculated by removing a fixed percentage from the remaining level of outcome after each year (Nicholls et al., 2012). Whilst relevant, the drop-off measure is affected by the visitor user group which is not fixed, but comprised of a changing set of new visitors each year alongside a decreasing number of long-term re-visitors. Whilst data were collected through the user surveys, analysis produced an unexpected finding according to the SROI method which is designed to take account of diminishing outcomes. On average, all user groups recorded no change to how much they experience outcomes from first using the building until the present. The visitor user group actually reported an average increase over time.

A possible explanation is the likelihood that factors aside from building design will influence users increasingly over time and enhance the outcomes they experience, e.g ., human factors like developing friendships will positively influence outcomes of well-being, re-visit intention or staff retention, whilst building management factors could negatively affect outcomes like stress or reduced productivity. However, the attribution measure is designed specifically to combat this issue. The lack of drop-off could also be explained by an increasing attachment of the users to the buildings over time. Contextualizing the quantitative data with the qualitative stories from the focus groups suggests this could be a factor. Further research is required to understand how design-related outcomes affect building users over time and whether there is a relationship between users ‘falling in love’ with buildings causing positive outcomes like well-being to increase over time.

The unexpected increases to certain outcomes reported by the visitor user groups were not accounted for due to the limited evidence for this phenomenon. This is recommended by the SROI method (Nicholls et al., 2012) in the unlikely event that an increase in value is reported rather than a reduction.

The deadweight and attribution measures identified in the user survey data were inputted against the relevant outcomes as percentages to be removed, producing a final impact-adjusted valuation for each outcome.

Calculating the SROI

The calculation of the SROI requires five separate elements: inputting the building costs, inputting the social returns, calculating the SROI ratio, projecting into the future and sensitivity analysis.

Inputting the costs

The building costs were calculated from the professional design fees, contract sum (construction cost) and the furniture, fittings and equipment (FF&E) cost. The point of interest was the impact of the additional spend above typical levels for a non-clinical healthcare building in the UK. Therefore, the costs were compared with an NHS benchmark for a similar build in order to calculate the ‘added cost’ of the buildings.⁵

The costs were sought during the project team interviews. All costs were inputted as commercial estimates⁶ due to the charitable nature of case buildings 1 and 2, whilst case building 3 was an add-on to an existing clinical scheme in an NHS hospital.

An NHS estimate was produced for each building based on the benchmark contract sum figure of £2040/m² for a new-build NHS primary care centre found in the Department of Health’s Healthcare Premises Cost Guide (2010, p. 22). Benchmark contract sum figures were calculated by the internal footprint of each case building, with professional fees estimated at 15% of the contract sum, a standard based on the secondary author’s industrial experience. As an FF&E benchmark could not be found for a non-clinical NHS setting, the figure collected during the primary research at case building 3 was modified to produce a benchmark of £24.25/m² and adjusted according to the footprint of case buildings 1 and 2. The benchmark fees, contract sum and FF&E costs were combined to produce a total NHS benchmark cost per building, which was subtracted from the commercial estimate to leave the ‘added cost’ of each case building.

The ‘added cost’ figures for case buildings 2 (fit-out) and 3 (retrofit) required further modification (Table 8) due to the complexity of capturing the impact of design in these build types. The impact of the fit-out cannot be isolated from that of the shell-and-core retail unit, and the impact of the retrofit cannot be isolated from that of the surrounding built form. The commercial estimate for case building 2 was increased to include the shell-and-core unit. For case building 3, however, as no reliable estimate was available for the adjacent glass corridor, the NHS benchmark was adjusted to represent a retrofit at 33% of the cost of a new build. The comparability of the ‘added cost’ figures is affected, as for case buildings 1 and 2 they represent the added cost of the full built environment (new build; fit-out and shell-and-core unit), whereas for case building 3 they represent the added cost of the retrofit only. It is apparent that a considerable degree of assumptions are required as part of the SROI method, emphasizing the importance of transparency as a core principle.

Table 8. Inputting the ‘added cost’ of the case buildings.

Case building	Build type	‘Added cost’ calculations
1	New build	Professional fees, contract sum and furniture, fittings and equipment (FF&E) cost for new build; minus NHS benchmark fees, contract sum and FF&E cost for new build
2	Fit-out	Professional fees, contract sum and FF&E cost for fit-out; plus estimated fees, contract sum and FF&E cost for core-and-shell unit; minus NHS benchmark fees, contract sum and FF&E cost for new build
3	Structural retrofit	Professional fees, contract sum and FF&E cost for retrofit; minus estimated NHS benchmark fees and contract sum for retrofit; minus NHS benchmark FF&E cost for new build

Source: Watson et al. (2016a, p. 296).

Inputting the social returns

A core SROI principle is to take a conservative approach in order to avoid over-claiming and to produce reliable and trustworthy results. Due to the interrelated nature of many of the reported outcomes, only well-being (and stress), performance (care and productivity) and NHS cost savings (due to reduced intention to access psychosocial support services through a general practitioner environment) were included in the analysis. The impact-adjusted value of each outcome per user group was combined to calculate the total value generated by the design of each building for the users in a year.

Calculating the SROI ratio

The SROI ratio was calculated by dividing the value of the outcomes by the added cost of each building. The SROI ratios represent the value created by the design of each case building for the building users in a year, per every £1 spent over and above the NHS benchmark.

Projecting into the future

A snapshot ratio was calculated after one year,⁷ with further ratios projected at useful time points across building lifetime to inform maintenance and re-fit decisions. Whilst discounting is not prescribed by SROI, it was considered appropriate to the commercial design and construction industry and Green Book discounting rates were applied over a 60-year period: 3.5% for the first 30 years and 3.0% from year 31 onwards. SROI ratios were calculated after 10 years, at which point a re-fit is typically required, and after 60 years, the typical lifetime of building design. These ratios represent the value created by the design for the building users over 10 or 60 years, per every £1 spent over and above the NHS benchmark.

It is worth noting that the discounted SROI ratios are likely to underestimate the true value created as they are based on the original number of visits seen in 2013, whereas the case buildings receive an increasing number of visits each year.

Sensitivity analysis

The sensitivity of the SROI analysis was tested to investigate which estimates have the most impact on the overall ratio. The standard sensitivity checks include:

• quantity of the outcome

• financial proxies used

• estimates of deadweight and attribution

• value of inputs

The aim is to calculate the amount each estimate needs to be changed to take the SROI ratio to 1.0, i.e ., from positive to negative, or vice versa (Nicholls et al., 2012). The purpose is to identify priority areas for resource allocation to maintain and improve the return generated over time. The visitor and staff user groups proved more significant in terms of value creation than the session leaders and volunteers, and the visitor outcomes of reduced intention to use NHS support services and well-being, and the staff outcome of performance produced the most value.

Reporting, using and embedding

Detailed SROI results are presented by Watson et al. (2016a) including a discussion of the effectiveness of SROI at capturing the social value of buildings. In addition, an end-user SROI report was produced for one of the occupying organizations interested in applying SROI to more of their centres. The current paper provides critical methodological reflection on the process of applying SROI to the built environment, including a summary of the results produced. Further research has been undertaken to investigate the usefulness of the various SROI data for decision-makers and design professionals of future non-clinical healthcare buildings more widely. A summary of the SROI results are presented below followed by a methodological discussion, including the challenges faced, modifications required and lessons learnt.

Results

This paper delivers critical methodological reflection on applying SROI to the three case buildings. However, a summary of the results generated is included in Table 9 to demonstrate the potential insights and knowledge that SROI can provide about the value of building design for users. The monetized data presented include total impact produced for the end users of each case building, impact/m² to aid comparison, ‘added cost’ of each building including professional fees, contract sum and FF&E costs, and SROI ratios at 1-, 10- and 60-year time points. Case building 1 produces considerably greater impact per year for its building users at £375/m² compared with £229/m² for case building 2.⁸ However, the SROI ratios are similar with £2.07 returned for every £1 spent over the NHS benchmark for case building 1, compared with £1.98 for case building 2, over a 60-year period. This is because the total ‘added cost’ of case building 1 is much higher at £4796/m² compared with £3063/m² for case building 2. The main difference in investment can be identified using the SROI data: £294/m² was spent on the professional fees for case building 2, representing only 36.5% of that spent on fees for case building 1 at £805/m². A fuller account of the results produced by SROI analysis in the built environment is provided in a separate paper (Watson et al., 2016a). However, this summary illustrates the latent potential of SROI as a novel form of POE which can capture and monetize user experience in occupied buildings, as opposed to measuring building performance from a user perspective, and disseminate these insights in a transferable manner with direct relevance to the profit-oriented design and construction sector.

Table 9. Summary of SROI results for the case buildings

Case building	Floor plan (m^2)	Total impact		‘Added cost’ of case building								SROI ratio
				Total		Professional fees		Contract sum		FF&E cost		1 year	10 years	60 years
			/m^2		/m^2		/m^2		/m^2		/m^2
1	360	£134,991	£375	£1,726,710	£4796	£289,840	£805	£1,265,600	£3516	£171,270	£476	£1:£0.08	£1:£0.65	£1:£2.07
2	300	£68,764	£229	£918,925	£3063	£88,200	£294	£688,000	£2293	£142,725	£476	£1:£0.07	£1:£0.62	£1:£1.98
3	106	£23,445	£221	£1,037,937	£9792	£109,296	£1031	£928,641	£8761	£0	£0	£1:£0.02	£1:£0.17	£1:£0.54

Source: Adapted from Watson et al. (2016a, p. 298).

Discussion

The following sections reflect on the challenges faced in trialling SROI in the built environment, both general and technical, the modifications required, and the lessons learnt from the process.

General challenges

A set of high-level concerns relating to the application of SROI in the built environment were considered. The first related to the design of the SROI method to measure a change, i.e ., before and after a social intervention, whereas building design does not represent an intervention in a traditional sense. In these cases the building users did not experience outcomes before and after the design meaning a baseline measure could not be collected. Therefore, the outcomes were measured as self-reported experiences, rather than before and after changes, in the context of the building design.

Secondly, the SROI method is designed to capture the value of social outcomes, and does not investigate and untangle the complex interrelations involved. No causal links can be drawn between the intervention and the outcomes. The outcomes identified and the data collected tend to be material, e.g ., number of participants trained. In contrast, investigating and understanding the dynamic interactions between design and users mediated by their social context (Watson et al., 2016a) is important for a true representation of the value of design, from which lessons can be learnt and applied in future developments. This is problematic when applying SROI to building design as a potential means to close learning loops to promote user-centred design for future development.

Thirdly, it is worth noting that the data-crunching stages of the method are far removed from the qualitative focus group data about specific design features. SROI cannot evidence that certain design features produce specific outcomes based on quantified evidence. The qualitative data from the focus groups and the quantified design features scores from the survey data can help identify the most significant design features. However, this is similar to the information available through existing post-occupancy methods with the exception that it is stakeholder driven, not pre-identified. It should also be noted that the design features sections on the user surveys are superfluous to the SROI method (as this information was not required to produce the SROI data and ratios which are based on the reported outcomes only). Whilst attention is drawn to the value of ‘good’ design overall, SROI does not quantify and monetize the value produced by specific design features. Further research is required to trial SROI in a different way, targeting specific design features and asking building users to identify outcomes they experience as a result. It is anticipated that the impact establishment stage would require considerable resources to produce a set of reliable and trustworthy estimates.

Finally, using an NHS benchmark to calculate the ‘added cost’ of the buildings can be perceived as an assumption that no outcomes would be felt in a standard NHS environment. Whilst there is no reliable way of collecting evidence of outcomes ‘before’ the building design, the deadweight measure does take into account how much the outcomes are reportedly boosted due to the current environment, and removes the difference. This difference represents what would have happened (i.e ., what would have been felt within NHS environments).

Technical challenges

The process of trialling SROI in the built environment faced a set of technical challenges, relating to the following themes: reported outcomes, surveys, financial proxies, impact establishment and calculating the SROI.

A significant challenge related to the interrelatedness of the outcomes: the causal links between improved well-being, relationship building, quality of care and services, and revisit intentions cannot be reliably extricated. Therefore, the final SROI only included reduced intention to use NHS psychosocial support services, well-being and performance outcomes to avoid over-claiming. A second challenge was felt in the identification of outcomes affecting stakeholder groups on different scales, e.g., the reduced intention of visitors to access psychosocial support services through NHS environments was included as a cost saving to the NHS. The occupying organization also represents a stakeholder on a larger scale, with outcomes like brand enhancement also accruing value due to the building design. However, they were not included in the SROI due to concerns about finding reliable indicators, suitable financial proxies and establishing measures of deadweight, attribution, displacement and drop-off within the time and resource limitations. Further research is needed that trials SROI in relation to outcomes experienced by a wider set of building users at different scales, e.g., Vischer (2008) outlines the relations between individuals users, groups of users and organizational scale users of buildings.

Maintaining access to the stakeholders over the time period required to complete the SROI analyses was problematic, particularly in two of the case buildings. Typically the practitioner aims to set up a dialogue with the key stakeholders and, as information is required, practitioners speak with them directly to understand their opinions. It was most successful in case building 1, considered a result of the users’ affection for the building meaning they were keen to offer their experiences. To combat this issue, a quantitative approach was taken by designing user surveys to gather all of the information required throughout the SROI process. This led to the implementation of rather long, unwieldy surveys which were off-putting for potential participants and generated low response rates, particularly by the visitors in case buildings 2 and 3. Case building 2 operates on an appointment basis, meaning visitors do not have the time to fill out a long survey when they are in the building. Case building 3 is managed on a drop-in basis where visitors only spend a short period of time, with similar implications for survey completion. Case building 1 also runs as a drop-in centre, but visitors are encouraged to spend time relaxing in the building and were more willing to spend time on the survey.

The SROI method emphasizes that the outcomes being measured should be directed by the stakeholders. As a result, each user group in each building required a separate survey, leading to an extremely resource intensive period of fieldwork designing, administering and collating data from 12 different surveys. The survey design proved problematic as there is no precedent for items and scales to measure user outcomes of design, rather than building performance criteria. The BUS questionnaires represented a useful starting point, but they do not measure outcomes in building users. Existing survey items for the measurement of well-being, such as the Warwick Edinburgh Mental Wellbeing Scale (WEMWBS) (Tennant et al., 2007), tend to measure long-term personal well-being rather than the well-being experienced due to design for a specific time period. Therefore, the scales on the BUS questionnaire were applied to newly designed items in an exploratory manner.

The identification of financial proxies represents another challenge as this is a new area of research with few centralized sources for outcomes of building design. Furthermore, different types of proxies exist, either organizational data or from the subjective well-being approach literature. Access issues to organizational data caused a reliance on the latter, but ideally these would be backed up with organizational proxies, e.g ., the cost of a service per hour. Best practice involves using a mixture of proxies as, for example, a service may cost £30 but produce a value 10 times that. The modification of the financial proxies for application to the built environment was also highly challenging. Considerable discussion was held before they were adapted from a yearly value to an hourly value to enable adjustment per average hours spent in the building by each user group. The visitor user group was especially complicated as the survey data did not map onto the organizational visits-per-year data. There was no reliable way of calculating how many individuals made up the visitor user group, and this drove the modification of yearly values to hourly values.

Establishing the impact of each outcome proved complex. Firstly, identifying how the impact measures applied to the built environment took considerable debate. A discussion was held about whether deadweight and attribution represent the same thing in the built environment.⁹ Secondly, the unexpected finding that the outcomes were remaining constant, or even increasing, rather than dropping off over time could not be reliably incorporated into the SROI analyses.

The final set of technical challenges related to calculating the SROI. Inputting the cost data proved problematic for a number of reasons. Firstly, accessing cost data from the project teams was difficult due to the pro bono nature of third-sector costs. Similarly, costs for case building 3 were also estimated due to it forming an add-on to an existing scheme. Secondly, consideration was required over what the costs should constitute, whether the full commercial estimate or an ‘added cost’ over and above a benchmark. The decision was taken to use the added cost scenario, leading to further decisions over how to benchmark for the various build types (new build, fit-out and structural retrofit). Whilst the build types were achieved with varying costs/m², the social returns relate to the whole built environment as there is no way to separate the impact of the fit-out from that of the shell-and-core unit in case building 2, or the impact of the retrofitted structure from that of the surrounding structure in case building 3.

It is not typical practice to calculate the SROI ratio before discounting has been completed. However, understanding the return on investment after one year was considered important, and a discounted and a non-discounted set of ratios could then be calculated over the 10- and 60-year time periods. In relation, using discounting rates to project social returns into the future was a considerable point of discussion. The value accrued from the buildings for the user groups should be understood as a yearly snapshot, as it will not be felt by the same set of individuals in the future. This is especially pertinent to the visitor user group which is constantly changing, making discounting to take account of the time preference of money less relevant to this user group. However, SROI has been applied to the built environment with the intention to disseminate the findings in the design and construction sector, where discounting is an accepted and expected activity, typically based on the Green Book guidelines.

Lessons learnt

The final output is the lessons learnt during the application of SROI to the built environment. Firstly, the volume of project team interviews could be condensed to include the client, architect and commercial manager, with any specialist consultants involved in the project as appropriate. Secondly, it is advisable to keep the survey design short and spend resources setting up a successful dialogue with stakeholders. As information is required during the SROI analysis, follow-up focus groups could be carried out with relevant user groups to gather qualitative data, rather than using surveys. This would provide a richer dataset but from a smaller group of participants, i.e ., quality over quantity. It would also remove the risk of misinterpreting quantitative survey data, and the challenge of survey design in a nascent field of enquiry without established items and scales.

Thirdly, it is pertinent to note that a more streamlined SROI process is possible by pre-identifying outcomes in order to reduce significantly the time and resources spent measuring, valuing and establishing the impact of outcomes that cannot be claimed for due to inter-relatability issues. The important outcomes were reduced intention to access psychosocial support services in NHS environments, well-being and performance. In similar analyses, these outcomes could form the sole focus from stage 3 onwards. It is not advisable to pre-identify outcomes prior to stakeholder engagement in stage 2, which provides key insights into user experiences of design by learning about the full range of outcomes. Furthermore, qualitative context is vital to ensure the many decisions and judgement calls made throughout the SROI process are informed and directed by stakeholder experiences.

Conclusions

Recent interest in the design quality of the built environment has produced a range of tools to capture post-occupancy feedback from the users of buildings. However, a reliance on MCA and the scoring and weighting of predetermined criteria has impacted on the transferability of results and their usefulness to profit-related decisions, and overlooked the outcomes experienced by users. The emerging ‘social value’ agenda has led to developments in social impact assessment, dovetailing with requirements to capture post-occupancy feedback from building users in a more meaningful and transferable way. The review of existing social impact valuation tools identified SROI as a well-developed method with significant potential for measuring and disseminating the impact of design on building users through the monetization of social value.

This research paper provides an account of the first applied social value research in the built environment, specifically trialling SROI in three case buildings. The results summary shows the potential of SROI to generate novel insights about the value of buildings for end users and its ability to communicate these findings in a powerful manner. The implications of transferring these findings include their application in commissioning and high-level investment decisions by clients and developers, as well as informing project briefs and acting as a source of user-centred evidence for project teams during the design process, facilitated by the variety of qualitative, quantitative and monetized data available. SROI represents an opportunity to engage with the end users of design to develop and share knowledge about their preferences and experiences and enhances the ability of practitioners to shape the built environment in positive ways. The discussion provided critical methodological reflection, including the challenges faced, both general and technical, the modifications made to the SROI method, and the lessons learnt. The discussion did not consider the effectiveness of SROI at capturing, measuring and disseminating the social value of buildings, which is addressed elsewhere (Watson et al., 2016a).

Recommendations for future applications of SROI to the built environment

• consider data access at an early stage and gather as much data as possible during the early scoping and stakeholder engagement stages to facilitate the complex valuation and impact establishment stages

• condense the number of project team interviews to conserve resources

• consider the quantitative/qualitative balance used to collect the various data required and spend resources securing an ongoing dialogue with stakeholders

• reflect on the environment where the user surveys will be administered and consider using pre-identified outcomes for a shorter survey design if necessary

• consider build type when inputting cost data, especially if benchmarked

• be transparent about how the valuations have been calculated

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCiD

Kelly J. Watson http://orcid.org/0000-0001-7674-6616

Additional information

Funding

This work was supported by the Economic and Social Research Council under Grant number ES/J500094/1.

Notes

1 For an example survey, see https://docs.google.com/document/d/1UlhfBh2Cb57XKKfYSAqVwpWODkE11zYya9ozaAeF71s/edit?usp=sharing/.

2 Seven-point Likert scales, as used in the BUS survey, provided a neutral point and a range of positive and negative points either side. Additionally, a nine-point scale was included, based on the BUS item for productivity, to understand percentage change of outcomes, with 0% as the neutral point and a range of –40% to 40% either side. Additional sections focused on the design features mentioned during the focus groups: firstly, to cross-reference the outcomes against the design features; and secondly, to rate the design features on a seven-point scale.

3 For example, the outcome of improved well-being was valued at £36,776 per individual per year (Table 5). The value of improved well-being per individual per hour is calculated accordingly: £36,766/(365 days × 24 hours) = £4.20. The visitor data for case building 1 showed the average time spent in the building per visit was 2.3 hours, and so improved well-being is valued at £9.66 per visit. The building received 10,517 visits in 2013 according to the secondary data gained from the charitable occupying organization. The total value of the outcome of improved well-being for the visitor user group for the year is calculated to be £101,594.

4 The exploratory nature of the study meant that deadweight data were not collected for all outcomes reported. Estimate deadweight proportions, based on available data for the corresponding user group, were used when necessary.

5 Alternatively, the costs could have been represented by the commercial estimates without comparison with a benchmark, or the charitable rates. Inputting the costs involves making a judgement call as part of the SROI process, emphasizing the importance of transparency throughout. The flexibility of the method allows factors like the purpose of the analysis and the intended audience to direct the way the costs are calculated.

6 If the SROI analyses were being performed for the charitable-occupying organizations, then the costs would be represented by the charitable rates they paid. These would be smaller figures as the project teams and contractors worked at pro bono rates.

7 Discounting is usually undertaken before the SROI ratio to calculate the total social returns created by an intervention over time, typically about 5 - 10 years after a charitable project is completed. The outcomes of design arguably last much longer, potentially the whole lifetime of a building, but as building users are not fixed in place, the impact will not be felt by the same group of individuals over time. Therefore, the yearly social returns were deemed to be an important finding, before projecting into the future.

8 The results for case building 3 are not directly comparable as they represent the returns on a retrofit scheme only, whereas case buildings 1 and 2 represent a new build and a fit-out of a shell-and-core scheme respectively, as outlined in the methods section.

9 Calculating how much of the outcomes would have happened anyway and removing this as deadweight could be considered as calculating how much of the outcomes were caused due to the building design rather than other factors.