Evaluating housing performance in relation to human behaviour: new challenges

The current context

In 2001, in a previous issue of this journal, Sir Andrew Derbyshire wrote:

How do we persuade the industry at large that POE [post-occupancy evaluation] is doable and worthwhile?

(Derbyshire, 2001, p. 82)

Nine years later, Building Performance Evaluation (BPE), in which POE plays a significant role, appears to be more established, with governments and their agencies recognizing the importance of finding out how well their building investments have performed against expectations. New schools, prisons, and hospitals are now being evaluated. Government offices are measured annually in order to benchmark and improve performance year on year. There is a growing archive of information on the performance of public and commercial buildings. But where is housing in this picture?

In the UK, 27% of all carbon dioxide (CO₂) emissions are related to housing (Department of Environment, Food and Rural Affairs (DEFRA), 2006). The situation is similar in other countries. In most developed countries with a mature building stock, there will be increasing policy recognition and pressure to reduce these carbon emissions through appropriate strategies for new housing and through large retrofit programmes for existing housing. Curiously, there are currently no UK government policy requirements for POE as part of its strategy to reduce carbon emissions in housing. This means that little real feedback exists on how housing is performing during occupation, which makes it difficult to ascertain whether targets are being achieved in reality, whether the design, procurement, and management strategies are actually working and whether occupants are actually reducing their demands and expectations (particularly in relation to so-called ‘efficiency gains’). Without validation at large and individual scales, there is a high risk that many strategies will not deliver what they promise. The Carbon Reduction in Buildings (CaRB) study in the UK (Firth et al., 2008; Lomas, 2010) and the Household Energy End-use Project (HEEP) study in New Zealand (Isaacs et al., 2010 and this issue) are at least the beginning of what could be a more substantial programme.

There has been relatively little published POE of housing compared with other sectors. There are many reasons for this, not the least of which is the difficulty of gaining a representative sample to benchmark in any particular housing development. Simply gaining access to people's homes, which are private by their nature, can present a real barrier. Traditionally, the evaluation of housing performance has consisted of either physical monitoring or occupancy satisfaction questionnaires, but quantitative and qualitative feedback are rarely related to each other as they span across the disciplines of building science and social science. The evaluation of user perceptions and behaviour in relation to building performance in housing is therefore an emerging research area.

This special issue explores the developing understanding of housing occupancy feedback in terms of occupants' expectations, perceptions, experiences and subsequent behaviour. It is particularly concerned with the research, policy and management implications of housing occupancy feedback as a multidisciplinary practice addressing climate change. The papers cover a wide range of investigations, but they all share one question in common: how can occupancy feedback help to produce better housing?

Helping occupants to help themselves

When considering the detailed design of buildings, van Dam, Bakker and van Hal's paper and Darby's paper both tackle the effectiveness or otherwise of providing occupants with energy feedback in the home. Van Dam et al. are concerned with the effectiveness of feedback from home energy monitors in reducing energy use over time. Their findings show that there is a distinct decrease in the level of energy savings originally made by occupants after only a few months. This is attributed to a lack of habit formation as well as poor design with overly complex interfaces. The authors show that where people adopt a regular habit of looking at energy monitors on a daily basis, they exhibit larger savings over time compared with others. An implication is to question the notion of mass-produced ‘one-size-fits-all’ home energy monitors and whether solely technological solutions (such as energy monitors) will actually achieve the desired results. It also suggests that a deeper understanding of the relationship between the user and these systems is needed.

Continuing the theme of the control interface between the user and smart meters, Darby discusses the level of ‘affordance’ that smart meters offer, i.e. their usability and effectiveness. In relation to proposals for advanced metering infrastructure, Darby points out that:

Taking control away from the customer cannot be relied upon to improve the situation: it may actually entrench and legitimize high-demand practices, disengaging customers from any need to consider and question them.

Like van Dam et al., Darby argues that effective forms of interface, feedback, narrative, and support need to be developed to reach more diverse populations and to reduce actual consumption. There is a concern that disadvantaged groups may suffer as a consequence of developments in metering and higher (time or use-based) tariffs, if such diversity is not taken into account. The inflated claims made by some about smart metering and home energy-management systems leading to energy reduction have not been substantiated, so more caution is needed when advocating it. The challenge is for the smart meter to prove itself in terms of developing a useful relationship with the user, not just through a screen or a bill, but through new forms of user engagement (understanding, expectation and behaviour) enabled by new design and media approaches.

Culture and normative behaviour

At the opposite end of the spectrum, Isaacs, Saville-Smith, Camilleri and Burrough have undertaken one of the largest occupant behaviour surveys in housing. The whole house is examined in this case, rather than just a single system. The results are surprising and show that New Zealanders are comfortable living with relatively low room temperatures. A cultural element behind the New Zealanders' fondness for colder rooms has been identified, as well as a lack of suitable heating appliances in their homes. Their work suggests that occupants' behaviour is driven by some kind of normative standard, or even a ‘pride’ factor. Further work is needed to identify exactly what the motivation is behind deliberately keeping rooms cooler than might be expected. The lack of widespread central heating also means that typically only one room is heated at a time, thus reflecting a more frugal set of expectations. Curiously, this approach to the provision of comfort that is localized on a room-by-room basis may be worth examining in places where central heating has been the norm.

Hendrickson and Wittman consider housing management structures and occupant participation. This approach is labelled Post-occupancy Assessment (POA) in an attempt to extend the POE concept into new territory. The study analyses how well three different housing developments are managed and the effect of this on waste recycling levels. The housing cooperative (co-housing) management practices that directly involved the occupants were found to engage occupants the most and led to more recycling and less waste generation. Although not all housing developments can be run cooperatively, there are clear lessons about the effectiveness of involving people directly in the local governance of their own housing developments, which induces more appropriate normative behaviour. The kind and shape of management structures can strongly influence people's commitment to using best practice for waste management both in their own homes and in the wider housing estate. The implication is that greater consideration of housing management practice is needed when evaluating housing performance. There are wider lessons for energy-management schemes.

Gill, Tierney, Pegg and Allan's paper demonstrates a method to account for the contribution of occupant behaviour to performance variation, which draws on Ajzen's Theory of Planned Behaviour (Ajzen, 1985). In their case study of a high-performance housing development, resource-conscious behaviours account for 51%, 37% and 11% of the variance in heat, electricity and water consumption respectively between the same type of dwellings. This shows the significant impact of behaviour on energy use over and above the physical design of dwellings. The user impact on consumption is not yet fully appreciated by either the mainstream housing supply or demand sides. Interestingly, occupants found ways to bypass some of the supposedly environmentally friendly features applied at the site. The reasons underpinning inhabitants' actions were due to partial understanding or because features were perceived to be unhelpful. This further shows that achieving long-lasting sustainable solutions must give primacy to user behaviour. User behaviour cannot be used as an excuse by designers for performance deficits or unintended consequences, but must be understood and influenced appropriately.

Williamson, Soebarto and Radford's paper is the most radical in its challenge to regulatory and normative practice. The authors suggest that occupants' individual aspirations and living requirements are often out of step with regulatory demands, but in some cases a building design that fails to meet regulation does not necessarily diminish its ability to perform as a low-energy building. Regulations and standards can actually ‘fail’ peoples' attempts to be more energy efficient precisely because these attempts do not align with conventions at the individual building level. The occupants of five award-winning houses studied, which fail key American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) standards for comfort, are found to prefer living with the non-conforming temperature differences as part of their chosen lifestyle and achieve low energy use this way. The authors argue that the occupants of the award-winning houses have chosen to live with the comfort performance of their homes as part of a wider engagement with the design principles which cannot be standardized.

All four of these papers (Isaacs et al., Hendrickson and Wittman, Gill et al. and Williamson et al.) deal with human behaviour in homes in relation to cultural norms at one level and the consequences of regulatory practice at another. Attitudes and expectations are a complicated mixture of normative behaviour, and assertions of personal beliefs derived from empirical experience and aspirational goals based on what is perceived to be achievable (Chappells and Shove, 2005). Any of these factors can be easily unbalanced if the wrong messages are being sent to the person concerned. In each of these cases the wrong message is dispatched by existing or proposed systems which thus need reforming to take account of the diversity of occupants' lifestyles and aspirations and engage more fully with managing expectations and behaviour (Shove et al., 2008; Cole et al., 2010).

New directions for housing evaluation

Gupta and Chandiwala's paper together with Stevenson and Rijal's both reflect the need for evaluation methods that can capture user behaviour in relation to housing performance. This begins to address the lack of discussion concerning evaluation methods in the domestic sector. Both papers build on the evaluation process outlined by Leaman et al., and go on to examine how effective are the methods used in their own studies. The first explores how these techniques can inform the design and specification of retrofit measures for existing housing prior to the design process of a particular project. The second study looks at a prototype new-build ‘zero-carbon’ demonstration house. In this way, the evaluation process in both cases is employed as a form of direct feedback into the design process for a housing project. Both papers also highlight the need for qualitative social analysis related to the physical performance of housing. Gupta and Chandiwala use POE methods to fine-tune the design for comfort and energy efficiency. Stevenson and Rijal explore new evaluation methods for housing, such as video analysis, activity logging, and analysis of information provision, all of which focus on users' relationships with control interfaces. They look at the degree of ‘interactive adaptability’ provided by such interfaces. This relates to the affordances which help users to more actively manage and control their environment. These design–interrogation techniques could help answer the plea made by Darby and van Dam et al. for energy-feedback systems design that takes more account of users' individual behaviour and needs.

A ‘new professionalism’

Leaman, Stevenson and Bordass reflect on their experiences within the non-domestic sector. They suggest that there is significant crossover into the domestic sector, while recognizing that domestic evaluation has its own challenges. The conclusion calls for government to take action to break the numerous structural deadlocks that prevent building evaluations from being properly funded. There is also a plea for:

a new professionalism that engages routinely with outcomes and consequences, and places more emphasis on integration, communication and applied knowledge.

Housing developers and their design teams are urged to explore how to integrate the lessons learnt from housing evaluation into their knowledge-management systems. The traditionally informal communication that takes place between customer services, maintenance, and development departments needs to be more articulated and recorded for reference so that mistakes identified from the evaluation are not repeated elsewhere. This could be captured in a domestic variant of the ‘Soft Landings’ approach first developed by Mark Way and others.

Many of the papers present facets of this ‘new professionalism’ with research that is directly engaged with the housing industry to help inform a new generation of housing. There is a danger, however, that too much emphasis will be placed on capital-intensive and complex technologies at the expense of passive strategies in the quest for low-carbon or ‘zero-carbon’ housing. This is clearly shown in Stevenson and Rijal's paper, where a combination of complex building envelope and complex energy technologies clearly defeat the developer's aim for a ‘zero-carbon’ prototype. The message is: simplify housing design where appropriate, and make it understandable for occupants.

Vale and Vale provide a challenging perspective on the purpose of housing occupancy feedback. As architects, educators and researchers who have been involved with sustainable housing for 40 years, they are in a position to examine critically how much energy performance has improved and whether the amount of resource consumption has reduced. The answer is not much, and it is getting worse due to a wicked combination of Jevon's Paradox (where occupants ‘take back’ their savings in terms of increased comfort or amenity) and population increase. They remind readers of important ideas and lessons from the past in order to avoid repeating mistakes as well as finding new potency in these ideas. Significantly, they suggest that POE needs to be repositioned. They argue that it is insufficient to ask questions about the ways that a building performs, particularly if it is a dwelling. To make a real impact, it is necessary to examine how the household performs and compare individual footprints in terms of resource consumption. This would lead to a more intelligent basis for comparison and create appropriate norms with wider social engagement to address the critical need for less consumption. It is not enough to presume that the information from ‘smart metering’ will encourage people to reduce their energy consumption any more than a car speedometer will reduce speeding, unless the speed limit is made clear along with the severe consequences of breaking it. Their recommendation is to use technologies that not only reveal real system limits, but also actually impose limits at the micro-level, as this is a proven means of changing people's values and behaviours. Although few can live in an autonomous house, there are many ways in which energy profligate actions can be limited. Certainly more thought and discussion needs to be directed at the implications and means of doing so.

Key lessons for housing performance

People use energy, not buildings (Janda, 2009). It is often the human element that confounds the energy predictions. This can be due to a lack of training or an inability to use the control systems on offer (Stevenson and Rijal, Gill et al.), or simply choosing to live a different lifestyle to that assumed by the model (Williamson et al.). When one allows for human attitudes, perceptions and motivations, human behaviour becomes a challenging area of investigation for building performance evaluation, and one that is often overlooked for this reason. Some of the key points from this special issue are:

•	Design affordances in housing need to be tested, evaluated, and improved in order to empower the user to take control of their environment and reduce energy use in a manner appropriate to their needs.
•	Housing management is a dimension that needs to be evaluated as part of POE. Management practices should be engaging more with the inhabitants and involving them more directly within the decision-making processes about resource use when possible.
•	Regulations and standards for energy efficiency and comfort are blunt instruments, often too general for specific circumstances. In some cases they do not allow for users' individual needs. Recognition is needed that a diversity of inhabitants and comfort scenarios exists and that these require accommodation within governance structures (specifically, within regulations and standards). In particular, different and changing cultural normative behaviour in relation to thermal comfort needs to be accounted for. This raises the larger question of whether buildings should be regulated or whether the focus should shift to inhabitants.
•	Feedback on energy use provided to the occupant within their home needs to be meaningful, consequential and habit-forming if it is to make a sustained difference.
•	Evaluation methods should identify the reasons why users behave the way they do and focus on the ‘interactive adaptivity’ provided by key control interfaces.
•	A POE on any existing dwelling should be carried out before designing any housing retrofit scheme in order to inform the design proposals more fully.

What next?

Housing occupants can use three or more times as much energy for heating as their neighbour, while living in exactly the same type of home (Gram-Hanssen, 2010). This suggests that even if the building fabric is robust and well insulated with suitable thermal mass, and the home has an efficient energy source, it will still be the inhabitant who ultimately determines how energy efficient a home will be. Even if the amount of energy consumed by the building for heating and cooling space is low, occupants will still be free to use as much energy as they like for appliances and hot water systems. Occupants need to have their energy-profligate ways challenged where necessary, but they should also be offered better control over their own comfort conditions through improved usability. This requires a better understanding of user expectations, attitudes, perceptions and behaviour, with more research targeted in this area. More sophisticated evaluation strategies that interrelate human factors directly with the physical performance of housing also need to be developed.

Given the profound changes that housing design is currently undergoing to meet the tough low-carbon agenda set by governments around the world, occupants need better guidance and vastly improved systems. A successful approach will allow inhabitants to feel empowered, rather than guilty, although reality checks provided by individual footprint and carbon taxes may be essential to demonstrate and reinforce the consequences of their actions.

Inhabitants are recognized as one of the best instruments for measuring housing performance, even if they are hard to calibrate (Cole et al., 2008), and their feedback can quickly demonstrate why a technology does or does not work. Vital feedback provided by the occupants as they inhabit their homes can be fed back into improving the modelling and design of housing as well its management and maintenance in order to reduce carbon emissions. This requires a formalized briefing, commissioning and feedback protocol, such as ‘Soft Landings’, to be developed for the domestic sector. This will help to ensure that these lessons are captured and fed back to the developers and the designers.

The biggest challenge of all is how to ensure that housing occupancy feedback becomes embedded and routine rather than restricted to demonstration or research projects. Should the ‘new professionalism’ be voluntary or should occupancy feedback be an imposed legislative requirement? There is a legitimate concern that over-regulation will simply result in gratuitous ‘tick-box’ culture that prevents an intelligent understanding of feedback. On the other hand, it is clear that voluntary occupancy feedback in housing has been languishing for a long time and perhaps the greater challenge is to develop more responsive regulatory processes.