Abstract
Although there is increasing interest in building performance, the people who procure, design and construct buildings seldom engage closely with the performance of the buildings they have created. This paper outlines the results of 14 case studies where designers and their clients used one or more techniques chosen from a portfolio of ten to evaluate their buildings or processes at any stage in the life cycle of a project. It is revealed that considerable value could be obtained for relatively low effort, helping to improve both the performance of the building concerned and the skills and insights of the participants; and that there was value in using established techniques that were robust, cost-effective and had benchmarks available where appropriate. It proved easier to undertake a survey than to get people together to discuss their experience. However, after the procurement process had started, it proved difficult to incorporate feedback, because everyone was already committed to a particular mode of operation.
En dépit d'un intérêt croissant pour les performances des bâtiments, tous ceux qui approvisionnent, conçoivent et construisent des bâtiments s'engagent rarement en ce qui concerne les performances des bâtiments qu'ils ont créés. Cet article présente succinctement les résultats de 14 études de cas se rapportant à des concepteurs et à leurs clients qui ont utilisé une ou plusieurs techniques sélectionnées dans un catalogue de dix techniques pour évaluer leurs bâtiments ou leurs processus à n'importe quel stage du cycle de vie d'un projet. Il apparaît que l'on pourrait obtenir une valeur considérable pour un effort relativement modéré, qui aiderait à améliorer tant les performances des bâtiments concernés que les compétences et les connaissances approfondies des participants et aussi qu'il était intéressant d'utiliser des techniques établies robustes, rentables et comportant des repères le cas échéant. Il s'est avéré plus facile d'entreprendre une étude que de rassembler des personnes pour débattre de leur expérience. Cependant, après le démarrage du processus d'approvisionnement, il est apparu difficile d'intégrer le retour d'information, chacun étant déjà engagé dans un mode particulier de fonctionnement.
Introduction
This paper outlines the results of case studies in a UK research project that ran from 2001 to 2004 and considered how to implement a finding from the Post-occupancy Review of Buildings and their Engineering (Probe) studies (Lorch, Citation2001) that post-occupancy evaluation (POE) needed to become commonplace for the construction industry and its clients. The project put together a portfolio of techniques, as described in the first paper of this series (Bordass and Leaman, Citation2005). The techniques were tested by a User Group of designers and clients in the case studies described here.
Portfolio of feedback techniques
Designers, builders and sometimes even their clients can be slow to learn from their experience and the performance of their completed projects. To improve the situation, feed-through from design and construction into operation, and feedback from and post-occupancy evaluation (POE) of the product all need to become routine. Attempts to produce a single standard POE method have been unsuccessful (e.g. Federal Facilities Council, Citation2001). A portfolio approach was therefore suggested (Bordass and Leaman, Citation2005), including both special-purpose techniques and more general improvements to the procurement process, e.g. Soft Landings (Way and Bordass, Citation2005).
This paper outlines the results of tests of a portfolioFootnote1 of feedback techniques. This currently contains ten general-purpose techniques developed in the UK, but is potentially infinitely expandable. The techniques were chosen for their applicability to a wide variety of building types. Where possible, they also had good track records, preferably with sufficient experience in their use to have created clear guidance on how best to do them and, where appropriate, robust statistics on their results so that benchmarks could be derived where appropriate.
The portfolio is stored on a database, with associated links to other resources and documents. A specially written translator program can load information from this and other databases onto the website (http://www.usablebuildings.co.uk/fp/index.html), which includes the following:
information on the techniques including their attributes and applicability at various stages throughout the life cycle of a building and a project | |||||
a standard style of user interface that can be adapted to different functions | |||||
a set of links to other resources and documents | |||||
Audit: including quantitative technical assessments, at present the CIBSE TM22 energy survey and assessment method (CIBSE, Citation1999), which was used in the Probe series of published POEs | |||||
Discussion: techniques that get people together to discuss what they are about to do, what they are doing or what they have done | |||||
Questionnaire: for occupants or to assist discussion | |||||
Packages: including Probe with its technical, energy and occupant surveys | |||||
Process: including Soft Landings (Way and Bordass, Citation2005) | |||||
Feedback user group
The User Group had 12 core members (nine designers, representing small, medium and large architectural, engineering and multidisciplinary practices, two clients and a facilities manager) who applied the techniques in their case studies. Other members received notes of the meetings and attended some of them, but did not provide any case studies. The Group proved to be an effective vehicle for promoting and discussing feedback techniques and results. In the course of the project, it met first to raise awareness, then to confirm plans, and then on several occasions to review results and opinions as they emerged. The group continues to meet, facilitated by the Usable Buildings Trust.
Initially, each member of the Group was going to evaluate one or more techniques in the Feedback Portfolio, giving a minimum of ten completed trials in all. In practice, the situation was more complex. Some members were new to feedback, others had feedback activities in place or under development, and some had participated in parallel feedback-related research projects. Consequently, there was valuable interaction between experiences inside and outside the project.
Where necessary, the research team helped users individually in choosing techniques, finding support if they needed it, and assisting with undertaking and reviewing activities as necessary. User Group members were able to report their case study findings in whatever manner they wished – the idea of a standard format was ultimately rejected because it was felt that a diversity of approaches would be more rewarding; and so it proved.
Overview of the feedback case studies
The case studies covered the application of feedback in 11 of the 12 organizations. They were concerned particularly (though not exclusively) with using the ten general-purpose techniques in the Portfolio at various stages in the life cycles of User Group members' buildings and projects. They covered a range of non-domestic building types and included new construction, phased construction, extensions, refurbishments and refits. Details are available on the website, where a portfolio of feedback results is also under construction, including those from the case studies. The full report of all the case studies may also be viewed and downloaded here.
Most User Group members were familiar with feedback in some form, but not with all the techniques in the Portfolio. All contributed experience to the group and drew things out of it. Their case studies were of two main kinds:
overview of an organization's feedback activities | |||||
account of the application of one or more techniques from the Portfolio to one or more projects | |||||
shows the coverage of the feedback techniques by the case studies where rows show the techniques, classified as outlined above, and columns show the organizations, in alphabetical order.
Figure 1 Classification of feedback user group case studies undertaken
User Group members were free to choose the techniques they preferred, from the Portfolio and elsewhere. The shading of the cells in shows how they were applied:
Black cells indicate where a technique was used without modification in activities that would not have been undertaken had it not been for the User Group's existence. | |||||
Cross-hatched cells show the use of a closely related but not identical technique. | |||||
Cells with horizontal bars show parallel development. For example, Arup was already using the TM22 energy assessment on related projects, RMJM initiated the Soft Landings idea (Way and Bordass, Citation2005) and six member firms were on the Soft Landings research team. | |||||
Vertical bars show organizations that had prior experience of or exposure to a technique, e.g. through involvement in a Probe study. | |||||
Diagonal bars show activities that could not be completed within the time available. | |||||
White cells indicate no activity, or at least not within those parts of the organizations represented on the User Group and at the time the case studies were taking place. | |||||
Figure 2 The case studies, classified by sector and stage in the life cycle
To the right in are stages in the life cycle in which the feedback was done and to which it was immediately relevant. For example, the prime purpose of Newnet's workplace survey of existing offices was to inform briefing and design of new offices to which the same staff was going to move.
The black cells in indicate the specific relevance of the case study exercises; the cells with horizontal lines are for more general processes applied within the organization concerned.
For every study completed, more than one did not happen because of difficulties in timing or due to a User Group member not being able to convince clients and/or colleagues that they should participate. Approval proved to be particularly difficult where a new process had to be applied (such a radical change is best agreed at the start of a new project, before contracts are signed), or where groups of people had to be brought together. Getting permission for a technical or occupant survey was generally easier, provided it was not expensive or disruptive – though even small sums for survey consultants often proved surprisingly difficult to raise.
Arup case studies
Arup, a large, leading international firm of engineers has been active in feedback for many years. Arup's research and development department has been responding to queries and distilling guidance into practice notes. In the past, its feedback activities and database on building operational performance were assisted – directly and indirectly – by commissions and reports from the fuel industries, government departments and the Building Research Establishment. During the 1990s, these sources had more or less dried up (an unfortunate by-product of privatization), and substitutes had to be found.
In 1999, Arup set up its ‘Building Feedback Initiative', which attempted to maximize the value from feedback undertaken for clients, from Arup's internal research and development investigation budget and through post-project reviews under its ISO 9001 Quality Assurance scheme. The feedback information collected is put on Arup's intranet, which not only contains technical resources, but also identifies who knows what. Getting hold of the right person is often the most efficient source of feedback!
Specific examples in Arup's case study included the following:
Arup's ‘Workplace Performance Methodology’, which includes Probe-like components of an energy survey, a BUS Occupant Survey, and supplementary environmental monitoring where necessary. In the course of the project, Arup was also completing feedback work (including occupant, energy and environmental surveys) on its own offices in Solihull, a pre-let building designed on sustainable principles, subject to market constraints. Lessons were applied in fine-tuning the building, in planning an extension to it and in projects for other clients that were applying similar design approaches. | |||||
Work for the developer Stanhope plc on the performance of its buildings at Chiswick Park to assist fine-tuning, to improve subsequent buildings on the site and to inform other developments, including its immediate successor at Hemel Hempstead. | |||||
Work for British Land, a major property company on understanding the energy performance of its properties. | |||||
Atkins case studies
Atkins, which is a large practice that has been broadening out from engineering to a wide range of design, support and management skills, focused on the handover POE work it had already started in primary schools. The process had been informed by BRE's guidelines on POE in the first year of occupancy (Jaunzens et al., Citation2003), but it was simplified for these smaller buildings. The technical lessons were informative, in particular the following:
A tendency for new buildings, designed to be low energy, to use less heating fuel owing to better insulation levels (but not as little as the designers had anticipated), but to consume significantly more electricity than their simpler predecessors. This was partly, though not solely, the effect of increasing electronic equipment. Other causes included building services, a switch from gas to electric catering equipment, and a tendency for equipment to default to ON whether it was needed or not. | |||||
More sophisticated controls installed to save energy can often fail to work properly, irritate the users or cause the users to adopt bad habits. Atkins reported that better understanding and feedback of equipment and user behaviour could rapidly improve design, improving both energy performance and occupant satisfaction. | |||||
Unexpected problems in heat distribution from low surface temperature heating, which is often installed now for safety reasons. | |||||
Design strategies for ventilation that conflicted with occupant behaviour. For example, classroom ventilation had been designed to work best with the doors closed. In practice, however, teachers insisted on leaving the doors open because culturally a closed door was seen as a signal that they were having difficulty keeping their class in order. | |||||
better handovers and occupant awareness | |||||
managing expectations with the new system: once a representative of the designers visits the site and monitors performance, occupants also expect swift remedial action | |||||
better support from the design and building team to deal with the problems raised | |||||
low response rate: established surveys have often found ways of getting high responses | |||||
difficulties in interpreting responses to questions where there is no history of reliable use or not enough results to allow robust benchmarks to have become established | |||||
Broadway Malyan (BM) case study
The architects BM have been involved in a number of feedback exercises including the DQIs (Gann and Whyte, Citation2003). This time, they decided to try out a technique new to them – the AMA Workware Toolkit – to review workplace performance and staff attitudes in an office they had designed. The purpose was to fine-tune BM's design of a new, larger office for the same company. With a small budget, Alexi Marmot Associates's (AMA) involvement was restricted to providing and analyzing self-completion questionnaires. The survey was administered by BM to guidelines set by AMA, including an initial discussion with the client about what items the survey should include. BM reported the following:
survey revealed that satisfaction and productivity in the existing building were already good; however, scope for improvement in the new building was also identified | |||||
survey exposed issues that had not emerged during the earlier briefing process with the client, e.g. the importance of the reception area to staff as well as visitors; a shortage of meeting rooms for seven to 12 people, and requirements for showers and cycle parking | |||||
benchmarking was helpful, but BM feared it could also be misleading if a building was not being compared with an appropriate reference dataset | |||||
BM also commented as follows:
As well as addressing problems, it was important to carry through into the new design aspects of the existing building that the staff really liked, e.g. its light and airy ambience. | |||||
Staff perceptions of the internal environment are based on long exposures, often at fixed workstations. There is no way that visitors – or even monitoring instruments in the officially prescribed representative positions – could judge this experience accurately. Perceived levels of control can also have a significant affect on occupant satisfaction. | |||||
Although designers are often unaware of problems that occupants experience, they may also fret unnecessarily about things that users actually find acceptable. BM had been concerned about its use of sliding horizontal patio doors for ventilation in some parts of the existing building (e.g. in relation to security, weather protection and draughts), but these aspects did not trouble the occupants, who understood how patio doors operated and could live with their limitations. This sort of feedback helps designers focus their future efforts on the things that really matter. | |||||
Buro Happold (BH) case studies
A life-cycle approach is increasingly important to engineers BH. By understanding how their buildings perform, they can improve sustainability through reducing waste and pollution (e.g. through energy efficiency), increase occupant satisfaction and productivity, and confirm the potential for reuse and recycling of existing buildings and structures. While primarily concerned with the ‘hard’ issues of engineering, BH increasingly recognizes the importance of the ‘softer’ issues of how people react and go about their work.
BH undertakes its own feedback and has been involved in several collaborative initiatives over recent years, including three techniques in the Portfolio: Soft Landings, Learning from Experience and the CIC DQIs. It also undertakes customer satisfaction interviews and has been developing its own handover and aftercare systems. The case studies reported were as follows:
A follow-through exercise on an energy-efficient office building, with a 12-month aftercare service and two further years of review. The building performed well, but fell short of its ambitious design targets. Some causes were identified, in particular problems with airtightness and controls (which have been tackled), and over-cooling of an equipment room (which the occupants preferred to continue). The outcomes include improved energy performance for the building and better design, prediction and benchmarking skills for BH. The water use of the building was exceptionally low, which set a benchmark for future projects. | |||||
‘Hindsight review’ of their work on the recent extension to the Natural History Museum, London, undertaken in the parallel ‘Learning from Experience’ research project,Footnote2 both as a test and to inform the design of the forthcoming Phase II. Records were reviewed, eight people were interviewed and the results compiled into a ‘learning history’ to record and disseminate the lessons learnt. The exercise confirmed the value of a structured process to tap into team members' tacit knowledge, and was thought to have been more effective than a workshop in extracting lessons learned. Ideally, BH would also have liked a workshop to discuss the issues raised and release more tacit knowledge held collectively by the team. | |||||
University of Cambridge case study with Edward Cullinan Architects (ECA)
The University of Cambridge is committed to follow-through and feedback. Its Mathematics Building by ECA was the subject of the final published Probe survey (Cohen et al., Citation2002). The new case study was of the Higher Education Design Quality Forum's (HEDQF's) post-occupancy review methodFootnote3 at the Faculty of Divinity. POE is part of the Higher Education Funding Council for England's (HEFCE) requirements for all projects it funds. HEFCE does not insist on a specific method, but HEDQF satisfies its requirements. While the Divinity building was not HEFCE funded, Cambridge decided to use it for its first trial of HEDQF, which was facilitated by one of its own project managers who had attended the one-day training course by de Montfort University.
The HEDQF method is designed to be done quickly – initially in one or two days, but in practice one day has proved sufficient. The facilitator then has to write it up. The method consists of four ‘fora’ – two in the morning and two in the afternoon. All four are attended by key figures (e.g. the client, the architect, the project manager and the facilitator), whilst others join the session appropriate to the matters in hand. The four standard forum topics are: context and design; construction and cost; space and management; and environment and sustainability.
The process was designed for people who might never have participated in an exercise such as this before and so could well feel threatened – particularly because it was at the end of the project when they might be criticized, but for the most part would have no resources to undertake further action. The facilitators are therefore trained to avoid confrontation and to concentrate on the positives, not dwell on the problems. However, at least on this occasion, problems were not glossed over either.
In general:
the conclusions on process, product, cost and performance were positive | |||||
the team had collaborated well and listened carefully | |||||
the building had met the brief of the user client; and the design had been instrumental in attracting more students, users and funding. Generous public spaces were part of this, but the Estates Department felt the building could have been more adaptable and space-efficient | |||||
there was scope for improvement in handovers, which should be seen as a process (as in Soft Landings) and not a single event | |||||
design principles and solutions for storm water drainage needed revisiting to ensure sufficient fail-safe redundancy (climate change will also affect design requirements) | |||||
the external motorized venetian blinds did an excellent job of shading – when they worked; however, they had been dogged with unreliability. This needed careful attention on future projects, for which the general conclusion was to keep things simple and to avoid moving parts, particularly if exposed to the weather | |||||
the forums were very valuable, and one day was sufficient to fit in all four of them | |||||
it was important for individuals actually involved in the project to attend (one firm sent someone else – but there is no substitute for the people who had direct experience) | |||||
more user representatives should have attended: this (and other aspects) would have assisted had the review been held in the building concerned, not in the Estates Office | |||||
a written summary was very useful | |||||
Department of Health (DoH) case study with Alexi Marmot Associates (AMA)
In 2000, the DoH developed a new accommodation strategy to improve performance and make better use of space. In this case study, the recommendations had been used to develop a brief for internal alterations to one floor of an elderly office building, and tested in applying it. The AMA Workware Toolkit was used to evaluate conditions and user perceptions and requirements before changes were made. It was used again afterwards to review the product, evaluate the success of the strategy and the design, and identify aspects needing fine-tuning. The specific tools used were to analyse the physical provision of space; how it was actually being used, and occupants' opinions of both the product and the change process.
The main alterations were to space planning, partitions and finishes. Much of the furniture was reused and the building services – although far from optimal – were little altered. Important ambitions were to change the ‘feel’ and unity of the space, the provision and management of meeting rooms, and the balance between individual, shared and open-plan offices. Specific comparisons were made between the new and old offices and the database of results from other organizations. The results largely verified the worth of the policy and the effectiveness of the design, but also identified details requiring attention. In particular:
the space had been much better integrated by minor but important changes to circulation routes | |||||
the elimination of most enclosed offices in favour of open areas, meeting and quiet rooms had been largely successful | |||||
overall satisfaction levels increased significantly, with very few people rating things as worse | |||||
people greatly welcomed the improved image, space, shared areas and meeting rooms | |||||
the improved communication was welcomed, but at a price of privacy and confidentiality | |||||
some occupants felt that they had not been consulted fully | |||||
The case study verified the effectiveness of the proposed accommodation strategy, included benchmark comparisons to show that the changes had been worthwhile, and revealed areas for fine-tuning the process, the product and of user awareness. It showed how feedback could be used to build organizational and individual confidence in a change programme, and to make sure that any investments were made to best effect.
Feilden Clegg Bradley Architects (FCBA) case studies
FCBA is committed to producing buildings that meet client needs and have a particular interest in sustainability. They are committed to feedback, e.g. having initiated the HEDQF system in the Portfolio and involved in tests of the Schoolworks POE technique which is being developed for schools.Footnote4 They believe that architects frequently fail to learn straightforward lessons from completed projects and end up repeating mistakes that could easily be avoided.
FCBA's case studies included examples of feedback in educational projects past, present and future; focusing on procurement routes, occupant satisfaction and energy performance. FCBA was on the Soft Landings research team and had hoped to include a case study of an application. However, since Soft Landings affects the whole procurement process, lead times to acceptance were long; and a proper case study would also have required a lengthy review of the impact over the life of a project, including the aftercare period.
FCBA is working on a student village at Queen Mary College, London, which had already benefited from some feedback from Phase 1. After being introduced to the technique through the feedback User Group, FCBA also set up some Learning from Experience sessions:
The first four-hour session in January 2004 was attended by the client and the design team. It reviewed their experience on Phase 1, primarily in learning lessons for the briefing, design and procurement issues on Phase 3, for which outline design proposals were being prepared. | |||||
A second session was undertaken in early 2005 after the completion of Phase 2. This time it included the contractor as well, reviewed both the process and the product, and extracted more lessons for Phase 3. | |||||
A POE of Phase 2 is also planned, after a year in operation (i.e. in 2005–06).
For the United Learning Trust, which is sponsoring at least six City Academies, FCBA has been working on Northampton Academy and a second project in London. Discussions are continuing on how the Trust and their design and building teams can learn from the exchange of information and feedback, and use standard assessment systems across their projects.
John Packer Associates (JPA) case study
The building services engineers JPA were the smallest firm in the User Group. They had intended to apply a variety of techniques in the portfolio (the HOBO Handover Protocol referred to in Jaunzens et al. Citation(2003), an occupant survey; and the TM22 energy survey (CIBSE, Citation1999)) to the refurbishment of offices for a government agency, but this proved impossible within the programme. Instead, they brought together experience from other projects.
The main one was the use of the CIC DQIs (Gann and Whyte, Citation2003) in the proposed refurbishment of a library under CIC's Trailblazer scheme. The DQI questionnaire was first applied to the existing library to find out what staff thought about it; and then (in the same session) to client and user perceptions of the design team's aspirations for the refurbishment. The staff's opinions about the effect of the changes were particularly illuminating: while they appreciated that space, construction, access and social integration were constrained by the existing structure and its location, they anticipated radical improvements in the use potential of the building, its engineering systems and its internal environment. This was encouraging, but also revealed that expectations would need to be managed carefully, given the constraints of the existing building and a limited budget.
JPA found that the DQI process was a valuable spur to discussion and reflection. However, they had some logistical problems, not surprisingly for a relatively new system. In particular, they did not find it easy to select individuals to fill in the DQI questionnaire; and were not sure of the validity of averaged results, which had been collected from different user groups which tended to work in different parts of the building, which had very different characteristics. Clearly more detail would have been helpful (some was provided, and DQI technique now provides more), plus opportunities to discuss the results. Another problem was how to involve members of the public meaningfully.
JPA had also made use of the CIBSE Citation(1999) TM22 energy assessment technique (originally developed to improve the speed, quality and consistency of energy assessment and reporting for occupied buildings with energy data available) at the design stage to help focus its computer modelling exercises and to manage the results. A greater transparency between design estimates and in-use outcomes is important to effective feedback and could become critical with the implementation in 2006–09 of a European Union Directive that requires energy certification of buildings in the European Union (European Community, Citation2003). JPA also drew attention to potential linkages between the BREEAM method of environmental assessment,Footnote5 which now includes a post-completion check. They saw exciting prospects in integrating TM22 and other techniques from the feedback portfolio into the process.
Reid Architecture (RA) case study
RA has undertaken design reviews and client satisfaction surveys for many years. Leading up to the current project, they had been getting more involved with POE, with a director leading a working group of the Construction Research and Innovation Strategy Panel (CRISP), and one of their architects taking a postgraduate degree that included POEs of three RA buildings (Carmona-Andreu and Oreszczyn, Citation2004). As a result, RA was developing a new feedback procedure to run from briefing and feasibility to design reviews, and finishing with checks at the end of the first year of occupancy. This was focused particularly on its requirements as an architectural practice, while Soft Landings is aimed at whole project teams.
RA's offices in London were refurbished in 2001 to bring together all staff in one location. A new entrance and lift tower gave the building presence and visibility. Designing for themselves allowed the office lighting to be more decorative and less bright than a standard commercial installation; and an innovative scheme of automated natural ventilation to be adopted, with summer cooling using chilled beams.Footnote6 Using CIBSE TM22, the energy use of the office was reasonable but higher than anticipated for various reasons including the following:
Optimistic expectations: these were adjusted using the recently developed ECON 19 tailored benchmarking scheme.Footnote7 Like JPA and others, RA understood the potential for using TM22 and tailored benchmarks at the design stage – allowing targets to be set that take better account of the actual design and likely use of a building. | |||||
High air leakage through the dampers of the automated natural ventilation system. | |||||
Poorly circuited and labelled lighting controls and decorative lighting on 24 hours. | |||||
Heating, cooling and ventilation controls that were difficult to understand. | |||||
RA's DQIs were administered using the online questionnaire (the earlier JPA case study had used the paper version). This showed that the building was a good all-rounder. In operating the DQI system, RA had some problems with the output, in particular:
the name of the building was lost | |||||
all the occupant's comments were lost | |||||
not all the questions were easy to understand | |||||
nobody bothered to use the weighting system | |||||
no benchmarks were available to help interpret the results | |||||
The results showed that the building was well liked and the only aspects worse than the benchmark were noise levels (the offices are densely occupied) and control over heating and cooling. The perceived productivity increase attributable to the building and its internal environment averaged an excellent 9.4% (the average in the BUS database is –3%). Productivity gains alone therefore easily justified the business benefits of the new office, which had many other advantages, e.g. image value in attracting clients. The classified comments indicated a few problem areas, including storage, automated blinds and comfort in really hot weather. Positive comments were obtained on the open plan (with noise levels regarded as acceptable) and on the noiseless ventilation and air-conditioning system.
RMJM case studies
The multiskilled design organization RMJM developed an ad hoc process for follow-through and aftercare. This later evolved into the Soft Landings system (Way and Bordass, Citation2005) in the Feedback Portfolio. Soft Landings covers the whole procurement process and provides a natural route for feedback. RMJM's initial experience was as follows:
Soft Landings works best where everybody subscribes to it as soon as they join the project. | |||||
Expectations need to be managed: the improvements must not be oversold. | |||||
Occupiers still need to take proper responsibility for what is now their building, and set up effective management and maintenance systems. The aftercare service is to help them get these right and deal with any emerging problems – it must not be used as a prop. | |||||
The weak links in the chain tended to be equipment suppliers (who could have less dedication to the client than the design and building team), and sometimes the building operators, who may be tempted to rely too much on the designers and builders and not take proper ownership of the operation, maintenance, information and training issues that rightly belong to them. | |||||
Project Darwin, initiated within RMJM but now offered as a consultancy service, aims to embed feedback more deeply in an organization. RMJM has a website with feedback information and design guides, regular feedback seminars, and it undertakes technical reviews of buildings shortly before handover with written and photographic records. Specific projects are also earmarked for a technical review two or three years afterwards, with an occupant survey where the occupier permits. RMJM also has a system of performance reviews of organizations and products.
RMJM included an example of an occupant survey at Ellersly House, an office building in Edinburgh, undertaken before embarking on a proposed refurbishment. Occupant satisfaction with the design and the furniture was already good, but there were complaints of poor interpersonal communication and scope for improvement in perceived productivity. The building was generally very quiet, but paradoxically (but not unusually, as there was little masking noise) this led to distraction from audible conversations. Designers and management were looking for creative new layouts that could be more stimulating and encourage informal communication whilst also being acoustically better. Staff interviews about the proposed changes were planned, followed if possible by a monitored pilot project before full application.
University of Oxford case studies
The University of Oxford illustrated a variety of opportunities for feedback, both during the life cycle of a project and in initial and routine operation of a large stock of buildings. It has also used feedback experience from estates staff and helpdesk records to compile a Design Philosophy document for building services that is used when developing specific client briefs and issued to design and building teams. A companion document is now being prepared on general building work.
One case study illustrated the use of structured feedback from the first phase of the Manor Road multi-faculty building to inform the second phase, started several years later and completed in autumn 2004. When concerns were voiced about technical performance and occupant satisfaction in Phase 1, a POE was instigated using Probe techniques (Lorch. Citation2001). The design of Phase 2 was already well advanced and planning permission was obtained, so alterations were only possible within the general appearance and envelope of the building and the agreed structural system.
The feedback exercise had a major impact on the planning and design of Phase 2, improving Phase 1, and future work by the university and the designers. It revealed several strategic problems; including widespread unhappiness with the internal offices that faced onto the main processional stair. The larger Phase 2 had intended to replicate this arrangement three times over. Instead, its internal planning was radically revised within the same footprint and structural system to incorporate a single main stairwell, with teaching rooms rather than offices beside it.
At a more detailed level, Phase 1's windows had difficulties with operability and glare, so Phase 2's were significantly modified. There had also been problems with furniture and lighting, so a test room was equipped to demonstrate and evaluate options. The new shading was not only more effective, but also gave better views out. Subject to successful experience in use in Phase 2 (and the initial feedback from the completed building has been good), this and other improvements will also be retrofitted to Phase 1.
In another case study, the university is putting the use of feedback and the refinement of designs from phase to phase into practice at its new science park at Begbroke. A generic building concept has been developed using a rationalized kit of readily available parts that can be obtained easily and built by anyone as the need for development arises. Review with other designers, contractors and potential users has led to further rationalization. In particular, the strategy has been shaken down into two interconnected building types. The main areas contain simple, adaptable, clear open spaces capable of many uses, connected to more complex spaces containing entrances, receptions, lifts, stairs and common services. As buildings are completed, the university will review performance in use and feed this experience through into modifications of the concept and the associated components. Initial feedback from the first phase suggests that the entrance areas may need to become more generous.
Review of the techniques used
The case studies support the Portfolio approach to feedback and that it is possible to get started at any stage in the life cycle of a project. People selected techniques they felt were best suited to the work in hand. Of the five groups in the Techniques Portfolio the following was found:
Occupant surveys were very widely used. The well-established questionnaire techniques in the Portfolio were clear, concise and easy to fill in, achieving high response rates. Their costs have dropped a lot over recent years owing to this refinement and largely to automated analysis. Some users were astonished how much insight a survey professional could get without even going to a building. Newer and ad hoc techniques were more troublesome: it seems to take a long time for a technique to bed itself in, particularly where high response rates and robust benchmarks are important to the analysis. | |||||
Energy assessments were well represented, often using – or at least informed by – the CIBSE TM22 method. Some people wonder about the focus on energy, as energy costs are small – but most of the User Group was convinced of its value and importance. An energy survey tells you about much more than energy, including design, build quality, installation, commissioning, management, maintenance, record keeping, user perceptions and behaviour, and the success or otherwise of both innovative and conventional techniques and technologies. Reducing CO2 emissions is also the biggest challenge being faced in the 21st century; and getting it done is an essential plank of sustainability. Starting in 2006, buildings in the European Union will also need to be energy-labelled under a European Commission directive: this is likely to be a spur to more feedback of all kinds. | |||||
Facilitated discussions between project team members were less widely held. It appeared easier to commission a user or an energy survey than to bring many people together for an unusual type of meeting. However, when discussions did take place, all participants found them valuable. No doubt, they will become more common as people get used to the idea. | |||||
Process change is what is really needed if feedback is to be completely routine. Initiatives such as Soft Landings are encouraging. However, such systems operate over the whole of a procurement cycle and on into aftercare – typically a minimum of five years, and hence have long lead times. They proved difficult to retrofit to projects that were already under way, because team members had already signed their contracts and were embarked upon conventional work plans. Nevertheless, there were encouraging glimpses of what might be. | |||||
As far as Packages go, both AMA Workware (for workplace assessment and change) and the broader Probe technique proved effective. In addition, whilst not always following the exact Probe model, many users found it useful to combine occupant and energy surveys so that they could relate both the ‘soft’ issues of people, management and culture and the ‘hard’ issues of technical and environmental performance. | |||||
Discussion
Making feedback routine
Ordinary people might reasonably expect designers and builders to be experts on the performance of the buildings they create. This is not normally so: those who produce buildings work on projects. These projects are about producing or altering buildings. Having produced one, they go on to the next – as do the project managers and the procurement wings of major construction clients. By and large, the providers do not stay around long enough to get much of an idea about how well the buildings they have produced actually work. As a result, occupiers may never make the most of the design potential of their building; and large differences between expectations and outcomes can occur virtually unnoticed. Consequently, designers can continue to repeat flawed prescriptions, and may not even realize when they have a success on their hands which they should be replicating. Those participating in the case studies were well aware of this, keen to use feedback to do better and often already active, though usually not on as broad a front as they would have liked.
The case studies demonstrate the value of feedback, the utility of the pick-and-mix portfolio approach and the potential for adding value to current projects, future projects, and to client and industry activities in general. They demonstrate that leading firms, large and small, are engaged in feedback and have a growing interest in developing and applying it. Some firms even regard mastering feedback techniques as being essential to improving the quality of their offerings, and so helping to ensure their professional and commercial futures.
Nevertheless, routine feedback is still rare. It used to be seen as a research and development effort at best – to be distilled into guidance for mainstream project teams, but seldom involving them directly. Even leading organizations have only recently begun to consider how to they might undertake feedback much more frequently, systematically and cost-effectively. Key problems to be solved include the following:
changing attitudes, getting clients and the industry to realize that follow-through and feedback are not an option, but an essential part of the process | |||||
changing that process, taking a broader view, as in Preiser and Vischer Citation(2005), and using techniques such as Soft Landings to pave the way by beginning to attach feedback and follow-through processes to familiar procurement procedures | |||||
developing a sound platform of techniques and benchmarks; the Techniques Portfolio has illustrated how this could be achieved on a pick-and-mix basis to help overcome entry barriers | |||||
finding the money: although everybody benefits, nobody wants to pay to get started | |||||
making widespread use of the knowledge gained | |||||
Getting started
Some of the barriers to getting feedback started are coming down, as demonstrated by the case studies:
Several firms were impressed by what experts could tell from occupant survey data alone, without even visiting the building – though of course visits and discussions bring yet more insights. In recent years, occupant surveys have been streamlined, questions simplified and standardized (though questions can be added to suit individual circumstances) and analysis semi-automated. Costs can be cut further if users can distribute questionnaires themselves (in accordance with clear rules) and key in the data. This occurred in several of the case studies. | |||||
The DQIs were also administered rapidly and have achieved considerable political support in the UK, particularly for government-funded projects. However, the relatively new technique still has some rough edges, with users finding difficulty both in applying it and in interpreting the results. The questionnaire is also longer and more difficult to use than established occupant surveys such as AMA and BUS, which have tended to become shorter, clearer and more modular as they have matured. At its present stage of development, User Group members regarded the DQI as working best as a spur to discussion during briefing and design development. | |||||
Energy assessments using the CIBSE TM22 method can be very cost-effective as well: the method was specifically designed to maximize the use of available information, to collect more only where essential (and not just nice to have), and to stop (usually after between a day and a week for a large building), when there was sufficient information for the task in hand. Users were also interested in its potential to summarize briefing and design data to improve design estimates, and to make a closer link between predicted and actual performance: this will be particularly important as we strive to reduce carbon dioxide emissions from buildings in order to arrest climate change (Bordass et al., Citation2004). | |||||
Discussions such as the HEDQF fora and Learning from Experience workshops can also be undertaken in a day or less, though these involve more people, take more organization – and then somebody needs to write them up (and sometimes to interview beforehand as well). | |||||
It appeared easier for people to commission a survey than to arrange a workshop, unless clients put their weight behind it. However, even the modest fees for well-established occupant surveys proved to be a significant deterrent – stopping several case studies and delaying others. Unanticipated expenditure, even if small, can be difficult for organizations,Footnote8 and designers and clients may be tempted to do things themselves. However, there are dangers in departing from tested procedures, in particular in quality assurance, a lack of streamlined procedures and an absence of benchmarks. The case studies indicate that the best combination may well be for disinterested professionals to do or support survey and benchmarking activities using proven techniques, with project team members closely involved, learning from the process, and pursuing the actions as part of their normal work and their specific aftercare activities.
Changes to procurement processes – in particular Soft Landings but to a lesser extent even the BRE checklist for POE activities – were more difficult to get started owing to their deeper impact and the difficulty of grafting them onto an existing system for which conditions of engagement had been agreed and contracts, responsibilities and programmes were already in place. It was best to start at the very beginning of a project – which also means convincing clients, who may not want to pay for additional services unless the value to be added is clear. Encouragingly, clients such as the University of Cambridge are alive to the benefits, and intend to take Soft Landings experience and proposals into account as part of their tender evaluation procedures for design and construction.
Value of feedback
The case studies identified a wide range of benefits: to the client, the user, the designers and the environment. As obstacles are removed, costs reduced and techniques become more standard, effective and reliable, follow-through and feedback could begin to be a routine part of building procurement. Perhaps the most immediate and demonstrable benefits occur when feedback immediately and directly affects briefing, design and management of future work. Case study examples included the following:
refurbishments and refits, as at the Department of Health, the Worthing Library (JPA) and Ellersly House (RMJM), where the same space (and sometimes the same people) can be surveyed before and after | |||||
phased projects, as at the Arup Campus, Queen Mary's College and University of Oxford, where insights from one phase can inform the next | |||||
relocations, as with the BM case study | |||||
Feedback is a valuable part of briefing generally. Indeed, some influential designers have said that this is the only time it is really worth doing – to examine past projects when planning future ones. However, this is very much the view of designers who are used to going away as soon as their work is physically complete and handed over; it denies other benefits of feedback – in particular in the aftercare period following handover – where not only is there the opportunity to improve a building's performance immediately, but for the design and building team to obtain experience and insight, take it straight back into their current and future projects, and pass their knowledge on to others too. Delivering better performance in use and closing the loop into briefing and procurement are complementary exercises. Indeed, where practical, it is useful for critical briefing requirements to be expressed in ways that can be assessed later as part of a feedback exercise.
Extending the feedback portfolio
Although the portfolio of techniques was small, the User Group felt it was more important to make better and wider use of a few techniques than to set out to increase their number and variety. Nevertheless, there were some specific ideas for expansion and development. In particular:
Making more use of POE techniques and metrics in briefing and in design. An example is using the CIBSE TM22 energy assessment and reporting method to assist with benchmarks, to summarize design data, and to provide better transparency between design intentions and assumptions and in-use outcomes. | |||||
Incorporating techniques from the portfolio into environmental assessment methods such as BREEAM in the UK and LEED in North America. These methods often started very much as design assessments, but are evolving to take more account of what is actually built and how it is used and managed. | |||||
The imminent requirement for building energy performance to be assessed, certified and in some circumstances displayed under the recent European Union Directive on the Energy Performance of Buildings (European Community, Citation2003). In the UK, this may be complemented by voluntary measures. | |||||
Widening the impact
Many techniques are quick and cost-effective, with direct benefits to the project and to the individuals involved. However, for maximum benefit, the insights also need to be disseminated within organizations and to the industry and its clients at large – knowledge management (KM) as it is now known. The case studies concentrated on getting the feedback information and using it on a project, not disseminating it within and between organizations. However, some firms on the User Group have also been putting considerable effort into their organizational systems to support feedback and KMFootnote9 and have also been involved in a parallel project on knowledge management in design offices, Spreading the Word, which is investigating how individual knowledge and scattered documents can be made into a shared resource for everyone to use. The project has already found that the numerous knowledge management books, software vendors and consultants have surprisingly little to offer design practices, and that traditional techniques are often empty ritual. It is therefore working with a number of practices to find out what really works and will report later in 2005.Footnote10
Conclusions
It was encouraging to obtain such a variety of case studies and largely positive opinions from the User Group. This confirms the utility of the pick-and-mix portfolio of techniques, and where value can be added cost-effectively to buildings and procurement processes through better follow-through and feedback. Members of the User Group see competence in these areas as important to their development, and most are also investing in knowledge management systems to spread feedback and other information from individual projects into their organizations.
A Portfolio of Techniques helps to break the ice where people are unsure and funds are short. Many techniques are now quick and efficient, so people can experience the activities and benefits rapidly, inexpensively, and at first hand. At the opposite end of the scale is Soft Landings. Although designed to run alongside any procurement system, it requires a high level of commitment at the outset, particularly from clients, who both have to change their ways and may also anticipate higher bills from the design and building team. Although the logic for using such systems is already clear to us, before taking such radical steps we suspect that many organizations will need to be exposed to more modest exercises (perhaps using one or two techniques from the Portfolio) before they become convinced of their benefits.
The enforced switch in emphasis from clients to designers that occurred when the CCC failed proved useful at the current stage of development. Clients want the benefits of more effective feedback and follow through, but most do not want to think about the detail. Design and building teams therefore need to learn what to do and how to deliver, so they can put a clear offering to their clients. Some organizations now want to do this, although progress with getting feedback, aftercare and KM systems effectively in place is still slow.
In planning the project, the team and its sponsors saw feedback and POE as essential to improving the all-round performance of our buildings and to support innovation, progress and learning. It was a shock when our initial research revealed that POE was often seen as academic and largely irrelevant to building procurement. An endemic problem was that the project team – often including the client representative – saw its tasks as done once a building was handed over. The research team therefore decided that the best way of getting feedback to stick was to aim for it to become a routine part of every project. The case studies illustrate what can be done and the value that can be added. The task now is to get the techniques widely adopted as part of all procurement routes, e.g. using the Soft Landings approach (Way and Bordass, Citation2005) in conjunction with other techniques, for example as described by Preiser and Vischer Citation(2005) and other references. User Groups are also planned that will concentrate on the specific priorities and requirements of individual building types or sectors (e.g. schools), whilst also maintaining connections to a broader network and making use of general-purpose techniques where possible.
Acknowledgements
The project was support financially by the Department of Trade and Industry (DTI) under its Partners in Innovation scheme, but the views expressed here are entirely those of the authors. The authors thank the DTI, their project managers Davis Langdon Consultancy, and the steering and user groups for continuing their support through a period of major difficulties. Finally, we thank the Usable Buildings Trust for taking up the challenge of leading the future development and for hosting the Feedback Portfolio website.
Additional information
Notes
5. See http://www.breeam.org
6. Briefly, chilled beams are finned pipes mounted in ventilated enclosures at ceiling level and cooled by down draught, normally – as here – without fan assistance.
8. The CIC also discovered this with the DQIs, so it now makes them available as ‘carnet’ packages, where only one order for a batch of surveys needs to be placed. Individual surveys can then be called down when people want to do them.
9. For example, Arup initiated a major feedback initiative in 1999; in 2004, Buro Happold appointed a doctoral student to help them look into the subject, and many of those involved in Spreading the Word are placing feedback (e.g. on people experience, projects, techniques, products and outcomes) at the center of their organizational information and communication technology platforms.
10. Meanwhile, for further information, contact [email protected]
Related Research Data
References
- Bordass , W. , Cohen , R. and Field , J. Energy performance of non-domestic buildings: closing the credibility gap . Proceedings of the International Conference on ‘Improving the Energy Efficiency of Commercial Buildings (IEECB'04)’ . April 18–22 . Frankfurt, , Germany
- Bordass , W. and Leaman , A. 2005 . Making feedback and post-occupancy evaluation routine 1: A portfolio of feedback techniques . Building Research & Information , 33 ( 4 ) : 347 – 352 .
- Carmona-Andreu , I. and Oreszczyn , T. 2004 . Architects need environmental feedback . Building Research & Information , 32 ( 4 ) : 313 – 328 .
- CIBSE . 1999 . Energy Assessment and Reporting Methodology: Office Assessment Method , London : Chartered Institution of Building Services Engineers . Technical Memorandum TM22
- Cohen , R. , Field , J. , Leaman , A. , Bunn , R. and Bordass , W. 2002 . Probe 23: Centre for Mathematical Sciences . Building Services , July : 47 – 52 .
- European Community . 2003 . Directive 2002/91/EC of the European Parliament and of the Council of 16 December 2002 on the energy performance of buildings . Official Journal of the European Communities , L1 : 65 – 71 .
- Federal Facilities Council . 2001 . Learning More from Our Buildings: A State of the Practice Summary of Post-Occupancy Evaluation , Washington, DC : FFC . Technical Report No. 145
- Gann , D. M. and Whyte , J. K. 2003 . “ Special issue on ‘Design Quality’ ” . In Building Research & Information Edited by: Gann , D. M. and Whyte , J. K. Vol. 31 5 , 314 – 411 .
- Jaunzens , D. , Grigg , P. , Watson , M. and Picton , E. 2003 . Building Performance Feedback: Getting Started , London : BRE Bookshop . BRE Digest No. 478
- Lorch , R. 2001 . “ Special issue on ‘Post-Occupancy Evaluation’ ” . In Building Research & Information Edited by: Lorch , R. Vol. 29 (2) , 79 – 174 .
- Nielsen , J. 1993 . Usability Engineering , London : Academic Press .
- Preiser , W. and Vischer , J. 2005 . Assessing Building Performance , Edited by: Preiser , W. and Vischer , J. Amsterdam : Elsevier .
- Way , M. and Bordass , W. 2005 . Making feedback and post-occupancy evaluation routine 2: Soft Landings – involving design and building teams in improving performance . Building Research & Information , 33 ( 4 ) : 353 – 360 .