The Use and Evaluation of a Simulation Game to Teach Professional Practice Skills to Undergraduate Architecture Students

Abstract

Architects are currently grappling to exploit new forms of communication made possible with developments in internet communication. At the same time, the construction industry is in a state of flux as novel project management systems are being introduced. Students need to understand the first principles of project management within the context of our changing environment. One of the best ways for students to learn about the legalities of the construction process is through role play and simulation, but there is a Catch 22. Unless students have a basic understanding of project management, the contractual process can be confusing and intimidating. Even fifth and sixth year architecture student are reluctant to ask practitioners questions, for fear of appearing ignorant. This paper describes the use of a web-based simulation game to deliver the Management, Practice and Law syllabus to Architecture Students. The web-based game allows students to critically observe the transformation of designs into buildings through the exploration of the contract management process. A questionnaire survey was used to assess the efficacy of the simulation game as a learning tool, and in particular the effectiveness of the web-based simulation in facilitating the development of professional practice skills in undergraduate Architecture students. The initial results of the assessment indicate that the simulation game is both a useful and complementary adjunct to traditional teaching and learning methods, as observed through the evaluation of outcomes, and helpful in developing generic professional practice skills of undergraduate students. Further game development will require more formal evaluation over a series of uses.

Keywords:

• Architecture Education
• Interactive Learning
• Simulation Game
• Teaching Evaluation

Introduction

Over the past few years professional organisations such as the Royal Institute of British Architects (RIBA) have commissioned numerous studies to gauge the current status of undergraduate Architecture education and to develop an agenda for improvement (e.g. RIBA, 1995, Stansfield-Smith, 1999).

These studies have identified a number of major issues such as:

• practitioner involvement;

• development of integrated curricula;

• new approaches to assessment of students, courses, and curricula;

• broader and flexible curricula and;

• utilisation of active learning approaches.

These initiatives have brought new impetus aimed at the continuous improvement of undergraduate and postgraduate Architecture education. The majority of architecture accreditation bodies worldwide now require the demonstration of learning outcomes.

Not only are students required to demonstrate technical skills but also outcomes that demonstrate the acquisition of skills and knowledge related to the more qualitative aspects of Architecture Education.

In the late 1980s, the construction industry faced increased national and international competition, stringent governmental regulations, and an environmentally-conscious populace. The industry also encountered issues such as the challenge of new technologies and materials and novel procurement processes, and construction of highly complex building projects (Murray and Langford, 2005). These forces place emphasis on the value of strong management skills required for the delivery of high-quality constructed facilities (Murray and Langford, 2005). That, in turn, added a new dimension to the Architecture profession and led to the evolution of practitioners who had a greater involvement in the management of the construction processes. As a direct response to the needs of the industry, many UK universities introduced undergraduate and postgraduate Architecture curricula that placed more emphasis on the development of project management skills (Stansfield-Smith, 1999).

Numerous innovative tools have been used for some time, to help students develop construction project management skills. The use of simulation gaming to aid student learning is not new. Au and Parti (1969) first suggested that computerised heuristic games could be used for the education of engineers and planners engaged in the construction industry. Scott and Cullingford (1973) described a scheduling game for construction industry training. Halpin (1976) presented a model for a project gaming system that allows students to plan, monitor and control hypothetical projects. Harris and Evans (1977) developed a road construction simulation game for site managers focussing on the planning and control of linear construction projects. Herbsman (1986) explained the use of civil engineering project management games in the US where players were required to participate in the design and execution phase of live projects. Rounds et al. (1986) described a construction game that simulates the progress and project reporting structure of an industrial construction project. Dudziak and Hendrickson (1988) developed a game for contract negotiations. Vehosky and Egbers (1991) explained the development and use of a simulation game for the management of the design of a construction project. AbouRizk (1993) and AbouRizk and Sawhney (1994) described the development and deployment of a construction bidding game that provides undergraduate civil engineering students with a thorough understanding of the components and methods of bidding. Fruchter and Krawinkler (1995) described the development and testing of a new computer integrated Architecture Engineering Construction teaching environment. Scott et al. (2004) described the use and evaluation of a web-based simulation game to teach planning and control to undergraduate construction engineering students.

It is clear from the above summary that a number of active learning systems aimed at training students in different aspects of construction management have been designed and deployed over time. Most of the systems mentioned above however, are either manual or have been developed with limited use of advanced computer technology. The existing systems are not geared towards the dissemination of knowledge related to the professional practice, the contractual process nor to the specific needs of Architecture students. Active participation of practitioners is also not utilised in the existing systems.

This paper describes the use, application and efficacy of a web-based simulation game as a learning tool in Architecture Education. The focus of the game under discussion in this paper is centred on the delivery of some aspects of the ARB/RIBA Part II Management, Practice and Law curriculum. This syllabus is arguably theoretical in content and it was thought that the development of a scenario-based set of simulations could provide a more effective and engaging mechanism for the deployment of teaching resources. The game provides an environment within which students can explore the management of practice, project management or (as in the current test case) the intricacies of contract administration and legislation.

The paper is divided into three sections. A description of the contract management class where the simulation is used, is presented together with an outline of the game itself. Based on a questionnaire survey, the paper then presents an assessment of the effectiveness of the web-based simulation in facilitating the development of appropriate management skills in undergraduate Architecture students. Finally, conclusions are drawn on the effectiveness of the game as a teaching and learning tool, limitations noted and suggestions for further research proposed.

Description of the Educational Context

The application of this scenario simulation finds a role within the taught course element of the ARB/RIBA Part II Management, Practice and Law curricula. The basic objective is to provide an introduction to the systems and documentation which are necessary for the implementation of an architectural project in the practice setting; more specifically, to demonstrate to students the nature of construction law, of contracts and of concepts of responsibility and liability. To this end, the web-based system aims to provide an interactive learning setting for the understanding and development of professional as well as team working and decision-making skills.

User interface, structure and development platform

The graphical user interface of the simulation game is designed to operate as a web-based interface and simulate as far as possible the critical observations undertaken by participants in a classroom environment. The game is essentially a series of scenario-based exercises designed to teach students about the intricacies of contract legislation. The scenarios within the game are typical of those that an architect would meet in practice. The student assumes the role of an architect overseeing a hypothetical construction project and acts as the contract administrator. The scenarios are grouped around three board aspects of running a project: carrying out of the works; the programme of works; and financial and contingent matters. Additional scenarios in the game relate the other responsibilities to any of the other parties involved in the construction process including issues relating to their contractual implications.

In order to address each scenario, participants are provided with on-line access to all documentation they require to solve a particular scenario through the use of electronic document libraries (see Figure 1). These libraries contain all documentation relevant to the individual scenario and any standard forms participants need to submit in order to complete the scenarios.

Figure 1 A screenshot of the Contract Management Simulation game

Actions taken by participants to solve an individual scenario are recorded in a Scenario History Panel (see Figure 1). This area allows participants to record what actions, notes and communications they make in order to resolve a scenario.

It also allows those who communicate with the participants, the game controllers, to enter their particular response to actions, notes or queries. Once a scenario is completed successfully, game controllers release the next scenario and the participants are then invited to resolve the next scenario and so on until the game is successfully completed.

The students are typically divided into 12 teams who vie with each other to see who can achieve the highest score of correct answers to a wide range of scenarios.

The simulation game is predominantly written in HTML/ASP using VB script and a small amount of Jscript. The application is hosted on a Microsoft Windows 2000 server running MS IIIS 5.0. The server itself is a Dell PowerEdge 2800. The game is accessed via a login page that accesses data from MS SQL Server7, which is hosted on the same server. All the data collected and used by the application is stored in a database device on the server system.

The use of the simulation game to teach professional skills to undergraduate Architecture students

Most undergraduate Architecture programmes are designed to be the first step in the development of professional practitioners. Ultimately, to achieve professional status, Architects must demonstrate that they have developed appropriate learning and skills.

The requirement for institutional accreditation places the onus on Schools to demonstrate learning outcomes as part of their undergraduate and postgraduate course provision. There seems to be some agreement on the common sets of generic skills expected of those entering practice amongst the professional architecture institutions worldwide. The signatories to the UIA accord of professional practice is an example where institutions are beginning to collaborate on matters related to professionalism. While design know-how features high on the list, there are significant requirements with respect to less quantitative skills in management, law, human resources, etc. It is necessary for Architecture Schools to demonstrate that their programmes meet all of learning outcomes as specified by the institutions. Traditional methods of testing and demonstrating learning and competence in design topics cannot, however, always be applied to more management-related subject matter. The simulation game under discussion has been used as a tool to teach project management skills to 5^th year students. Many of the concepts associated with project management are unfamiliar to undergraduate students. The students have limited practice experience so many of the concepts are new to them.

The simulation game provides the students with an opportunity to learn about the intricacies of contract management in a low-risk, controlled environment. The simulation is based around a series of tasks relating to the deployment of JCT98 building contract. The contract management simulation is essentially a game of question and answer, the purpose of which is to familiarise students with the intricacies of contract administration and the legalities of the construction process. The students’ objective is to manage the construction process in response to a series of events as set out by a chronology of likely on-site scenarios. To this end they enter into a real-time correspondence with the client, contractor and other consultants as required.

Evaluation Process

The evaluation presented below relates to a class of Part 2 students from the Department of Architecture at Strathclyde University, who played the computer simulation game as part of the 5^th year Design, Management and Practice class in March 2006.

The success of the simulation activity could not be assessed using as a comparison student performance in the game across the years as these data were either not available or not in a form that would make for meaningful comparison. All students on the class participated in the game, so it was not possible to construct a control group for evaluation purposes. This is not necessarily a weakness of the study, as Lucas (1997) has suggested that the measurements of differences in student grades over time, has limited value given the changing nature of student cohorts from one year to another. As an alternative, Lucas (1997) recommended obtaining feedback on student perceptions of a learning activity as a means of assessment.

On the basis of these recommendations, questionnaire tools were developed to elicit perceptions of the simulation game system from students involved in the class. A similar approach was adopted to elicit the opinion of students undergoing training for the ARB/RIBA Part III examination (Agapiou, 2006). In that case, the Part III students were asked to assess how the computer simulation worked as a group exercise, how it compared with more traditional approaches and to identify the best and worst features of the web-based system.

The main focus in this paper is an assessment of the effectiveness of the web-based simulation in facilitating the development of appropriate professional skills in undergraduate Architecture students.

To evaluate whether the simulation had succeeded in making the learning process more effective in accordance with the class objectives, we posed a number of questions to the student body relating to the understanding of the contractual process; ability to apply contractual concepts, relevance of the class lecture material and the development of written and communication skills. Other open-ended ended questions to allow comment and suggestion were included. The questionnaires were compiled and administered with the assistance of the University’s Centre of Academic Practice and Learning Enhancement.

The questionnaires were issued to all students at the end of the session, who were asked to complete the survey prior to their departure from the computer laboratory. A total of 60 questionnaires were issued to students. Of the total number of questionnaires administered, responses were received from 50 students, representing a response rate of 83%. A breakdown of the students’ responses to the questionnaire and a discussion of the results are presented in the section below.

Discussion of questionnaire survey results

The results of the questionnaire survey indicate that the simulation game helped students understand and apply contractual concepts and principles as 60% of students agreed with question 1 and 2 (see Table 1). The students also thought that it helped them exercise their professional judgment more effectively (question 3). Nevertheless, for many students the activity did not help them understand the course material as only 41% agreed with question 4 (see Table 1). This may suggests the tutors did not link the activity sufficiently to the course content.

Table 1 Students’ views of the Simulation game

		Mean		Frequencies (n = 50)
Question No.			Strongly agree (%)	Agree (%)	Neutral (%)	Disagree (%)	Strongly disagree (%)
1	It helped me understand the intricacies of legal contracts and the contractual process	3.7	0	60	34	6	0
2	It enhanced my ability to apply contractual concepts and principles	3.7	0	60	33	7	0
3	It helped me exercise professional judgment on contractual issues more effectively	3.5	0	58	33	9	0
4	It helped me understand the relevant lecture material	3.3	0	41	40	19	0
5	It increased my written communication skills	3.5	0	47	32	21	0
6	It increased my verbal communications skills	3.4	0	44	43	12	0
7	It increased my ability to read contract documents	3.5	0	58	22	20	0

There was only moderate agreement with the proposal that the simulation game increased written and verbal communication skills with 47% of students agreeing with question 5 and 44% agreeing with question 6 (See Table 1). The comments about group work were more encouraging (see Table 2).

The results indicate that 50% of students said each member of the group contributed equally to the exercise (question 8), and 60% of the respondents said there was a high level of cooperation in their group (question 9). This is evidence of the development of team working skills, although a change in these is not measured by the questionnaire survey.

Table 2 Students’ views of their group

		Mean		Frequencies (n = 50)
Question No			Strongly agree (%)	Agree (%)	Neutral (%)	Disagree (%)	Strongly disagree (%)
8	All members contributed equally to the work	3.5	0	50	28	22	0
9	There was a high level of co-operation in my group	3.7	0	60	27	13	0

Table 3 Students’ views comparing the simulation game with other exercises

		Mean		Frequencies (n = 50)
Question No			Strongly agree (%)	Agree (%)	Neutral (%)	Disagree (%)	Strongly disagree (%)
10	It helped me communicate more effectively in a professional capacity	3.6	0	63	29	8	0
11	It helped me understand how to better organise a group to work as a team	4.0	0	80	16	4	0
12	I have a greater awareness of professional responsibilities	3.6	0	58	34	8	0
13	I enjoyed doing the exercise more	3.9	0	73	20	7	0
14	I found the exercise more difficult	2.7	0	11	48	41	0
15	It was more work	3.4	0	41	39	20	0
16	It took more time	3.5	0	41	38	21	0
17	Group work was more difficult	2.9	0	46	39	15	0

Students were asked to compare the contract simulation game exercise to other class activities. Their opinions (shown in Table 3) were that the exercise helped them communicate more effectively in a professional capacity (63 % agreed), helped develop team-working skills (80%) and that it raised greater awareness of professional responsibilities of Practitioners (58%). The majority of students did not find it more difficult or more work, but almost half agreed it took more time. The activity was obviously extremely popular with students, as shown in Tables 4 and 5. When asked for their preference, 88% of students chose the simulation game over ‘traditional’ exercises and 90% recommended it for use in class the next year.

Table 4 Students’ views on whether the exercise should be repeated in the following year

Would you recommend that the exercise be repeated in 2007?	Number	Percentage (%)
Yes	45	90
No	5	10
Total	50	100

Table 5 Students’ views on whether they prefer the Simulation game or traditional exercises

Overall, given the choice between the Simulation game and more traditional exercises which one would you prefer?	Number	Percentage (%)
Simulation game	44	88
Traditional exercises	6	12
Total	50	100

The open-ended questions in the survey identified a number of operational improvements that would help reduce the students’ administrative/operational activities and increase the realism of the exercise. The responses demonstrated that the simulation game had given the students a better understanding of the intricacies of the contractual process and experience in using the appropriate administrative documentation. Most students preferred the game to other types of class exercise, though some suggested changes to the allocation of time. There was also a realisation and appreciation that the exercise was developing the skills and knowledge that traditional teaching and learning methods would allow. These indications support those of Anderson and Lawton (1991) who found students have strong preference for simulations over case study teaching methods.

A critical factor in the success of the web-based simulation is the clear identification of student interaction with the system. The system is currently designed for utilization by Architecture Students in professional practice settings although with some adaptation the exercise could easily be used in other Built Environment programmes including Quantity Surveying, Civil Engineering Management. The simulation game also lends itself for Continuing Professional Development purposes.

Conclusions and Limitations

This paper has described the use and efficacy of a web-based simulation game to teach professional practice skills to undergraduate Architecture Students. The main benefit from using the game was that it helped students understand the contractual process and exercise their professional judgment more effectively. When these comments are linked with responses about understanding and applying contractual concepts and principles, it is clear that most students benefited from the introduction of the simulation game into the 5^th year practice class. However, in order to formally demonstrate the usefulness and effectiveness of the game, a systematic evaluation of learning needs to begin. The initial testing carried out so far suggest that this evaluation needs to identify the skills acquired and learning outcomes being effectively achieved. It should also allow the detection of weaknesses or areas for improvement in the simulation game. The intention is to continue the evaluation over a series of simulation uses of the system. This will allow the accumulation of a large body of data for a range of skills and learning outcomes. Use of the game to date, indicates that it a useful and complementary adjunct to traditional teaching and learning approaches. Initial results from the evaluation of learning outcomes support this observation. The formal evaluation of effectiveness will be continued with a view to further game development.