A two year case study: Technology Assisted Project Supervision (TAPaS)

Abstract

During their final year undergraduate project a student may feel under-supported, stressed or isolated. In an internally funded project we set out to investigate the benefits of using a diverse blend of collaboration and communication tools alongside traditional methods of final year project supervision. We established separate formal and informal communication channels between the supervisor and their project students and a community of practice of students and supervisors was set up using twitter or a web forum. Using a wiki as a collaborative workspace and repository, student project pages were created and virtual supervision was blended with face-to-face supervision using electronic logs. At first some students were dubious about our approach and disapproved of at least one of the tools used. The supervisors were also initially sceptical of an increase in workload due to the multiplicity of tools used. In this paper we present how the staff and students benefited precisely because of the diverse range of tools used. The methods used resulted in transparency of students’ and supervisors’ actions, however, lessons were learnt about how to address student concerns about plagiarism in such an open environment.

Introduction

Technology is increasingly changing the ways in which academics conduct their duties, so much so that it is often presented as a panacea. However, in our opinion pedagogical motivations should be the real driver for any use of technology within education. Knowing the pedagogical needs of the learning and teaching situation and finding a suitable match of tools has been our approach in selecting the technology we used here.

During their final year individual project, undergraduate students often feel isolated (Armstrong, 1997). This may be due to the level of tutor support and peer interaction that they have become used to during the earlier years of their degree (Popov, 2003). Peer interaction is considered to be one of the key elements of course design that supports deep learning (Biggs, 1989). But in a traditional setting a supervisor commonly uses individual face-to-face meetings, paper based logs and emails for supervision purposes, none of which are suited to peer interaction. In this case study we propose the use of a novel mix of communication and collaboration tools alongside the traditional methods used for project supervision and we show how a community of practice (Lave and Wenger, 1991) forms through carefully chosen open and shared communication and collaboration tools. The results demonstrate that pedagogically-chosen technology for project supervision is beneficial to both students and supervisors in several ways.

Research questions

In order to demonstrate the usefulness of Technology Assisted Project Supervision (TAPaS) for students and supervisors we set out to answer the following research questions:

1. Is the use of a diverse blend of communication and collaboration tools beneficial in combating the feeling of isolation in project students?

2. Is the use of a diverse blend of communication and collaboration tools beneficial in motivating project students?

3. Is the use of a diverse blend of communication and collaboration tools beneficial in making communication effective and efficient between a project supervisor and their students?

The target population for our study was final year undergraduate degree students in the department of Electronic and Computer Engineering at the University of Portsmouth. There was a total of 32 undergraduate students being supervised by four supervisors over a two year period.

Approach and technology

To balance supervisor workload we either replaced half of the face-to-face contact with virtual supervision or made face-to-face meetings ‘on demand’ alongside virtual supervision. We also proposed that all project documentation be maintained in electronic form and we used the following mix of tools:

• Twitter — a social networking and micro-blogging site used to keep up to date with friends and colleagues. It allows the user to post short updates (limited to 140 characters per message) or web links that others can see on their own twitter accounts, often answering the question “what are you doing?” Users can choose who they want to follow, thus controlling the information they receive

  or

• Web forums — software that allows multiple users to discuss different topics, often organised into well-structured threads. Users can post their own messages and read messages from other users under each topic. Unlike twitter, forums do not have limits to message length

  and

• Wiki — software for collaborative live editing of web pages online. Users can register and edit web pages and the software maintains version control by naming each edited page differently. Users often use different coloured text to help identify their own comments. Really Simple Syndication (RSS) can be used to be informed of recent changes on these web pages, just as it is used to keep up to date with news websites. Often people working on wiki pages use twitter to keep collaborators informed of changes they have made.

All project documentation was stored in a shareable electronic format. For regular progress reporting and flexible virtual supervision we used e-logs (electronic versions of paper based project log books) which were either offline electronic documents sent as an email attachment or maintained on an ongoing basis on a wiki. Wiki pages were also used for maintaining a live project plan, recording meeting minutes with the supervisor and keeping miscellaneous notes.

Case study

At the University of Portsmouth, the foundations of Technology Assisted Project Supervision (TAPaS) were laid in 2006/07. The principal author introduced e-logs to blend formal face-to-face supervision with formal virtual supervision. This allowed students to work and share their work with the supervisor simultaneously. Timely and specific guidance was provided to the students using the e-logs in return. As a result, e-log based supervision replaced half of the project meetings. The following year the principal author introduced a combination of the micro-blogging site twitter and a wiki. Twitter provided an informal communication channel and the wiki a more versatile one which could be used both formally and informally. There were five students under one supervisor involved in this first phase of the study.

In the following year, the Faculty of Technology at the University of Portsmouth awarded TAPaS funding for a wider study, involving the co-authors. The aim of the funding was to evaluate the technique used for project supervision and identify its benefits. By design, the team decided that each supervisor would use a slightly different set of tools suited to their style, but this nevertheless provided us with the data we needed to do our analysis. These differences are summarised in Table 1. There were 27 students and four supervisors (A, B, C and D) in this second phase of the study.

All supervisors chose to use either twitter or forums as an open and shared communication channel while only three supervisors also chose to use a wiki as a collaborative working repository. One supervisor used only twitter and did not use the wiki or the e-logs.

Table 1 Mapping all four supervisors’ (A, B, C and D) choice of tools over two years

Tools	e-logs	Paper logs	Wiki	Twitter	Forums
2007/08	A	Not used	A	A	Not used
2008/09	A, B and C	D	A, B and C	A, C and D	B

Note: All supervisors used some of these tools in addition to some face-to-face meetings

Methodology

During the pilot phase of study (year one) qualitative data capturing student expectations was collected at the beginning of the year through open ended research questions exchanged via email. The students were then asked to present their reflections and experiences of the tools used in a final report. This helped us in planning the study for the following year. Data from all five students is presented in a ‘work in progress’ paper (Malik, 2008).

Similarly, in the second year we decided to capture the initial expectations and experiences of our students through a survey completed four weeks into the project (see Table 2). However, due to difference in supervisor preferences of the tools used, we had to use a slightly altered questionnaire for each supervisor and their students (as per Table 1). This resulted in a varying number of respondents to different questions, as shown in Table 2. A total of 13 of 27 possible respondents actually completed the survey. The questionnaires had (up to) 19 questions that required responses on the Likert scale (Likert, 1932) as well as a number requiring free text answers. One response was largely neutral and has been taken out. A 44% (12 of 27) response rate is, in our opinion, representative of the group (given all 27 students had been invited to respond) and mirrors the low sample size per study found in the research literature for project supervision.

Table 2 Students initial views four weeks into their project in year two

Wiki (8 respondents)	Strongly agree	Agree	Neutral	Disagree	Strongly disagree
Was helpful	5	3
Was a difficult interface to use	2	3		3
Department should continue using it	1	7
Helps build a sense of community		5	2		1
Gave me a sense of ownership		5	2	1
Twitter/web forum (12 respondents)	Strongly agree	Agree	Neutral	Disagree	Strongly disagree
Was helpful	3	6	3
Was a difficult interface to use		2	3	4	3
Department should continue using it	2	6		4
Sense of community formation	2	4	3	3
Open and shared communication was useful	2	7	1	2
E-logs (10 respondents, unless stated otherwise)
Was helpful (9)	3	5	1
Logbooks should only be shared with supervisor	3	4	2	1
Logbooks should be shared with all students and supervisor		3	1	3	3
Prefer paper log over e-log			3	5	2
Use both paper and e-log	1				9
Miscellaneous (12 respondents)
TAPaS is better than pure face-to face supervision	2	6		1	3
Virtual contact and feedback useful	2	7	2	1

All four supervisors met once at the start of the project and again for a focus group at the end in which they detailed their approaches and compared their experiences. The views shared are recorded in Table 3.They also discussed some common issues and highlighted some good practices.

Students who completed the survey were invited to participate in a focus group at the end in order to capture their experiences during their project. We collected views from at least one student from each supervisor, with five out of the 27 students attending the focus group. In the focus group, the students individually reflected upon each tool used and later discussed their experiences as a group (see Table 4).

Results

In a survey carried out four weeks into the project (see Table 2), the students were very positive about using wikis and e-logs, with all eight respondents in agreement that these were helpful. But a significant minority (three out of eight) thought that the navigational aspects could be improved. Despite this, all eight respondents stated that the department should continue using wikis for project supervision. Nine of the 12 respondents have found twitter or web forums useful for their project work and eight stated that the department should continue using them. Table 2 also shows that a significant minority (four out of 12) disagreed or strongly disagreed that TAPaS was better than purely face-to-face traditional supervision. We found that these students dropped the use of twitter and/or wikis altogether. On closer inspection, it transpired that these four students were supervised by supervisors C and D. For supervisor C only one of their seven students engaged with twitter and wikis. The other six students did not come to the focus group interview either.

Supervisor D could only engage with three out of nine students on twitter. He had chosen to use only twitter (and not a wiki or e-logs) and three of his students disagreed with the statement that “the department should continue using twitter”. This suggests that twitter as a tool on its own was not appealing to the majority of students — neither was it particularly useful for the supervisor. In supervisor A and B’s cases all students engaged more uniformly with the tools.

Table 3 summarises supervisors’ reflections which came out of the focus group. They all agreed that the ‘regularity of contact’ they had through e-logs was ‘a good thing’ and that ‘it enables timely and specific (need-based) guidance’. This suggests regular contact between the students and their supervisors.

All supervisors agreed that the formation of a community of practice saved time and made the answering of common questions more efficient. A drop in student emails compared to previous years was noted, as communication took place via other tools. With twitter messages of 140 characters or less, the move to this method of communication was, over time, very useful as students learnt to ask succinct questions of each other and the supervisor.

Live plan monitoring (wikis) and e-logs gave supervisors ways to apply pressure to students as necessary and the wiki was considered to be a one-stop repository for all content relating to the project. The multiplicity of the tools used was potentially cause for an increase in the supervisor’s workload, although this was counter-balanced by a reduction in the number of face-to-face meetings and emails, the increase of succinct questions on twitter and by the facility to respond to commonly asked questions in the open.

Table 4 shows data from a focus group interview with students after the project finished in year two. When discussing potential problems with the tools, students mentioned their concerns about others plagiarising their work from the shared e-logs. These concerns are already addressed by the tools themselves, as all records on the wiki are time stamped and it is easy to trace the original author. This information, however, needs to be better explained to students at the beginning of the project so that such concerns can be alleviated early on. An alternative approach is not to share e-logs or to ask permission before sharing.

Table 3 Supervisor’s reflections on tools used and issues concerning project supervision expressed in a focus group

Tool used/ issue	Views	Supervisor(s)
Paper logs	Encourages reflection, deeper learning and engagement Visible track record and additional insight into students’ working styles Not robust, as student can write it up towards the end. Date stamping done by the supervisor from time to time can avoid this	All All A,C,D
E-logs	All the benefits of paper logs but more convenient Flexibility — space and time independent Hidden pressure to submit regular work on students Timely and specific guidance is enabled Regular exchange has proven to be a good thing	All All All All All
Twitter / forums	Community of practice can be formed and this saves you time Drop in number of emails as a result Flexible — space and time independent (blurred boundaries) Blurs social boundaries too Benefits from the wider community (if there is one)Useful for making announcements and staying in touch with students Informal channel for communication Helps point each other to wiki pages where more detailed work takes place Additional channel for gaining an insight into students’ working patterns Encourages to-the-point messages (140 characters) and low effort	All All All All All All A,C,D A,D A,B A,D
Wiki	One place to aggregate all content related to project Easy place to put meeting minutes as the meeting happens Hidden pressure due to online minutes and monitoring tools used Peer pressure building as feedback given on common milestones is shared. Yet another tool to give insight into students’ work practices If used to host e-logs then you can get most up-to-date entries	A,B,D A,B,D A,B,D A,B,D A,B,D A,B
Face-to-face meetings	Useful for detecting plagiarism and avoiding it An opportunity to inspire students Initial momentum building Richer communication environment Agree plan of action and review progress Tends to be formal	All All All All All A,C,D
Common issues prior to TAPaS	Identifying and avoiding plagiarism Student motivation Monitoring progress Student isolation	All All All All
Good practice	Online community of practice formation Use of online and face-to-face blend for supervision Recording meeting minutes with students Tailored supervision for different students Making use of peer pressure on students where possible	All All All A,B,C All

Even from early on in their projects, the data collected (Table 2) shows that five out of eight respondents agreed that using the wiki helped them to build a sense of community, while six out of 12 agreed that twitter and web forums were also useful in this way. Qualitative focus group data from both staff and students noted that using twitter or web forums for communication purposes created an informal channel that linked to ‘a wider community of students to get access for questions that need answering’ and that ‘if someone needed help, others would come in and help.’

There were, on average, 87 twitter messages per student (65 messages including the web forums) over the course of the project. Twitter’s use here is very similar to the “recommender” system talked about in Engineering Challenges (2009) where humans (instead of machines) recommend links to relevant resources. Links shared in response to quick questions asked on twitter often lead to further reading of online articles recommended by peers. These articles are likely to contain help or at least directions to further help for answering the original question.

Table 4 Selected student views captured in a focus group after the end of the project in year two

Tools used	Views
E-logs	The logistics of using e-logs is better than using a paper based log They are a flexible tool to collect and share work with the supervisor It had generated interest in other students about my project I was worried about plagiarism due to sharing of my e-log Feedback is detailed and consistent Good for demonstrating to the supervisor where you are As everything is recorded it motivates you to demonstrate more My handwriting is bad so it helped me record stuff better It was quick and easy to do an e-log The written feedback is better as verbal you tend to forget. This way you can come back to it anytime you want You can backup e-logs but not paper logs Doing it regularly was motivational/forced us to do the work
Forums	It helped to organise the project better as you could ask questions about different stages of the project and someone or supervisor would respond to the query The questions could be very detailed or very short and responses from the supervisor were very helpful If someone needed help others would come in and help and the supervisor also will help from time to time Wider community of students to get access for questions that need answering
Twitter	It has multiple interfaces so you could choose a way to link to it We used it to communicate with supervisor mainly and some milestone related tasks to help each other We meet regularly outside twitter as we three are friends There was little or no overlap on the projects — so we could not help each other much — project process is common and we used twitter for that during the project duration 140 characters was good as it meant we asked specific questions about the project and got quick specific answers about it It won’t be suitable to post code and discuss code or more technical questions for that we would use forums or wiki and share the link Wider community of students to get access for questions that need answering
Wiki	Provided a one stop place for all my project related work/e-logs/plan etc. Using wiki to record meeting minutes gave me and my supervisor the ability to know where my work is and recording it meant that the current status is agreed and I could progress from there Looking at each other’s meeting minutes/plan on the wiki was good as it was a competitive environment to be in The comments on my work from my supervisor using the wiki was helpful Other student’s work and supervisor comments on it helped me give an idea of what things to consider for my project report/work Sharing the work we produced on the wiki with an employer is great idea

Sharing minutes of face-to-face project meetings with the supervisor through the wiki was seen as an effective way of getting ideas for one’s own project: ‘Other students’ work and supervisor comments on it helped me get an idea of what things to consider for my project report/work; it had generated interest in other students about my project.’

Students commented that being able to see each others’ meeting minutes created a more competitive working environment: ‘Looking at each others’ meeting minutes/plan on the Wiki was good as it was a competitive environment to be in.’ While supervisors can try to motivate students by lauding others as good examples, evidently being able to see the actual work done by others with ones’ own eyes has an additional positive effect on individual student motivation.

As all work recorded on a wiki is time stamped and can be seen immediately by the supervisor, this creates additional pressure to work regularly on the project and thus improved the overall progress: ’Doing it regularly was motivational/forced us to do the work; as everything is recorded it motivates you to demonstrate more.’ Peer interaction and high motivation are two of the four elements that encourage deep learning (Biggs, 1989).

Students also appreciated the convenience of having all project-related resources in one place on their project wiki page: ’It provided a one stop place for all my project related work/e-logs/plan etc.’ E-logs helped the students to avoid some of the traditional problems with paper-based logbooks, such as forgetting the logbook when coming to a meeting or to work in a lab: ’The logistics of using e-logs is better than using a paper based log; they are a flexible tool to collect and share work with the supervisor.’

Conclusions

We have demonstrated how multiple formal and informal communication channels between a supervisor and their students can be established by using a diverse set of tools. Although we saw a mixed take-up of the tools used, we observed that where the students did use the tools they were quite positive about their experiences. The tools only worked well when used in parallel, as each tool addresses a specific project supervision need. Some students were worried about others being able to plagiarise their work and these concerns could be alleviated early on by ensuring awareness of the time stamped nature of wiki contributions. These channels afforded peer interaction and support which in turn helped combat the common feelings of isolation in project students. Being open and shared, these channels ensured transparency in supervisors’ and students’ actions. The supervisor could regularly monitor student progress and provide timely and specific guidance and the ensuing written feedback on specific work was seen by students as helpful, detailed and consistent. Where the guidance was in the open, it proved motivational for other students in the group and helped them to generate ideas for their project. Student emails were discouraged in favour of shorter communications on twitter or messages on web forums. By answering common questions in the open, supervisors could increase their efficiency. All this, plus reducing the number of face-to-face meetings, helped balance supervisor workload.

Acknowledgements

The principal author would like to thank all the supervisors who agreed to be a part of this case study for all their support and hard work. The authors would also like to thank Mr Tim Goodhead, Dean of the Faculty of Technology at the University of Portsmouth for making this possible by granting the funding for this project and also Mr Tim Morgan (Associate Dean Academic, Faculty of Technology) and Dr Misha Filip (Head of Department Electronic and Computer Engineering) for supporting this work.