Messenger in The Barn: networking in a learning environment

Abstract

This case study describes the use of a synchronous communication application (MSN Messenger) in a large academic computing environment. It draws on data from interviews, questionnaires and student marks to examine the link between use of the application and success measured through module marks. The relationship is not simple. Total abstainers and heavy users come out best, while medium level users do less well, indicating the influence of two factors. The discussion section suggests possible factors. The study also highlights the benefits of support and efficiency of communication that the application brings. Although there have been many studies of synchronous communication tool use in the office and in social life, this is one of the first to examine its informal use in an academic environment.

Keywords:

• instant messaging
• computer-mediated communication
• social computing
• informal communication
• ethnographic research

Introduction

The background environment in which this study took place was a large open-plan 500-seat Very Large Computing Centre (VLCC); often referred to as ‘The Barn’, which is open for the use of all students on one campus of a Scottish university. The facility has not turned out to be the idyll that was envisaged and this has provided an interesting field for sociological research (Buckner and Davenport 2002).

The case study reported here was prompted by observing students using MSN Messenger in computing tutorials. MSN Messenger is a synchronous communication application offered free by the Microsoft Corporation. (The latest version is called Windows Live Messenger.) Students were observed using the application to keep in touch with friends. Some students said that they used the application as part of their studies, either in informal study groups or as a way of coordinating group projects or simply to ask for help when stuck.

The institution has an ambivalent approach to the use of the application. It is not provided as part of the official computing service. Some members of staff regularly use it to communicate amongst themselves and with students. Some members of staff ban it from tutorials, while others ignore its use.

This study investigates the benefits and disadvantages of the application to students. It compares existing literature, interviews, questionnaire data and assessment marks. The next section is a review of relevant literature. Then come the findings from an interview phase, the results of a questionnaire and then a comparison with student marks. Afterwards, there is a discussion section to compare the findings of each part of the investigation. The text of the questionnaire is provided in Appendix 1.

The application

There are a number of synchronous text communication products that are similar to MSN Messenger. For the rest of this communication, the term Messenger will be used to refer to any generic synchronous messaging application.

An instant messaging system typically comprises a server, which is maintained centrally, and a client, which runs on a user's machine. There are a number of similar instant messaging systems available (Microsoft, AIM, ICQ …). ‘System’, in this context, refers to the server that coordinates the messages and communications between clients. These are similar, but not interoperable.

Messaging clients typically take the form of a management window, in which users may see the names of those with whom they normally communicate (or buddies), and a conversation window, in which users may see ongoing conversations (Figure 1). The management window usually shows who is online at that moment.

Figure 1 The two Messenger application windows.

Interaction with people

In a questionnaire study of the campus ecology, de Bakker, Sloep, and Jochems (2007) report that students use Messenger for (in order of frequency): discussing tasks with colleagues, sharing files, cooperating on tasks, discussing course material, gathering content for coursework, reflecting on colleagues' work and teacher guidance.

Despite Messenger's usefulness, some feel guilt about the time they put into Messenger communication. Green et al. (2005) report that online communication makes people feel happier during the time while they are engaged and that this is mildly addictive. For some reason, the boost in mood seems to be more pronounced in women. In their survey conducted on US college students, over half said that they used the Internet too much. In a separate piece of research at Rutgers University (Kubey, Lavin, and Barrows 2001), 9% reported that they may have become dependent on the Internet, i.e. that they wished that they used it less, but found it difficult to control. Herbsleb et al. (2002) confirm that there are workplaces where the culture regards ‘informal chatting’ as a waste of time or socially undesirable. It would be difficult for the application to gain ground in such an environment.

There is also evidence (Kubey, Lavin, and Barrows 2001) that Internet usage in general may have an adverse effect on student success. In particular, 14% reported that their academic performance had been hurt by Internet usage, 40% reported that it kept them up late, 42% reported that they felt tired the next day, while 20% reported that they had missed classes as a result of Internet use the previous evening.

Many office users leave the Messenger application open while they are doing regular desk work. It seems to lend itself to continuous partial attention (Stone 2007), with users glancing at the application while doing other tasks. In an ethnographic study of Messenger use in an office environment, Nardi, Whittaker, and Bradner (2000) observed that it is relatively easy to read or even originate messages while on the phone or in a face-to-face conversation at one's desk. Avrahami and Hudson (2006), using special software to measure the habits of volunteers, determined that Messenger conversations with colleagues tended to be more intense and last a shorter time than those with friends and family. In the latter case, it is quite common to have intermittent conversations, stretched out over a very long time (Nardi, Whittaker and Bradner 2000).

One of the positive effects of Messenger is to facilitate relationships, hopefully generating social capital (cf. Hall and Widén-Wulff 2008) in the family and workplace. In a South Korean questionnaire survey, the favourite media for students were found to be Messenger, mobile phone and short message service (SMS) messaging (Kim et al. 2007). Messenger was used in expanding relationships with weak ties i.e. they will invite into their circle of buddies people who are members of their social circle but not necessarily close friends. It was seen as a family medium that supports some work relationships also.

The initial background review of the general use of the application prompted questions about its use and usefulness in the academic environment, which the following study attempted to answer.

Methodology

An exploratory ethnographic approach was adopted. The investigation followed a three-stage methodology. The first step was a grounded assessment of emergent issues (Dick 2002) through unstructured interviews in which respondents were encouraged to raise personal issues, concerns and interests around the use of the technology. Representatives of the various facets of university life were selected: students, staff and computer personnel. The student interviewees were a convenience sample. The author simply walked through the computing facility and looked for students who had Messenger open. Users were invited for a free coffee and an interview. Most agreed, but polite refusals were accepted in good part. It was an opportunity to collect data from a cross-section of students from a variety of different disciplines. Interviewees were encouraged to talk freely, but the interviewer would probe whenever a statement aroused his curiosity. A review of the literature had highlighted the topics of mutual helping, times when the application was activated and deactivated, social use and support. Seed questions on these topics were introduced in order to prompt discussion of these categories. Nielsen and Landauer (1993) showed that the emergence of new topics from the collection of qualitative data follows a Poisson curve; after seven interviews it was felt that points were being duplicated and new themes were no longer emerging. The qualitative analysis of both literature and interview data led to an iterative process of grouping themes. These themes provided the basis of a structured data collection tool in the form of a questionnaire.

The second stage was to apply the structured data collection tool in a classroom setting. The questionnaire (Appendix 1) was administered in the autumn of 2006 to the students on the two computing modules on which the author was timetabled. Almost all students present in the two lectures filled in the form, resulting in 53 completed returns. The topics covered in the questionnaire included frequency of use, contexts of use, use for study, social use and the location of the student's home. Most answers were given on a five-point Likert scale.

Having processed questionnaire data, reported use of Messenger was compared with recorded marks in order to help fill in the emergent picture. The literature indicated that Messenger use might reduce success as measured by module marks. Correlation gave suspicious results, later explained by the parabolic nature of the relationship, so the data were graphed. Only statistical data were extracted, from which it was impossible to identify individuals.

Initial interviews

Students

In all, seven student interviews were conducted and transcribed. Only the findings relevant to this communication are reported.

A natural question was whether the student had ever used the application to ask for help with their work or to help others:

Z: Yes, yes … Not so often. Once or twice a week, I think.

R: Mostly I ask my friends in [my home country]. Some of them know quite a bit 'cos normally I'm stuck on computer stuff like 3D modelling and stuff like that, and it's really convenient for me to use MSN.

M: Yep. All the time. Constantly. I would even text somebody and say ‘go on line, I'm stuck’. So they go on MSN and they can help me with it … The [computing] centre is big and it just saves me at lot of time. And you don't interfere with their studying the same as if you went over there. If it was just something easy I would ask over MSN, but if it was something big I'd go over there of course …

Some felt that the application did not lend itself to discussions about work, but was a good way to make them happen: Student G, in a group project, would not discuss work over the application, but would use it to set up meetings. A would not use Messenger to discuss work:

A: No, I don't think. Just to contact them and speak. It's easier to speak when you can have people in front of you and to explain the problem they've got and all that thing. I think it's a better way than via writing and all things.

Students voiced a fear of addiction and lost time. G confessed to being easily distracted into surfing. D had seen friends sucked into wasting time on the computers and was wary of letting it happen to himself:

D: There is a case for people being on MSN and being on it too long. You know, if you allow it to interfere with your coursework here and your work studies here, it can become … like you need to know when to stop and … like if you saw people coming up in your Messenger and you're online. If you just made an attempt to always start off a conversation, you could be on it for always, without even knowing that time passed. Some people let it get a grip of them in a way, like you know so … I don't know.

Both M and R regularly used Messenger to locate friends and invite them for a break or arrange a social activity for later. They were used to friends reciprocating in a similar way:

R: I use it mostly to chat with my friends and keep in contact. It's one of the alternatives to using the phone. Because while I was in the [computing] centre some of my friends in Edinburgh were on MSN. So it's just like ‘OK were going to meet up in a few hours’ instead of having to call each other. It like saves you money as well. It's just easy. It, like, pops up on the screen and says ‘want to meet up at 2 pm outside of school’ and I say ‘yes’. It's really easy and convenient.

Most of those interviewed said they were studying away from home and used the application to maintain relationships ‘back home’:

L: I think it helps people keep in contact with their family back at home when they're away at uni. So it's good for me so I can still chat to my friends occasionally when I'm here.

R: I started studying back home in [my home country] and it was really convenient for me because I moved away from my home city so that's when I first set up an MSN account because it was convenient talking to my friends that lived far from me.

In its curious mix of academic and social elements, the VLCC had developed an atmosphere comparable to that of a library. It was difficult to speak to M to invite her for an interview, because she listens to music on headphones while she works and cannot hear external sounds:

M: I can't work at home – there's too many distractions, so I come in here to work and I have it on all of the time … I've got the Internet and I have everything at home. I just feel better working at uni.

A: We are thirty [compatriots], so we share our ideas, or the way we are working for one subject or another. We try to work together with our flatmates. Yeah we share our work … So we are working in computers or we stay here to work or in the library place.

Student Z spoke of the centre's sheer size; if he wanted to find somebody, he would do it over Messenger rather than stand up and look for them.

Teaching staff

The computing facility is arranged into long tables (‘clusters’) of 12 computers, six either side. Tutorials will typically run over two or four of these tables, running for one or two hours at a time. Teaching staff adopt a variety of policies towards ownership of these: some reserve them exclusively for students on their course; some allow interlopers to come and do their own work if there are spare computers. The lecturers consulted had a variety of approaches. One bans the use of Messenger and email from her tutorial groups completely. Others were more flexible:

LB: I walk round the tutorial group. If they're doing tutorial work, I stop and ask them how they're getting on. If they're looking at the Internet or emailing people, then it's not my tutorial that they're doing. Therefore, unless I have cause to worry about them, I won't have any further contact with them in that tutorial.

How much of their tutorial time – at a guess – does it use?

LG: Some of them not at all, others allow it to dominate their tutorial session. I tend to interfere then, to ensure it doesn't. I ask them how they're getting on with the work which was scheduled for that lab. At that point they usually find it difficult to answer and it causes them to start doing something.

One lecturer estimated that when Messenger is permitted in tutorials, it takes up about 5% of the student's time.

Some lecturers had set tutorial exercises in which students were asked to set up group conversations over Messenger and discuss a set topic. Aside from these exercises none of the lecturers consulted had ever seen Messenger being spontaneously used for work. This observation may not be significant, for in order to actually read a Messenger window one would need to be intrusively close to the screen. Many students have the routine habit of hiding (minimising) their Messenger window on the approach of a lecturer.

There were also anecdotes about Messenger being used for cheating in supervised tests conducted in the VLCC.

Computing facility staff

The manager of the computing facility said that Messenger was not a standard part of the computing provision she offered. Rather, students downloaded and installed the software by themselves. She would prefer that students would not install their own software, but locking down installation rights would make it more difficult for some legitimate applications to operate.

The questionnaire

A 19-item questionnaire (Appendix 1) was administered to the students on two computing modules, one software engineering, the other human–computer interaction (HCI). Below is an analysis of those results relevant to frequency of use, social support, academic support and disturbance of concentration. Percentages are rounded for readability.

Students were asked for what proportion of the time they kept the application activated while they were sitting at a computer, irrespective of use. Some students (7.5%) reported that they never used the Messenger application at all. Almost three quarters (74%) said that Messenger was activated for at least half the time when they were using a computer. About half of the respondents (49%) replied that they had it on all the time. A note on confidence intervals here: to a 95% confidence level, and with this small sample size, this would mean 49% ±13% for a larger population. This small data set therefore only has value in illustrating this modest case-study.

One of the more interesting discoveries of the interviews was the role that Messenger plays in providing social support. A third (34%) of all the respondents reported that they regularly used the application to keep in touch with their family; this number rose to over half (56%) for the 62% living away from home. Most (70%) said that they used the application regularly to keep in touch with friends who were not enrolled at the university and this number did not change significantly for those living away from home. Over a quarter (29%) wished that more people they knew used Messenger.

Students were asked how much they used Messenger to give and receive help in university work: three quarters (75%) have sometimes used the application to ask buddies for help and a quarter (25%) do this regularly. On the giving side, 77% said that they have been asked for help via this medium.

Almost all (85%) have disabled Messenger when they needed to concentrate on work. Most (69%) do this regularly. Fifteen percent said that they worry that they are putting too much time into the application. All but one of the latter fell into the categories of those who use Messenger for most or all of their computer activity. The module marks of this small group of students averaged five percentage points below those of the rest of the sample, so their answers may have been insightful.

The outcomes of the questionnaire indicated that most students used the application for most of the time they were at a computer. It is used by most to keep in touch with friends. Those living away from home use it to keep in touch with their families. The application is used to a significant level for peer support. Nevertheless, most students recognise that Messenger affects their ability to concentrate and will disable it if necessary.

Module marks

One of the aims was to determine the effect of Messenger usage on the educational process. The results of Question 1 in the questionnaire (‘I use Messenger: never, some days, about half the time, most days, always’) were compared with the percentage marks that students gained in some of the assessments. Taking each category from question one, e.g. ‘I never use Messenger’, the average mark was taken of all students giving that reply, over two study modules. In each case, coursework and exam marks were entered separately, yielding a data set of 90 values. The means and standard deviations of the samples are given in Table 1.

Table 1. Summary statistics of the coursework and exam marks used in the analysis.

		Number in module	Number in sample	Sample mean	Sample standard deviation
Software module	Coursework	28	22	73.0	9.5
	Exam	28	19	66.8	14.2
HCI module	Coursework	32	25	53.6	12.7
	Exam	30	24	58.2	13.8
Group project module	Reflection	54	21	47.3	14.1

The results of the comparison are shown in Figure 2. These show that the students who did not use Messenger at all achieved the highest marks. Students in this category come out a full seven percentage points ahead of the next category. The next highest category was that of the students who said that they used Messenger on ‘some days’. The surprising result of this histogram was that students who said that they used the application ‘always’ came out ahead of those who said that they used it ‘most days’ or even ‘half the time’. The histogram appears to be parabolic in nature. This suggests the operation of two different effects, one favouring complete abstainers and the other favouring heavy users.

Figure 2 The averaged marks of two modules compared with student use of Messenger.

Deliberately looking for a data set in which heavy users of Messenger might do well, a set was taken from part of the group project module, in which students were asked to reflect on the experience of being in a group. Again, the student marks were compared with the questionnaire question about general Messenger usage (Figure 3). The sample was restricted to students who had also attended modules to which the questionnaire was administered, so there are only 21 samples in the set. Again, the category scoring highest marks was that of total abstinence. However, it appears that habitual users are also at an advantage, coming out 13 percentage points higher than the light user category.

Figure 3 The marks for a piece of reflective writing compared with student use of Messenger.

Finally, a review was done of those who left the application permanently on and who achieved higher than average marks in the HCI and software engineering modules. This was an attempt to use the existing data to determine why their performance was so good. A slightly higher proportion was regularly asked for help (30% as opposed to 25% for the rest of the sampled population). This may have been simply because they were better than average performers. They were slightly more inclined to disable the application when they needed to concentrate (77% as opposed to 69% for the entire sample). A significant number were living away from home (54% as opposed to 38% of the entire sample – 16 percentage points).

Discussion

It is simplistic to use module marks as a measure of the educational process. Marks are a short-term foreground measure of how well the students are learning useful technical skills. Meanwhile, the process of education, learning to read, write, think and communicate, is a long-term activity which continues in the background. It is hoped that the two are related, but that relationship is not a direct one. Marks may be viewed as correlated with the educational process and also with student engagement. However, under the pressure to produce a large quantity of credit-bearing output, students may be losing the resources which would have contributed to deeper learning (cf. Entwistle 2000).

Many regard Messenger as a distraction. Whereas the literature reports guilt about the time put into Internet activity, only a small percentage reported these feelings in the questionnaire. Nevertheless, in other studies Internet use has been reported to be addictive, and the high take-up rate reported here may be evidence of that. In the interviews, students reported a wariness of being sucked into wasting time. In the questionnaire, most reported disabling Messenger when they needed to concentrate, demonstrating an awareness of the difficulties of continuous partial attention.

The literature confirms that there are those both inside and outside academia that regard Messenger as a waste of the users' own resources and that internet usage may have an adverse effect on student success. Staff reported a low opinion of Messenger's usefulness. It is sometimes banned from tutorials. In the sample reported here, the highest scoring students were total abstainers. In tutorial work, students were observed to keep the application deliberately below the horizon of notice, which may explain why some staff say that they have never seen it used for academic work. The academic environment is still one in which individual achievement is valued highly, so it is reasonable to expect a suspicion of achievement which is not individual. The results reported here do indicate a tension between an independent work ethic and the support of the hive.

There is evidence that Messenger use contributes to academic success. The literature reports that students spontaneously use the application for a variety of academic tasks. Nevertheless, in the interviews here, it was commented that Messenger is better for organising discussions than for having them. In this case study there is strong evidence that heavy users do well. The parabolic nature of the histograms may indicate that it is of different usefulness to different study styles. An alternative explanation might be that heavy users have mastered the application and integrated it into their university lives. It was noted that those heavy users who did well tended to include more students living away from home. An explanation of the latter finding may be that we had a number of excellent advanced-entry French students who brought with them a culture of using the application.

When it was observed that heavy users performed better than some lighter users, it was conjectured that sociable and socially aware students might be drawn to use Messenger as a way to continue their natural networking activities. The second piece of coursework selected for investigation was intended to place socially aware students at an advantage, and thus emphasise this trend. The trend was indeed emphasised, which lends evidence to the hypothesis. Indeed, evidence from the Learning Experience (LXP) project funded by the UK Joint Information Systems Committee (Conole et al. 2006) indicates that students in the UK are now spontaneously using a variety of communication tools to set up communities of practice for mutual support, including advice, resource sharing and improvement suggestions on work.

In the university world many students are adapting to a new learning ecology while simultaneously learning how to live away from home. Support is now recognised as having a far greater impact on retention and academic success than was perceived in the past (Sidle and McReynolds 1999). The literature reports that one of the most popular uses of Messenger is to keep in touch with friends and family. This is confirmed by the questionnaire, in which students reported that Messenger was used regularly to keep in touch with family and friends not attending the university. There was only a slight difference in the extent of this use for students living away from home.

Conclusions

The journey represented by this study started out with the idea that the use of MSN Messenger and similar social networking applications may be a drain on resources, which detracted from student education. The question of whether it is a drain or an augmentation of student resources does not appear to have a clear answer. It was envisaged that Messenger had certain advantages, but that these would be outweighed by the time spent playing with the application. The investigation has revealed evidence that overall, for the population of students investigated, this may indeed be true. Those who abstain from Messenger do seem to do better than the overwhelming majority who do use it. However, an observation of the data has shown that of those students who do use the application, heavy users seem to do best. Having established a link, it should be the subject of a further investigation.

In retrospect, a larger sample size would have enhanced the credibility of the questionnaire phase. The two graphs show reassuringly similar curves for different sets of marks. Nevertheless, a larger sample would have reduced the suspicion that the phenomenon displayed was an artefact of this particular set of students. The exercise was an opportunity to validate the approach, which could be refined in future work.

A further piece of work might be to investigate those heavy users who do particularly well, and examine their characteristics. These students will certainly have acquired knowledge about the effective use of the application, which they might be persuaded to pass on. The dissemination of good practice in earlier study skills modules might lead to a general enhancement of performance among the wider student population. A further potential investigation might be a comparison of Messenger use with learning styles. Topics might include an investigation of what kinds of learners are most and least attracted to the application, and what kinds of learners find it most useful. Indeed, it may turn out that highly socialised, networked students make better learners, whether they use the application or not.

An unexpected revelation encountered during the study is the amount of support that students gain from the application. It is tempting to ignore students' social needs when considering academic advantages. Nevertheless, support has been shown to impact retention and academic success, even though that impact is difficult to measure. The concept of support may be extended to the academic domain too. Many believe that formal or informal peer tutoring is at least as powerful as guidance by academic staff. A study of networking effects within cohorts may yield results that could then be reverse-engineered to enhance retention. Peer support is certainly more available and far less resource intensive in an academic environment.

Acknowledgements

The author would like to thank all those who supported this study in various ways. Anonymous students voluntarily contributed time to providing data for this study. Sandra Cairncross advised on the questionnaire. The editors, anonymous referees, Keith Smyth, Colin Smith and Hazel Hall read and advised on drafts of this communication while it was being prepared.

Appendix 1. Instant messenger questionnaire