Scoping the Potential for Anytime-Anywhere Support Through Virtual Mentors

Abstract

Anytime-anywhere student support is a growing area of investigation as indicated by its inclusion in a range of University policies and academic research. A range of technologies can be applied to support teaching and learning, however they have limitations in providing students with anytime-anywhere support. This article proposes a holistic approach, which blends different available technologies in a Virtual Mentor. A Virtual Mentor is not normally intended to replace a human mentor but can offer support when a human mentor is not available. This new approach is contrasted with currently used technologies to show its advantages.

Keywords:

• Higher Education
• ICT Supported Education
• Mentoring
• Virtual Mentor

1. Introduction

This paper scopes the benefits of a new generation of technology designed to support teaching and learning in Higher Education. We refer to these systems as Virtual Mentors (VMs) as their sole function is to support students in a way that complements human mentors. See [15, 20, 21] for some interesting sources on traditional mentoring of students in Higher Education. The role of human mentors in Higher Education is to provide support to students in a variety of ways including: encouraging independent learning, helping them to understand how they learn and to build self-confidence, encouraging successful interaction with other students in group projects, guiding students to write a dissertation, as well as, in some cases, provide pastoral support.

The academic support provided by mentors can be valuable as students, particularly in their early years of university education, face a wide range of problems which can limit the quality of their experience, inhibit learning and ultimately influence their career direction. Sometimes these problems reduce the quality of the educational experience causing the student to interrupt or even leave education [14]. The transition to a system which instructs in a different form, requiring more independence and critical analysis, is challenging. Added to this, evaluation of academic progress, usually through a number of stressful examinations, can be traumatic and discouraging [27]. Even when a student has acclimatised and is committed and enthusiastic to grow intellectually in such a challenging environment, there remains a sustained three or four year commitment. The quality of the support universities provide to their students makes a difference to retention [19, 25].

The system considered in this paper is directly related to current efforts by academic organizations on improving their services to students. For example, University of Ulster’s policies on induction, transition and retention (see [22, 23, 24]), attempt to enhance the experience of students in Higher Education, especially through the first year when they join a new learning environment. The overall strategy proposed in those policies includes making key information available (eg on their courses, evaluation methods, etc.), to provide extra support to new students (eg explaining the induction process and how to participate in it) and offering students opportunities to meet with peers and members of staff. Other steps include facilitating the acquisition of pedagogical material, the reinforcement of learning skills, and promoting engagement of students with university life.

Such support systems help close any gap that may exist between University expectations and student experiences. For example, the STAR project (Student Transition And Retention) is aimed at increasing student retention, particularly at the important transition point between school and university. The programme found that among the elements that impact on students entering University were social changes, such as the way they interact with students/teachers, challenges in the curriculum, study methodology and assessment. Part of the outcome of this project was the advice to apply the following measures: enhance student experience: induction as a process rather than as an event; well managed group work, field work, and social events; reinforcement of cohort identity (eg, teaching small groups together); student mentoring systems to assist rapid integration; and more contact with fewer staff. For example, [5] show that easily accessible meetings with studies advisers was highly important for students entering university and played a role in their retention while [6] highlights the importance of out of class contact which goes beyond didactic issues. Similarly, [9, 10] highlight the importance of academic advice and the provision for clarification of questions students may have.

These findings suggest student support in the initial phase of Higher Education is appreciated by students. However, typical stages of student progression through university are much more extensive. Figure 1 highlights student experiences that may require extra support from the academic environments. These important aspects of a student academic life cannot be ignored as they contribute to student satisfaction. They are described below and later on it is explained how the Virtual Mentoring system can contribute to support them.

Fig. 1 Progression of student related support elements

Induction is an important stage for students to get to know a new environment when they make the transition from school to higher education. The STAR report suggests that some students may withdraw from University if they do not feel connected with their peers and the educational system they have to interact with. Early signs of such withdrawals may be where, as students progress in their studies, some may feel less inclined to attend class. Therefore a supporting process that can facilitate their engagement with the activities can be useful, not only to facilitate attendance but to ameliorate fears that might otherwise lead to students taking drastic action. Supporting the understanding of specific material and of how it relates to the whole curricula is also part of the process of keeping the student engaged with the learning environment.

Assessment is an important and often stressful aspect of academic life and therefore a system that can help the student to understand its importance, how it is organized and how she/he can approach it will benefit the student’s understanding of the process and improve her/his confidence. Collaborative activities can be extremely important to the effective delivery of some learning material and to some type of learners. Hence support on facilitating collaboration and maximizing its effect is an important element of an educational system. Finally supporting the delivery of teaching material and facilitating the learning of that material by the student is obviously an important aim for any technology deployed in an educational framework.

Ideally, permanent support for each student at all times and circumstances can diminish the impact of the problems described above. However, this is unrealistic for several reasons: 1) the student-tutor ratio is often prohibitive, 2) students can have problems at any time and place making responsive support unrealistic, 3) the time and context where students decide to do study and undertake non-contact work makes tutor availability difficult, and 4) the range of problems and strategies for supporting and alleviating them may be beyond the scope of one tutor or a single team. The focus of this article, therefore, is to explore to what extent this gap between the ideal and the achievable can be supported by technology.

This section provides a brief overview of how our system relates to other pedagogic technologies, so that we facilitate the understanding of why a Virtual Mentor (VM) can advance the state of the art.

A VM uses an avatar [3] to take advantage of spoken-dialogue technology [12] but supported with a Knowledge Base that provides interesting answers to the users. Still, a VM should not be confused with a tutoring system [1], which is a program specialized in a specific technical topic and a set of challenges which are used to encourage and measure learning of that topic.

A VM extends “blended learning approaches” [13] (such as WebCT) which have passive recipients and rely on asynchronous communication. The success of online approaches depend on the availability of the teacher to meet over the internet at an agreed time to answer questions in real-time or connect asynchronously to provide guidance and answer questions.

Our proposal complements a Smart Classroom environment [2], supporting the student on less technical topics. In our project the student can bring part of the learning environment (mentoring element) with her/him wherever s/he is.

VM also differs from the mobile platforms [18] in the type of interaction sustained by presenting a natural interface and reasoning engine that allows the student to interact and obtain feedback and guidance on a variety of academic life related issues.

Next section introduces the research methodology. A data analysis follows in Section 3. Section 4 explains the concept of Virtual Mentoring systems. Section 5 argues that our proposal is different and brings added value to the current offering. Section 6 describes the current status of development of the system and Section 7 provides a summary of the proposal and an analysis of its implications.

2. Research Methodology

This section explains the steps taken during a scoping exercise to determine:

• the perceived value of a VM system;

• whether there was a positive perception about the role and development of VMs on at least part of the student community.

The first steps investigated whether and in which aspects students thought a VM system to be useful. The answer to that question was sought through an optional and anonymous questionnaire.

A decision was made on using questionnaires rather than interviews as the way for data collection [4] as the former were thought to be more time effective (a larger number of opinions can be gathered simultaneously) and also students can feel their opinion can be provided truly anonymously. Given that it was useful to gather as much of their opinions as possible to influence the design of the system, closed questions were used to address specific issues (for example, whether they thought a VM can be useful). Closed questions were combined with open questions (in the form of free text response) to allow students to enumerate the services they imagine such a system could provide. Most of the questions were dichotomous to hasten the completion of the questionnaire.

A questionnaire for human mentors was also designed, the rationale being that their opinion was highly valuable for the development of the system. The vision for this system is that human mentors see the virtual mentoring system as a complement to their role and an optional tool they can resort to when supporting students. The questionnaire for mentors was designed with a different philosophy, as it was important to allow them to expand on important topics as much as they felt necessary, therefore all questions were open.

3. Data Analysis

The questionnaire that was designed for students can be seen in Figure 2. It was passed to two different cohorts of students; one was part of an undergraduate course and the other part of an MSc course. The questionnaire was optional and there were 24 responses from a total of approximately 40 students invited to take it.

Fig. 2 Questionnaire provided to the students to gather requirements

There was a diversity of the students’ ethnicity (approximately half of them come from outside the UK) and gender (approximately 30% Female and 70% Male). The technical background was more uniform as the primary degree of most of them was in Mathematics and/or Computer Science and they were taking a Computer Science related course. It is clear then that all those taking the questionnaire had some degree of high level involvement with technology. It is beyond the scope of this paper to compare the research cohort’s acceptance of VMs with those studying a non-Computer Science related subject but it will be an interesting future exercise. Important issues found in the results of the questionnaire answered by the students are that:

• Question 1 showed 2/3 of students faced situations where they needed support but it was not available;

• Question 2 showed 3/4 of students reported that they believed a VM system would have helped in those situations;

• Question 6 showed about 50% had issues they would not wish to discuss with the human mentor, however 75% would be happy to ask the virtual mentor if anonymity was preserved; and

• Question 9 provided a diversity of topics the students thought the system can help them with. These topics covered: availability, socialization, support to prepare for assessment, easiness of access, facilitation of contact with the human mentor, studies advice, guidance on preparation of assignments, and general guidance.

They provide a good indication that, at least for a cohort of students with a technical background, there is a perceived potential usefulness associated with the VM.

The opinion of two course directors was requested as they are mentors with significant experience in the role (see questionnaire in Figure 3).

Fig. 3 Questionnaire given to human mentors

Their comments were informative and complementary to those provided by the students. The reasons they cited for students’ visits were: students who were struggling to perform as expected or to make decisions due to a poor and/or uninformed previous choice, circumstances that result in poor attendance, non-submission (or late submission) of coursework, non-attendance at exams, poor academic performance, choice of modules, transfer to other courses either during or on completion of current course, problems arising from team work. One mentor observed that at the root of those with the more severe problems were: poor choices of course, work commitments, and the perception that attendance at lectures is unnecessary.

A validation exercise was conducted with students in the first year of the Interactive Multimedia Design (IMD) course. A total of 53 students signed their consent to participate in the exercise. The first questionnaire the students answered prior to seeing the tool was about gathering their opinion on how they thought the system should be and the questions were as in Fig. 2 The answers were statistically aligned with the previous cohort reported previously. In question 9 the most popular/useful/clear answers were: availability and speed of the service”, “Easier access to university services and policies”, and “Information about placement and projects”.

We gathered students’ opinion based on the questions listed in Fig. 4. Approximately 70% of students highlighted 24/7 availability as the biggest asset. Other advantages also cited were the possibility “to ask questions they would not dare to ask a human” (connected to questions 6 and 7 of previous questionnaire), and aesthetical features (remember this cohort of students is enrolled in an IMD course). The least useful features were on presentation features of the version of the Virtual Mentor they were exposed to (eg, ‘answers in speech bubble do not shown up long enough’, and “quality of speech”), and limitations of knowledge of that version. The features which students found most reliable were coverage of knowledge (some of them seem to have the perception that being connected to the web makes the avatar capable of answering on a variety of topics) and confidentiality. Others have diverging views on this and thought the limitations will be on the specific knowledge, which would not be able to compete with a human.

Fig. 4 Validation Questionnaire

These surveys indicate that students and teachers are happy to receive and give, respectively, a greater range of support strategies and highlighted a range of issues of interest, which are being taken into consideration for the system proposed in this paper.

The idea of the VM system is that it provides some degree of help throughout as many of those support steps listed in Figure 1. But as the system is at an initial state of design and development, there is a focus on the three first of those stages listed in Figure 1: induction, retention and attendance. This decision is based on typical Software Engineering principles to develop systems of such complexity by which a system is developed and tested in iterative refinements of increasing complexity [17]. As the project progresses in the future it will move along the list of issues listed in Figure 1 to provide a more comprehensive support.

4. VM: a Virtual Mentoring system

Current “Virtual Mentor” educational systems rely on a human being available to read enquiries and process them either in real-time or off line mode. Our use of the words “Virtual Mentor” however refers to a fully automatic system that can autonomously interact with the student and find ways to assist. The VM and the student still have the option to forward a query to a human mentor but in principle the system will assess the problem and will try to do all it can to provide a solution/advice on its own.

The most ambitious vision of the VM system aims to address different dimensions of student support, from provision of information, to pedagogical assistance and social interaction. An important dimension is the embodiment of social and pedagogical awareness into the system so that part of its expertise is consistent with state of the art knowledge on how a tutor should interact with a student in a typical session.

Figure 5 shows the architecture of the proposed system. A student (portrayed at the top) can interact through a computer which may allow (optionally) an input through webcam and/or microphone providing the VM’s perception. This input will be stored in specific databases (DB) from which a software agent [26] equipped with general reasoning and pedagogical skills and specific information on a particular institution will offer advice. This advice will be stored in internal databases, which the avatar will use to communicate with the student. The student can request that part of that interaction is passed to her/his tutor or may decide to send an email to the tutor so a channel between the system and a human mentor is also available.

Fig. 5 Basic architecture of the Virtual Mentor system

Let us illustrate now how the proposed system can contribute to support students through those issues highlighted in Figure 1. Students can be supported by such a system during induction by helping them find information on how to use the library. The user can conduct a dialogue enquiring the system how to, for example, search for articles published in journals. The Virtual Mentor does not necessarily possess the information itself but must know where the information is. The VM does not compete with the University’s services, instead it acts as a facilitator to the resources the University already has, reinforcing the bridge between the organization and the student. A system like the VM can make use of “web crawlers” which can navigate a specific internet domain searching for specific type of information [7].

After the early stages of engagement between student and University the student will need a different type of support. The focus will change to provide guidance on more dynamic elements of academic life; this can range from information on modules at the beginning of each semester to encouraging reflection on performance across modules in a semester. To provide a service at this level, subtle knowledge such as the student’s learning profile and perception of her/his emotional state can help the quality of guidance provided.

Attendance issues can be considered by the system in an effort to encourage students to keep a regular engagement with each module. This can be supported for example by increasing the understanding of the student of the importance of a specific lecture within the wider content of the module. This may require help from lecturers to provide online information that can be automatically identified and used by the VM system.

The other supporting dimensions mentioned in Figure 1 (understanding, assessment, collaboration and supporting teaching and learning) can be facilitated by the VM by using intelligent tutoring, social networking facilities and specific information on the modules taken by a student.

A VM with the architecture described above, and the blending of different existing technologies amalgamated with new software that can interact with a student to identify needs and provide meaningful feedback, provides a more holistic approach than the any current offering provided to the students. The next section explains precisely why this is the case.

5. How our system compares to other approaches

In this section the main features of our proposal are compared to the features of systems listed in Section 2. First it is clear that the system exceeded the idea of an avatar, a spoken-dialogue or a vision system as our focus is on creating a holistic system that uses these three elements as part of the interaction module of the system. For example, the system presents itself to the user, through an avatar, with the purpose of having a more human friendly interface. After this initial interaction with the user, spoken-dialogue and image processing technology will gather rich information from the student in a more natural way. That is, by using natural language understanding and vision, the system will gather important cues on how the student feels in a way similar to the way a human would. Naturally it is clear that there is a high level of complexity in achieving this. Still, progress made recently in related areas and mentioned in Section 2, is encouraging and it is an interesting area of research worth pursuing to investigate how much and how effectively this can be achieved.

Tutoring Systems do not necessarily have the interface (avatars with spoken-dialogue and image processing capabilities) required for a virtual mentor. Typically researchers in the area of Tutoring Systems have not explored these types of interfaces as a strong feature or a defining characteristic for their systems. The most important issue from our perspective is that many current systems are focused precisely on tutoring whilst for a virtual avatar the focus is on mentoring. Providing knowledge and coaching on technical issues can be part of the interaction but is not all of it. Mentoring will also try to engage with the student in relation to topics, which are not necessarily technical, for example, studying methodologies, sources of information within the University, specializations and optional modules, academic performance and vision of the student of himself before and after graduation.

Our system will bring more than “online assessment and feedback”, as one problem with email is that feedback may not be delivered at the time when it was most needed. Instead the lecturer provides feedback when available and then the student will read it when convenient. Online feedback remedies that because it implies a previous agreement to meet at a particular date/time through an online connection so both student and teacher interact in real-time. The problem here is that it demands the teacher to be available when the student needs help which is not always feasible.

It significantly extends “blended learning approaches”, which are passive recipients of well organized information and rely on either asynchronous communication (the student sends a query which will be answered by the tutor at a later stage or will depend on the teacher being on line for an answer in real-time). The intelligence of the system depends again on when the teacher is available. A VM will have embedded intelligence, which will help the student beyond the document repository and the offering of a channel to post a query to a human mentor. It should also provide meaningful help anywhere/anytime, particularly on occions where the traditional means are not available.

It differs from a Smart Classroom as a student has to physically go or connect to the classroom environment to benefit from it. In our project the student can bring part of the learning environment (mentoring element) with her/him wherever she/he is.

It differs from the mobile platforms in the type of interaction sustained with the student which goes beyond efficient methods to access information but also present a complex interface and reasoning engine that allows the student to interact and obtain feedback and guidance from the system on a variety of academic life related issues.

Fig. 6 Form to set up the set of rules that provide the inference engine the system uses to answer queries

6. Current State of Development of the Prototype

The VM system is currently a prototype development. The project has benefited from a development fund grant provided by the Joint Information Systems Committee (JISC), supported by the Higher Education Academy (HEA). The system has been implemented in a combination of PHP and a specialized system that allows the definition of an avatar, the later provided by a commercial product offered by Media Semantics.

The intelligence of the system is limited and consists of a set of rules that defines what types of queries it can process and the answers associated with them. Figure 6 shows the form that can be used to define such a set of rules. The quality of the service is dependent on the coverage of those rules. For the time being that is enough to illustrate the basic parts of the system. Although that is naturally an aspect of the system that can and should be improved, it is enough at the prototype level to illustrate the basic functionality of the system.

The current version of the system has two main modes of assistance. One is guided by the VM, for example, a general introduction is offered and at the end options are offered to the student on more topics that can be explained by the system. The user can type some of the words that have been offered and follow that guided trail of explanation or depart from it with a fresh question which the system will try to answer by searching answers related to the keywords included in the question. The knowledge of the system is proportional to the rules in the Knowledge Base. As expected, the VM does not know about absolutely everything, and when it does not have a meaningful answer it will provide an answer which can entice further questions. We expect to complement this initial knowledge-based with a more intelligent-based approach which benefits of areas like Machine Learning and reasoning [16, 26].

The interface of the system is shown in Figure 7. Several features of the avatar can be personalized by the user. In contrast, the set of rules has to be maintained by the central development team and is the same for all students accessing it.

Fig. 7 A session with the Virtual Mentor

There are still several fundamental elements to be explored in this system that have been left untouched during these first versions of the prototype. On one side there is the pedagogical dimension. Students see it as another option while some human mentors see it as a remedial option that can be helpful when they cannot address specific problems. This can lead to a misuse of the system and information has to be put in place to ensure all sides share a view on when and how the system is to be used.

Human computer interaction can play an important role in increasing the understanding of the cognitive and emotional state of the student through recognition and interpretation of face gestures and body language [8, 11]. This visual perception, together with the understanding of the student through natural language, can greatly improve the student’s engagement with the system. Cultural awareness and personalization facilities are also relevant. Another important element to address at all stages of development of the successive prototypes is privacy. The student has to be adequately empowered to decide what information can be shared outside the virtual mentor, with whom and when.

The current focus is on increasing the knowledge of the system in the areas chosen to support students, the intelligence of the system to provide good guidance and on increasing the flexibility of the interaction by allowing spoken and visual input/output.

7. Conclusions

Human mentors are irreplaceable, but they may not always be available when and where the student needs help. A virtual mentor can assist students in some of those cases when a mentor is not available but they need support.

The complexities of the cases a VM can deal with efficiently are still to be fully explored and the complexity is such that clearly it could take years to develop the system to a comprehensive level of functionality. The extent of support provided depends on many factors, including the type of technology deployed. This article focused on the initial development, which is more involved with the scoping of the technology, the early detection of problems (and potential “bottlenecks”) as well as the identification of opportunities and potential areas of application.

There is an enormous potential in this concept and different areas can be progressively covered as the VM system is enriched with a more sophisticated internal logic to support pedagogical advice and more powerful sensing devices (eg, through spoken dialogue and image recognition) to understand more accurately the student’s needs. For a system of this type to provide effective assistance it has to be capable to perform an intelligent analysis of the information provided by the student (description of the query and its context). It also requires the understanding of implicit, subtler, messages (eg, tone of voice, face expressions, and body language), which may help the system to be more sensitive and sensible in providing its answers. The system may need to query the students on a number of issues before clearly understanding what the problem is, it should therefore be able to organize and sustain a dialogue with this aim in mind (eg, to clarify a question).

Other challenges are related to the way the system engages with the students and how the advice that it provides is consistent with the educational environment the student is immersed in. It will be interesting to investigate: how successful is the communication between the student and the VM, how useful the advice is perceived to be by the students, how the system can contribute in providing individual and global views of specific students’ and cohort’s perceptions of a module and how that can lead to improvements in pedagogical content delivery. These provide a very interesting intersection amongst pedagogical/social elements and technology deployment, which can be investigated by teams of the Social Sciences and Engineering faculties of this university and also provide fruitful interaction amongst other different units of universities across the UK.

A prototype has been developed, which provides a platform for experimentation. This platform can provide an interesting opportunity for the academic environment to increase the options to support students, a valuable and flexible tool for students, and a stimulating challenge for researchers investigating the way technology can be used to improve Teaching and Learning.

Acknowledgements

The content of this document has been enriched with various colleagues of the University of Ulster, in particular, Vicky Davis, David Glass, Sarah Maguire, Peter Nicholl and Ian Taylor.

Research in this project benefited with funding from JISC and the Centre for Higher Education Practice of the University of Ulster.