A Web-Based Environment to Support Online and Collaborative Group Recommendation Scenarios

Abstract

Today there are many recommendation scenarios that involve groups of interrelated users intending to participate in a group activity. Though there have been attempts to establish group recommendation scenarios, most of them focus on off-line environments. In this article, we present a novel web-based environment that supports online group recommendation scenarios. Specifically, we propose a COllaborative Advisory CHannel for group recommendations called gCOACH. First, we present the conceptual architecture of gCOACH, which is domain independent and is a web-based environment for enabling the interaction of users from anywhere. Second, we introduce the multiple interaction modalities of this environment developed to communicate, coordinate, and persuade the group in a case-based group recommender. We demonstrate its usability through a live-user case-study.

INTRODUCTION

Recommender Systems (RSs) help users search large amounts of digital content and services by allowing them to identify the items that are likely to be more attractive and useful (Ricci et al. 2011). RSs are playing an increasingly important role in many online scenarios (Wang, Wu, and Chou 2010), for example, e-commerce services are one example of these scenarios (Resatsch et al. 2007). In particular, RSs have the ability to provide even the most tentative shoppers with compelling and timely product suggestions. Moreover, groups of interrelated users can take advantage of many recommendation scenarios such as movies, trips, restaurants, and museum exhibits. But to date, much of the research in the area of RSs has focused mainly on recommending products to individuals rather than groups of people intending to participate in a group activity (O’Connor et al. 2001).

These types of scenarios have motivated recent interest in group recommendations, and, to date, a variety of early-stage systems have been developed in domains such as group webpage recommendation (Pizzutilo et al. 2005), recommending vacations or tours to groups of tourists (McCarthy, McGinty, et al. 2006a; Jameson and Smyth 2007), recommending music tracks and playlists to large groups of listeners (Crossen, Budzik, and Hammond 2002; McCarthy and Anagnost 1998), and recommending movies and TV programs to friends and family (Yu et al. 2006).

The role of the group recommender system (GRS) is to make suggestions that reflect the preferences of the group as a whole, while offering reasonable and acceptable options to individual group members. In this regard, a key aspect of group recommendation concerns the way in which individual preferences are captured and stored in the user profiles. These user profiles are combined to recommend a product (or a set of products) for the group at the completion of the group session. Though there have been attempts to establish group recommendation systems, most of them focus on user profiles extracted from off-line environments. Taking into account the growth of activities and environments that offer group interactions (e.g., virtual spaces or ubiquitous personalization services), we consider that group recommendations in online environments are becoming commonplace. When recommendations are made for a group of online users, new challenges and issues arise for providing compelling item suggestions as well as new interaction mechanisms to keep the members of the group aware and to facilitate communication among them.

In this study we consider a web-based environment that supports online group recommendation scenarios. The proposal is called COllaborative Advisory CHannel for group recommendation (gCOACH). In particular, gCOACH aims at being an online framework that facilitates group interaction and communication among members. For this reason, gCOACH uses a conversational case-based recommender (Smyth 2007), which is a form of recommendation that is well suited to many product recommendations. It uses the description of the product base in terms of a complete set of features instead of using the preferences (provided mainly off-line) from other users to make a recommendation, as occurs in collaborative filtering recommenders (Schafer 2007). Conversational recommenders have proven to be especially helpful for users with ill-defined needs and preferences. Notice that recommender systems can be distinguished by the type of feedback that they support; examples include value elicitation, ratings-based feedback, and preference-based feedback (Smyth and McGinty 2003). Specifically, we use a form of user feedback called critiquing (McGinty and Reilly 2011), by which a user indicates a directional feature preference in relation to the current recommendation. Feature critiques typically take the form of a directional or replacement. Directional critiques effect an increase or decrease over numerical feature values. For instance, a less expensive camera when the price of the current recommendation is $300 implies . Replacement critiques allow for the substitution of any value for a nonnumeric feature. For example, a different manufacturer if the current recommendation is a Sony camera means . Critiquing is a form of feedback that strikes a useful balance between the information content of the feedback and the level of user effort or domain expertise that is required.

In summary, the main contributions of this article are two-fold. First of all, gCOACH not only elicits users’ preferences in the form of critiques during the session in an environment that is domain-independent, it also provides multiple interaction modalities and interface components to maintain awareness of members’ decisions and to provide suggestions for them. Second, we evaluate the usability of the proposal based on a live-user study.

The rest of this article is organized as follows: the next section reviews related work on group recommender systems; “gCOACH: Group-Based Collaborative Advisory Channel” describes our proposed gCOACH framework; Next, “Evaluating the Usability of gCOACH” discusses the usability of the proposal; the final section gives conclusions and future work.

RELATED WORK

Most of the research on group recommendation investigated the core algorithms used for recommendation generation. Two different strategies have been mostly used for generating group recommendations: aggregating individual predictions into group predictions (aggregated predictions; Berkovsky and Freyne 2010; Masthoff 2004) or aggregating individual models into group models (aggregated models; De Silva, Moessener, and Carrez 2009; Masthoff 2011; McCarthy, McGinty et al. 2006b; Senot et al. 2010). Differences among these strategies are in the timing of the data aggregation step. Apart from the recommendation generation, group recommender systems can be distinguished based on the different features used in the design of the GRSs (Jameson and Smyth 2007; Masthoff 2011). In this article we are interested in group recommenders that work in an online environment. For this reason, we pay attention to those features that heavily influence preference elicitation.

Preference elicitation refers to the manner in which information is acquired from users. For example, preferences may be acquired by asking users directly (explicit preference elicitation) or by inferring their preferences from their actions and feedback (implicit preference elicitation). In the case of the former, systems such as the Travel Decision Advisor (Jameson 2004) and PolyLens (O’Connor et al. 2001) both acquire preferences by asking users to specify them explicitly, either in the form of preferred features or item ratings. In contrast, group recommender systems such as FlyTrap (Crossen, Budzik, and Hammond 2002) and Let’s Browse (Lieberman, Dyke, and Vivacqua 1999) acquire preferences by monitoring a user’s interactions. Specifically, we consider that preference elicitation is also related to (1) the way a user interacts with the GRS, (2) the types of domains that the GRS can work with, (3) the outcome of the GRS, and (4) the group size.

First of all, considering user interaction, most of the approaches use an off-line interaction with the users. That is, users provide some feedback and the GRS provides a recommendation without further user interaction with the system (e.g., O’Connor et al. 2001; Garcia et al. 2012). However, in an online interaction, users are active and they are engaged in a collaborative session. For example, Travel Decision Forum (Jameson 2004) and Collaborative Advisory Travel System (CATS; McCarthy, McGinty et al. 2006b; McCarthy, Salamo et al. 2006b) allow this kind of interaction. Our proposal, gCOACH, follows this vein by supporting online group recommendation scenarios.

Second, according to the domains that GRS can work with, nearly all GRSs are defined for a specific domain. For example, a recipe recommender (Berkovsky and Freyne 2010), CATS (McCarthy, McGinty et al. 2006b), and Travel Decision Forum (Jameson 2004) deal with the tourist domain, and Yu et al. (2006) propose to recommend movies and TV programs to friends and family. There are a few proposals that are domain-independent (Garcia et al. 2012) but they use an off-line environment for group interaction. It is worth noting that gCOACH is domain-independent, too, and, as said above, it allows online interaction with the users.

There are habitually two forms for presenting the outcome of the GRS: a single recommendation that must be a successful selection for the group (e.g., McCarthy, McGinty et al. 2006b;) or a list of items ordered according to the preferences of the group members (e.g., O’Connor et al. 2001). There are some GRSs that generate a different outcome, for example, (Jameson 2004), instead of returning recommendations, they return a list containing the group preferences. Our proposal uses a conversational case-based recommender that aims at guiding the user over the search space by using critiquing as feedback. For this reason, it is more convenient to use a single recommendation instead of a list of products ordered. However, it is remarkable that gCOACH defines two areas for keeping a list of recommendations. The first is devoted to storing the preferred products of the group and is called the stack area. The second area receives and stores suggestions in a proactive way. These suggestions might come from the GRS itself or from other group members.

Finally, not all the GRSs are capable of working with any group size. Some GRSs are limited to small groups of two, three, or four users (Lieberman, Dyke, and Vivacqua 1999; McCarthy, McGinty et al. 2006a), and some others have no limitation on the group size (Yu et al. 2006). In gCOACH, we can define any group size, although it is true that small groups will benefit more from the environment than large groups.

gCOACH: GROUP-BASED COLLABORATIVE ADVISORY CHANNEL

This section presents the gCOACH framework, which supports online group recommendation scenarios. This framework allows several users to participate in a group activity that involves searching for a product for the whole group. First of all, “Conceptual Architecture of gCOACH” presents the conceptual architecture and it details, in depth, each layer. Next, the interaction modalities defined in gCOACH are shown in detail in “Interaction Modalities in gCOACH.”

Conceptual Architecture of gCOACH

The conceptual architecture of gCOACH is a web-based environment developed on a client-server model for enabling the interaction of users from anywhere. Figure 1 shows the conceptual architecture of our proposal divided into three main layers: a Space Client, a Space Server, and a Space Recommender.

FIGURE 1 Conceptual architecture of gCOACH framework.

Specifically, the Space Client is devoted mainly to the view layer. It is in charge of capturing the interaction modalities and the communication among members of the group. The Space Server acts as a controller and it is responsible for connecting the users with the recommender. Note that Figure 1 depicts some events among the Space Client to the Space Server and the actions among Space Server to the Space Recommender. Finally, the last layer contains the model of the whole architecture, which includes the Group Conversational Recommender algorithm (GCR)—which contains the individual user model (IM) of each member of the group and the set of products available to recommend (i.e., known as case base (CB) in conversational recommenders)—and the Group Management Module (GMM). The following sections describe in depth each of these layers.

Space Client Layer

This layer is a space that offers interaction, collaboration, and awareness among users. This space allows concurrent connection of one or more users (i.e., note that Figure 1 displays only two users for helping in the reading comprehension). Users can interact with a recommendation object, with a stack object, with a suggestion box object, or with an awareness object. As Figure 1 details, the stack object is shared among users whereas the remaining objects are independent of each user.

A Recommendation Object (RO) is an object that represents a product in the interface. It contains product information and the description of all features of the current recommendation, interactive elements to perform critiques, and collaborative elements to perform collaboration and communication among users (e.g., sending a suggestion to another user). For example, in a ski domain, the product information includes one or more images of the resort and the accommodation, each feature is described as a pair description-value (e.g., price-$1000), and the interactive critique elements available for each of the features (greater than, lower than for numerical features such as price, or different if the feature is nominal such as country). Note that the recommendation object is domain independent because its elements are general enough to encompass a wide range of domains.

A Stack Object (SO) is an object that contains all the products that are of interest for the group members. It is a common object for the group and it is updated each time a user operates over it. Users add or remove their preferred products and the whole group will be aware of the main preferences of each group member. Apart from this, SO also stores the level of satisfaction of the whole group for every particular product. This information helps users reach a consensus.

The Suggestion Box (SB) object contains a list of products that have been suggested to the user by any one of the members or by the recommender itself. This object also stores the level of satisfaction of the user with regard to the suggestions received.

The Awareness Object (AO) includes a list of products with the information of the current product view of each of the members of the group. This object keeps the group aware of the current navigation of each group member and the level of satisfaction of that particular user with regard to the product he is currently viewing.

The user’s interactions in the Space Client layer trigger five types of events that are sent to the Space Server. These events are: the CritiqueEvent, the SuggestionEvent, the PreselectedEvent, the ThrowWasteBasketEvent, and the FinishEvent. The CritiqueEvent occurs when the user performs a critique by touching the critique elements on the RO. The SuggestionsEvent is activated when the user wants to collaborate with another user by performing a drag and drop of a product from RO to SB. With this event, the user adds a product to the list of suggested products of a particular member of the group. The intention of the sender is to provide one recommendation that satisfies their requirements and to draw the attention to it of another group member. To the contrary, instead of drawing attention to one member as is the case of the SuggestionEvent, the PreselectedEvent is aimed at focusing the attention of the whole group. It occurs when the user performs a drag and drop from RO to the list of products in SO, denoting that this product is of user interest, and it is being stored as one of that user’s main preferences. During the recommendation process, users change their minds and maybe one of the previously preferred or suggested products that were of interest are no longer preferred. To enable an update of a user’s preferences, the ThrowWasteBasketEvent lets the user perform a drag and drop from SB or SO to the waste basket area. Finally, there is an event for finishing the recommendation process. This event is called FinishEvent and it occurs when the user wants to do the final selection of the desired product from the list of products in SO.

The Space Client layer also receives two events from the Space Server layer: the DisplayRecommendationEvent, which displays a new product recommendation on the RO, and the UpdatingSharedAreasEvent, which updates the shared areas (SO, SB, and AO) with the collaborative interactions from other users.

Space Server Layer

This layer is responsible for the communication between the Space Client and the Space Recommender. Basically, it has three components. The first component is the Users Management Module, which stores and manages users’ information such as user identification and statements (critiquing or sending suggestions), and the RO the user is interacting with. The second component is the Content Management Module, which is responsible for updating the RO visualization (i.e., product image and features values) in each recommendation cycle. The third component is the Communication Management Module, which maps a user’s events from the Space Client to recommendation actions in the Group Conversational Recommender module and, in reverse, recommendation actions to user events.

In the following, we present how the user’s events are mapped to the actions into the Space Recommender. Note that “Space Recommender Layer” details in depth the action in the Space Recommender.

1. The CritiqueEvent is converted to the CritiqueAction, as described in Signature 1. It contains the identification of the user who performed the critique, userId; the current recommended product, productId; the feature that receives the critique, featureId (e.g., price); the type of critique, typeCritique (i.e., greater than, lower than, or different); and the critique value, critiqueValue (i.e., the current value of the feature that has received the critique).

   (1)

   For example, critiqueAction(Red, HotelGarniStGeorg, price, , 419) describes that Red user sends a critique about Hotel Garni St Georg for obtaining a recommendation product with a price higher than $419. As described in “Space Recommender Layer,” this critique is later stored in the individual user model (IM_j).

2. The SuggestionEvent is transformed to the SuggestionAction (see Signature 2), which contains the identification of the user who performed the suggestion, userFromId; the user to whom the product is suggested, userToId; and the suggested product, productId.

   (2)

   For example, SuggestionAction(Red, Blue, HotelGarniStGeorg) is showing that Red user sends product Hotel Garni St Georg to the Blue user.

3. The PreselectedEvent is mapped to the PreselectedAction, as described in Signature 3. The PreselectedAction contains as parameters the identification of the user who performed the preselection action, userId; and the product added to the list of products in SO, preSelectedProductId.

   (3)

   For instance, PreselectedAction(Blue, HotelGarniStGeorg) indicates that Blue user wants to add Hotel Garni St Georg to the stack.

4. The ThrowWasteBasketEvent is converted to the ThrowWasteAction (see Signature 4) whose parameters are the identification of the user who performed the throw waste basket action, userId; the product that is sent to waste basket, productDeletedId; and the area—SB or SO—that contains the product, areaId.

   (4)

   For example, ThrowWasteAction(Blue, HotelGarniStGeorg,SO) removes Hotel Garni St Georg from the stack entered by user Blue. A product can be removed by several members of the group. This is necessary because users select one of their preferences in the stack as the final product chosen in the recommender before finishing the recommendation session.

5. The FinishEvent maps to the FinishAction described in Signature 5, which contains the identification of the user who performed the finish action, userId; and the individual product selected, productSelectedId.

   (5)

   For example, FinishAction(Red, HotelGarniStGeorg) means that Red user has chosen Hotel Garni St Georg as their preferred product. Thus finishing the recommendation process for her or him.

Now we depict the actions received from the Space Recommender layer in the Space Server layer and how it sends to the Space Client layer.

1. The Recommendation action is transformed into the Communication Management Module to the DisplayRecommendationEvent previously introduced in “Space Client Layer.” As shown in Signature 6, it contains the identification of the user who performed the critique, userId; the new recommended product, productId; and a list of features (i.e., the value for each product features), featureValues; Later, from the Communication Management Module, the mapped event is sent to the Content Management Module, which is responsible for updating the RO visualization through the UpdatingSharedAreasEvent (i.e., product image and features values) in each recommendation cycle.

   (6)

2. The UpdatingSharedAreas action is connected to the UpdatingSharedAreasEvent. The definition of this action is described in Signature 7, which shows that the updated area (i.e., SO, SB, or AO), areaId; the preselection, suggestion or awareness product identifier, productId; the entity that generates the update (i.e., the update action may be triggered by a user in case of the SO and SB, or by the system in case of SB and AO), entityId; and the compatibility of the current product with the individual user model (IM_j), compatibility.

   (7)

Space Recommender Layer

The Space Recommender layer is composed by two different modules: the Group Conversational Recommender (GCR) Algorithm and the Group Management Module (GMM). Next we describe each module in detail.

First, the Group Conversational Recommender Algorithm contains a case base (CB) of products, a set of individual user models (IM) (i.e., one for each user in the environment), and a group preference model (GM). Specifically, the case base contains a set of products or items for recommendation in which p_i is the ith product. Each product is described with a set of a features (e.g., price, duration, or location). Each user is associated with an individual preference model, IM, that is made up of the critiques that they apply throughout the course of the recommendation session. We denote the individual preference model of a particular user j as IM_j, where each element in IM_j is a single unit critique. As new critiques are made by a user, their preference model is updated as described in (McCarthy, McGinty et al. 2006a; McCarthy, Salamó et al. 2006a). This involves the addition of new critiques but may also involve the removal of past critiques if they conflict with, or are subsumed by, the most recent critique. In addition, a GM is also maintained by combining the individual user models and associating critiques with the users who contributed them. The GCR algorithm is a module that performs the recommendation process, controls data access (massive products with diversity of features), and updates group and individual user models during the recommendation process.

Briefly, the recommendation process starts when the user is inside the Space Client and performs one critique about a product feature through a Critique Element displayed on a Recommendation Object; this critique is sent to the Group Conversational Recommender (GCR) through the Communication Management module placed on the Space Server. Once received, the CritiqueAction in the GCR algorithm updates the IM_j and the GM with the new preference and it selects the next recommendation from the full set of products in the CB. The recommendation is generated based on a quality score that considers both the similarity with the previous recommendations and the compatibility to the individual and the group models, as described in McCarthy, Salamó, et al. (2006a, b). In this work we have used the compatibility score defined in Reilly (2004), but it is possible to use a different score in the framework, such as those based on reinforcement learning (Salamó and Escalera 2012; Salamó et al. 2005). Next, the Group Conversational Recommender module sends a new recommendation to the Group Management module. This module, using both the Communication Management and the Content Management modules, starts a Recommendation action to be displayed on the RO of the user who performed the critique.

Second, the Group Management Module (GMM) coordinates the performed users’ interactions using the collaborative elements in the Space Client. In particular, GMM contains a set of users U with all the information related to each user and the stack that stores the preselected products. Moreover, for each member of the group, GMM stores the personal information of the user, the current recommendation (i.e., the last product recommended by GCR to the user that is displayed currently in the RO), the individual selection of that user when she or he finishes the recommendation process, the awareness that contains the current product view of each user and the compatibility to this user, and the suggestions that are a list of products received either from others or proactively from the recommender. GMM receives several actions from the Communication Management Module: (1) the SuggestionAction stores the product suggested to the list of suggestions of the receiving user; (2) the PreselectedAction adds a new product in the stack; (3) the ThrowWasteAction removes the selected product from the stack or the list of suggestions; and (4) the FinishAction finishes the user session. Note that GMM coordinates users’ interactions. The particular implementation of the interaction modalities for communicating and allowing suggestion among users are described in more depth in next section.

Interaction Modalities in gCOACH

Figure 2 depicts the interface screen shown to each user in his or her browser. The example interface shown focuses on a skiing package domain. It is divided into several areas, each with a specific interaction modality (i.e., individual or social) and functionality. Note first that each of the users is represented by a color (look at the top of the figure). In the example, the user is red. This color is used in the interface to denote to whom belongs each of the objects in the interface (i.e., RO, SO, SB, and AO).

FIGURE 2 The main gCOACH interface with a skiing package view.

The first area, denoted with number 1 in Figure 2 represents the RO in the conceptual architecture shown in Figure 1. This area is devoted to individual interaction, which has three different views: category view, subcategory view, and product view. In gCOACH, the user interaction starts at a category view, which focuses the user on a specific category of the products (in our case, we use the location in the domain used in the experiments, see Figure 3a). Once a category is selected, the users move to a subcategory that, in the domain analyzed, corresponds to a resort view (see Figure 3b) where they can see all the available hotels. If the user selects one hotel, he or she arrives at a specific skiing package, as depicted in Figure 2.

FIGURE 3 Due to space limitations, we show only the individual interaction of the two views.

Here, in the product view, a product is described in terms of its features and the particular value for each of them. Additionally, each of the features contains one or two buttons for performing critiques (i.e., these are the critique elements in Figure 1). The user is able to make a critique,¹ which affords the user an opportunity to provide informative feedback. The user feedback represents the CritiqueAction in the Space Server layer (see “Space Server Layer”). This feedback is introduced to the IM and the GM. Next, the GRC algorithm uses this informative feedback about the user’s taste and it answers this action by replacing the product displayed with a new recommendation that better matches with the preference expressed (i.e., the RecommendationAction, which maps to the DisplayRecommendationEvent in Space Server layer, described in “Space Server Layer”). Currently, the GRC algorithm implementation is the one proposed in McCarthy, Salamó, et al. (2006a, b). Furthermore, when individual users arrive at a particular product recommendation that satisfies their requirements and they wish to draw it to the attention of the other group members, they can do this by performing a drag and drop and adding it to the suggestion box of another user or to a stack area, which is a social interaction modality.

The second area in the gCOACH interface is for keeping awareness among members of the group — see AO in the conceptual architecture and look at number 2 in Figure 2. This area is refreshed by the system through the UpdatingSharedAreas action, which maps to UpdatingSharedAreasEvent formalized in “Space Server Layer.” In addition, this area contains a list of colored boxes, each representing a group member. Each colored box shows which product is currently being looked up by each user. Users can browse this product by performing a click on it. The colored box contains a 0–5 hearts ranking that represents how compatible the product is to the user who is currently looking at it. The ultimate goal of this heart ranking is to know which products the users are currently interested. With this goal in mind, this area also allows the user to make suggestions about the current recommendation in area 1 to a specific user by doing a drag and drop of the product into the target user box. Then, the suggested product will appear in the suggestions box of the target user. These collaborative actions correspond to collaborative elements in the conceptual architecture (see Figure 1).

The third area is the suggestion box, depicted in Figure 2, number 4, which is represented as SB in the conceptual architecture as shown in Figure 1. These suggestions may come from any one of the group members (i.e., the SuggestionsAction described in “Space Server Layer”) or as a result of a proactive suggestion of the recommender algorithm (i.e., the UpdatingSharedAreas action, previously described in “Space Server Layer”). The GRC algorithm suggests a product to the whole group when one or more cases exceeds a certain critical compatibility threshold with respect to the GM. As in the previous area, each product is identified by the user’s border color and shows a few features including a compatibility ranking by the current user. Furthermore, the option of clicking on the product to take a look at it in the area of individual interaction is also available.

Another social interaction modality is the stack area (shown in Figure 2, number 5), which represents the SO in the conceptual architecture. It serves as repository of particular holiday recommendations the user is interested in and it is also useful for drawing the attention of the other group members to a particular product (i.e., the PreselectedAction described in “Space Server Layer”). The stack stores summaries of the user’s recommendations, and displays compatibility information relating to group compatibility. Each product recommendation appears boxed with the color of the user who added it to the list and shows a summary of its features (in the skiing domain, hotel name, resort name, kind of resort, number of stars, and price). In addition of this content, a measure of compatibility between the current user and the item appears through a visual ranking of 0–5 hearts (0 hearts means a nonsuitable product for the current user and 5 hearts means a perfect item to choose). In this area, the display is done through the UpdatingSharedAreas action shown in “Space Server Layer.” At any time, when users detect an attractive product in this area, they can open it in the individual interaction area to examine it by performing a simple click on it.

We consider an area for removing products that the user is not interested in any longer; this area is called the waste basket area (see Figure 2, number 3). It is used for the disposal of products from the suggestions box area or the stack area. This functionality is activated when the user performs a drag and drop from one of these areas to the waste basket area (i.e., the ThrowWasteAction described in “Space Server Layer”).

Finally, Figure 2 shows at the top of the interface a Finish session button, which can be used by the user to finish the session. At this point, the user receives a new webpage with the information of the products that she or he has added to the stack. The user has to choose one of them as the final decision product. Once all the users have selected their final decision product, an automatic consensus (Salamó, Mccarthy, and Smyth 2012) is performed and a product is sent to each member as the final chosen product.

EVALUATING THE USABILITY OF gCOACH

In this section we evaluate the usability with real users. Given the high fidelity of our prototype, we used the summative usability evaluation method, which focuses on gathering both qualitative and quantitative data (Bowman, Gabbard, and Hix 2002).

Setup and Methodology

We recruited 44 participants, diverse in features such as age, gender, computer skills, and experience in web-based environments. These participants were divided into groups of four heterogeneous participants for each test. The test was performed using a SKI domain that contains 153 European skiing holidays described by 41 features related to the resort (e.g., country or transfer time) and the accommodations (e.g., rating, price, or restaurant facilities). The test was conducted by a moderator and an observer. Users were requested to join in a group using an initial webpage of the interface and to perform a search task of their favorite ski vacation. The test protocol consisted of four stages:

1. Pretest interview: In this stage the moderator welcomed the user, briefly explained test objectives and asked about their previous experience with ski vacations and recommender systems.

2. Training: During this stage users were freely navigating on the web interface of gCOACH. They were asked to locate a predefined product using individual or social interaction modalities. The training stage finished when users discovered this product.

3. Test: In this phase users performed, without guidance, a test task that consisted of selecting a product that best satisfied the group preferences for going skiing together. To this end, users were asked to navigate, communicate, and provide suggestions with the aim of finding a consensus among the group to purchase a product. However, users were free to finish the search process once they found a product that best satisfied their preferences. Among the products in the stack, the user selected the preferred one. When all users finished the searching process among the group-preferred products (one for each user), GCR recommende the one that best satisfied the group. This product was shown as the final product for the group. During the task, a computer recorded the test session and the observer made annotations.

4. Posttest questionnaire: Users were asked to fill out a Web form that contains a satisfaction questionnaire consisting of 10 questions (see Table 1) that users answered using a five-point Likert scale, where 1 corresponds to “strongly disagree” and 5 to “strongly agree.”

After the test, we analyzed the posttest questionnaire to extract relevant data concerning usability. The next section describes this evaluation analysis.

Analysis of Usability

To collect user satisfaction measures we designed a posttest questionnaire, depicted in Table 1. Figure 4 presents collected data using a bar chart and a pie chart.

TABLE 1 Posttest Questionnaire

Question Number	Statement
Q1 (Effectiveness)	The items recommended to me matched my interests.
Q2 (Learnability)	The interface provides an adequate way for me to express my preferences.
Q3 (Learnability)	The recommender’s interface provides sufficient information.
Q4 (Learnability)	I became familiar with the recommender system very quickly.
Q5 (Usefulness)	It is easy for me to inform the system if I dislike/like the recommended item.
Q6 (Usefulness)	The recommended items effectively helped me find the ideal product.
Q7 (Satisfaction)	The recommender made me more confident about my selection/decision.
Q8 (Satisfaction)	The interface let me know easily where the rest of my teammates are at all times.
Q9 (Satisfaction)	Overall, I am satisfied with the recommender.
Q10 (Intention to use in the future)	I will use this system for buying products in the future.

FIGURE 4 Usability analysis of gCOACH interface.

Figure 4a depicts the results obtained from the posttest questionnaire. Note that these results are related to the subjective perception of users but are quantitative data, which gives us valuable information about users’ perceptions of usefulness and usability of our gCOACH framework. Overall, the quantitative results obtained from the questionnaire are very satisfactory. It is worth noting that 88.4% of the responses were ranked with 3 or more points, 2.3% of responses correspond to a minimal score (1 value), 9.3% replied to a question with value 2.

Considering the learnability of the gCOACH (i.e., questions Q2 to Q4), 84.1% of participants’ responses show that the users found the system easy to learn and evaluated this aspect with 3 or more points. The majority of the participants (35 of 44) positively evaluated the Q2 question (i.e., 80.0% of participants ranked this question at over 3 points). Moreover, the obtained result for Q3 is very satisfying, too, as 39 participants ranked over 3 points (i.e., 88.6% of the participants) and none of participants replied to this question with a minimal score. In addition, the answers to question Q4 show that 84.1% of participants evaluate this aspect positively (i.e., 37 of 44 ranked this question with over 3 points).

With regard to the satisfaction of users, results of Q7 to Q9 show that 86.4% of the participants positively evaluated this aspect with 3 or more points. In this way, the result in Q7 shows that 85.0% of the participants perceive that the recommender gives them more confidence about their selection (i.e., 35 participants that evaluate this question with over 3 points). Additionally, the results in Q8 are satisfactory, too; 79.5% of the participants ranked with 3 or more points and the result in Q9 indicates that the majority of users were satisfied with the system (i.e., 95.5% ranked this question with 3 or more points).

In addition, users’ opinions reference whether gCOACH is useful to them (i.e., usefulness aspect); responses to Q5 and Q6 answers depict that 90.9% of the participants evaluated it with 3 or more points. Specifically, 84.1% of the participants ranked with 3 or more points the Q5 question (i.e., 37 of 44), and it’s worth noting that in the case of Q6 all of participants ranked this question with 3 or more points. Moreover, when the users answer about their intention to use this system in the future (Q10) only 1 participant ranked with a minimal score, which means that users have a good perception about the usefulness of the gCOACH system.

Finally, with regard to perceived effectiveness by users during the recommendation process, results of Q1 show that 97.7% of the participants evaluated positively this aspect (i.e., 43 of 44 participants ranked this question with 3 or more points).

We also asked users about the area on the interface where they paid more attention to the recommendations received: the stack area, the suggestion box area, the awareness area, or if there is not a preferred area. We report the results of the users’ perceptions of most useful area in Figure 4b. Forty-eight % of users prefer the suggestion box area as the main source of information about members’ activities and for choosing a product. Note that suggestions received in this area come from teammates or are proposed in a proactive way by the recommender algorithm. Next, the most preferred area with a 34% value is the awareness area. This means that group interaction is highly influenced by observation because users prefer to observe which teammates consult about the products and then select one of them. Nine 9% of users prefer the stack area as the main source for recommendations. Finally, 9% of users do not have a clear preference for an area because they have been looking equally at all of them.

CONCLUSIONS

In this article, we argued that there have been many recommendation scenarios that involve groups of interrelated users intending to participate in a group activity. But to date, most of the state-of-the-art approaches have focused on particular domains or on off-line environments. We have described in depth our proposal called gCOACH (COllaborative Advisory CHannel for group recommendation), which is an independent domain, web-based environment that supports online group recommendation scenarios. The conceptual architecture of the environment has been described. Moreover, we have explained the developed user-interaction modalities available for the users to enhance communication, coordination, and persuasion among the group participants. The usability of this novel interface has been evaluated by live users. The results show that 88.4% of participants responded positively to the various social interaction modalities. The results also depict that, mostly, users prefer to be aware about the products that are suggested by other group members and it is their main influence for choosing products. Considering all the interaction modalities, as future work, we plan to add the behavior implicitly detected in the gCOACH interface to define roles (i.e., leader, collaborator, or follower) among the teammates to influence the group recommendation algorithm.

FUNDING

The work of David Contreras is supported by a doctoral fellowship “Becas Chile.” This work has been supported by project “DIANA: DIscourse ANAlysis for knowledge understanding” (TIN2012-38603-C02-02) from the Spanish Ministry of Science and Innovation.

Additional information

Funding

The work of David Contreras is supported by a doctoral fellowship “Becas Chile.” This work has been supported by project “DIANA: DIscourse ANAlysis for knowledge understanding” (TIN2012-38603-C02-02) from the Spanish Ministry of Science and Innovation.

Notes

¹ With a critique, the user express a preference for a specific feature in line with their personal requirements (e.g., cheaper or higher star rating for hotel, etc.)