User roles and team structures in a crowdsourcing community for international development – a social network perspective

ABSTRACT

The principles of crowdsourcing are increasingly applied in social contexts like development projects. In this study we explore a crowdsourcing community, which aims to find innovation to enhance conditions for women and girls in developing countries. Overall, the observed community shows a high level of collaboration and reciprocal dialogue. We further explore differences between teams and individual community members. While on the individual level we located four different user roles distinct in their interaction and contribution behavior, on the team level we identified the importance of distinct user roles on team performance. We contribute to the theory of crowdsourcing by illustrating that context and purpose of crowdsourcing initiatives may influence the behavioral pattern of users. This study contributes to theory about virtual teams by providing a better understanding about team structures in the context of crowdsourcing. Further we add insights to the junctures between crowdsourcing and social innovation in the context of open development.

KEYWORDS:

• Crowdsourcing
• ICT4D
• open development
• social innovation
• user roles
• teams

1. Introduction

Social innovation will be essential to overcome today’s complex and urgent development challenges like overpopulation, poverty, resource scarcity, or natural catastrophes, to drive progress and impact the lives of the people. Overcoming such problems – indeterminate in scope and scale – tends to require bottom-up, decentralized processes, and the involvement of a wide range of people covering broad capabilities and possessing different skills and interests (Bisgaard, 2009; Levin, Cahore, Bernstein, & Auld, 2012).

Particularly promising new approaches derive from the emergence of digital technology such as Web 2.0 and the entry of information and communication technology in international development (ICT4D) (Walsham, 2017). ICT4D research today embodies more than just the sole delivery of information and communication technology (ICT) to the poor, but considers ICT as an enabler of bottom-up collaboration and co-creation that include the poor as active producers and source for innovation (Heeks, 2008; Thompson, 2008).

A potential way to actively include broad audiences to problem solving, establish communities and produce social innovation are crowdsourcing initiatives (Christensen, Baumann, Ruggles, & Sadtler, 2006). Through applied web 2.0 functions individuals can share their ideas, interact and collaboratively work on proposed problem descriptions (Füller, Jawecki, & Mühlbacher, 2007). The notion to exploit the collective intelligence through crowdsourcing (Raman, 2016) has lately gained importance in the field of international development (Hellström, 2016).

Literature suggests that in order to successfully manage crowdsourcing communities, an understanding of different user roles and behaviors (e.g. contributions, knowledge sharing and social interactions) within the community needs to be achieved (Pedersen et al., 2013; Welser et al., 2011). Hence, research to elaborate structures that enable ideation for social change (Cajaiba-Santana, 2014) and the dynamics of ICT-mediated development projects (Smith & Elder, 2010) is plausibly desirable, yet investigation into the application of crowdsourcing for international development is scarce (Hellström, 2016).

To gain an understanding about the underlying dynamics of online communities in general, various studies have investigated network structures and communities’ user types based on participation and contribution behavior (Murray, Caulier-Grice, & Mulgan, 2010). Such user behavior and user types might vary (Hautz, Hutter, Füller, Matzler, & Rieger, 2010; Hinds & Lee, 2008; Nolker & Zhou, 2005), depending on the context and purpose of a community. Hence, the first research question of this study explores the types of user behavior that can be found in a crowdsourcing initiative focusing on social innovations in the specific context of open development projects. In this sense, this research contributes to literature by elaborating differences of behavioral patterns of users in the social context compared to existing research on crowdsourcing initiatives.

Second, this research investigates the formation of teams and the performance of teams in crowdsourcing communities focusing on social innovation. Researchers agree that teams can outperform individuals and understanding what makes teams successful is of rising interest (Cooper & Kagel, 2005; Dissanayake, Zhang, & Gu, 2014). Literature about how the constellation of teams influences their performance is rare (Balkundi & Harrison, 2006; Benefield, Shen, & Leavitt, 2016; Oh, Chung, & Labianca, 2004). Therefore the second research question focuses on team performance compared to individual performance and how the configuration of teams influences team success. We contribute to literature by providing a better understanding about team structures and which user roles are most valuable in teams.

To process the research questions we investigate one case hosted on the crowdsourcing initiative openIDEO.org. The platform was specifically initiated to create an open global community and generate innovations to solve social problems (Boudreau & Lakhani, 2013). The investigated case deals with the highly relevant topic of gender and inequality of women (in low-income communities) (Walsham, 2017). More precisely, the case aims at improving the lives of the millions of women and girls living in areas of poverty in developing countries (openIDEO, 2017).

For our investigation, we apply social network analysis (SNA). The approach allows a precise visualization of the social network and the interaction within the community. Kane, Alavi, Labianca, and Borgatti (2012) claim that social media networks like the crowdsourcing platform at hand “require a new agenda” compared to the established field of SNA. Therefore a revisited application of SNA appears promising.

The remainder of this article is structured as follows: In Section 2 of this study, we introduce literature around the concepts of ICT4D and the shift to open models in international development. The principles of crowdsourcing are illustrated and the connection to international development is established, before the participation behavior of users and their roles within online communities are highlighted. Section 3 presents the investigated community and the applied methods. We present our findings in Section 4, and conclude with a discussion and implications.

2. Literature review

2.1. ICT4D and open development

In current times, the application of information and communication technology towards international development (ICT4D) possesses great significance for international development and the capability to improve the lives of less materially advantaged members of societies in developing countries (Walsham, 2017).

Alongside the evolution of ICT and Web 2.0, ICT4D shifted from a top-down concept, where the poor are considered as passive users and a focus on the delivery of ICT, to concepts of ICT as enabler of bottom-up collaboration and co-creation that include the poor as active producers and source for (social) innovation (Heeks, 2008; Thompson, 2008).

Research is not merely about how to implement ICT appropriately within a developmental context (Walsham & Sahay, 2006), but is interested in how ICT can act as an “engine” for self-determined development that unleashes individuals innovativeness and entrepreneurial spirit (Thompson, 2008). ICT is no longer just hardware, software and user behavior, but an “architecture of participation” (Thompson, 2008) that has the capability to generate new social innovation (Heeks, 2008) and empower and transform individuals and societies (Smith & Elder, 2010) and address the roots of a societal problem (Chalmers, 2012).

This shift allows open models that facilitate information-networked activities (Bentley & Chib, 2016; Smith, Elder, & Emdon, 2011) and enable the sharing of ideas, and the reuse and revision of content. Open models increase the transparency of processes (Smith, Reilly, & Benkler, 2014) and foster participation and collaborative behavior (Bentley & Chib, 2016).

These novel information networks impact the way how international development is done (Smith et al., 2011) and offer a new range of participatory methodologies (Chambers, 2010). Various authors such as Thompson (2008) and Heeks (2010) refer to this change as “Development 2.0”. Hellström (2016) further states that “ICT approaches and innovative, smart solutions are recognized as critical to solving development challenges, including using ICT tools to leverage the power of collective intelligence and knowledge” (Hellström, 2016, p. 639).

As an applicable ICT tool, crowdsourcing could be named. The decentralized collaboration and shared knowledge in a crowd-based initiative is assumed to provide benefits to more people in more effective ways, in comparison to sole traditional practices (Bentley & Chib, 2016).

2.2. Crowdsourcing platforms as a tool for international development

Since the first article about crowdsourcing by Howe (2006), it has become a well-established tool to foster innovation and collaboration in various fields, such as business, research, or government (Bott & Young, 2012).

The term is defined as “[…] the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call” (Howe, 2009, p. 99). The basic assumption behind crowdsourcing refers to the notion of “the wisdom of the crowd” (Howe, 2006), wherein large groups that are working jointly are considered to create more knowledge and information and therefore intelligence that yield to a higher value compared to the work of individual users (O’Reilly & Battelle, 2009).

The potential of crowdsourcing was demonstrated in various ways, starting with open source projects like Linux (Howe, 2006), where programmers voluntarily work jointly on software solutions, the collaborative content creation of Wikipedia, or innovation and idea contests like InnoCentive, where a large group tries to solve technical problems and find new innovations.

Applying crowdsourcing as a tool aiming to exploit the collective intelligence (Raman, 2016) is a rather new phenomenon that lately gains importance in the field of international development (Hellström, 2016), the humanitarian sector (Poblet, 2011) and social innovation (Füller, Hutter, & Friesüller, 2012). However it has been investigated by only a small number of researchers, especially in the context of international development (Hellström, 2016).

Crowdsourcing in its variations none the less has the ability to achieve socio-economic impact in various sectors (Hellström, 2016) and “change the reality of civic participation in many developing countries” (Bott & Young, 2012, p. 47). It facilitates the cooperation across multi-stakeholder environments (Lettice & Parekh, 2010), collective action and aligned interests (Neumeier, 2012) to contribute to the public good and endorse change in the social system (Cajaiba-Santana, 2014). Most social problems are of multifaceted nature and high difficulty (Jankel, 2011; Lettice & Parekh, 2010). To come up with solutions, it is important to gain deep insights into the cause of a societal problem (Lettice & Parekh, 2010), the environment and the affected people (Lettice & Parekh, 2010). Merging locals, sponsors, social entrepreneurs and other stakeholders and creating networks that lead to supportive communities is crucial (Hoang & Antoncic, 2003).

The most prominent example of crowdsourcing for international development is probably the open source platform Ushahidi (Hellström, 2016). The platform firstly was used 2007–2008 for monitoring the post-election violent acts in Kenya (Hellström, 2016; Meier, 2011). Notably, the code was spread to different countries (e.g. South Africa, Congo, Pakistan) and used besides violence detection to map natural disasters, drug shortages, and monitor elections (Hellström, 2016). A further example of crowdsourcing took place in Egypt: the open source project HarassMap was initiated to geo-locate sexual harassment (Young, 2014).

Besides coding, information sharing and mapping, crowdsourcing is increasingly used to create collective knowledge, form communities and foster collective creativity and innovation (Bott & Young, 2012). Collaborative communities include examples like Travel2Change that combines traveling with voluntary work (Füller et al., 2012) or the online volunteer program of the United Nations Volunteers, that claims to bring motivated people together and jointly help development organizations to address development challenges (Hellström, 2016).

The focus in this study is on a crowdsourcing initiative that aims to find and develop innovation to improve the conditions of women in a developing country. The crowdsourcing initiative is accessible for everybody, possesses – like most other initiatives – an online component and is designed as community rather than contest (Boudreau & Lakhani, 2013). While contests focus on the maximization of diverse contributions, communities anticipate a coherent and value-creating whole by aggregating various contributions (Boudreau & Lakhani, 2013). Thereby free information sharing and the possibility to collect and combine ideas facilitate success for such initiatives (Boudreau & Lakhani, 2013).

It is claimed that key to a successful crowdsourcing initiative are the individuals that are actively attending (Pedersen et al., 2013). Hence, inspiring, attracting and retaining individual participants through the right design of the initiative, including the right combination of social networking tools are of high importance (Bott & Young, 2012).

Pedersen et al. (2013) hint to the importance of understanding the participants’ behaviors, needs and motivations. It is argued that complex descriptions of social innovation processes and communities are needed in order to deliver new insights into a concept not yet explored in innovation literature (Cajaiba-Santana, 2014). We follow the research suggestion of Cajaiba-Santana (2014) regarding the search for structures to enable agents to engage in the development of ideas that promote social change. To achieve greater knowledge of participants’ behaviors and the dynamics within the crowdsourcing initiative we determine distinct user roles participants possess within the community.

2.3. User roles in crowdsourcing communities

Previous studies provide important insights into the identification and conceptualization of different user roles in various online communities and crowdsourcing communities. As indicators of distinct user roles the frequency of participation and the volume of contribution were often used (Hautz et al., 2010). For instance, Kozinets (1999) forms four user types (tourists, minglers, devotees, and insiders) in virtual communities of consumption, according to their relationships with and to the community. Since researchers have acknowledged the overlaps between networks and communities (Hautz et al., 2010), SNA and the theory of social capital is commonly applied for investigating user roles in online communities (Malinen, 2015). In this vein, Koch, Hutter, Decarli, Hilgers, and Füller (2013) revealed six user roles in an online community in a political context, namely motivators, attention attractors, idea generators, communicators, masters, and passive users. Also Füller, Hutter, Hautz, and Matzlerüller (2014) identified six different user types (masters, socializers, idea generators, efficient contributors, passive idea generators, and passive commentators) in innovation contest communities. Although the community includes a hybrid structure with cooperation and competition, the basic social structure of online communities is met (Füller et al., 2014). Toral, Martínez-Torres, and Barrero (2010) discovered the user type “brokers” in an open source project. Guo, Zheng, An, and Peng (2017) explored a collaborative innovation community for new product development. With the help of SNA and cluster analysis they revealed six user roles (project leader, active designer, generalist, communicator, passive designer, and observer). Overall, the aforementioned user types are essential for the information flow within the community as they act, as intermediaries between experts and peripheral users.

Research is also required on the specific dynamics of new forms of ICT-mediated sharing, cooperation, participation, and collaboration in the context of open development (Smith & Elder, 2010). It is frequently argued that the context and purpose of communities influences the type of users and their behaviors (Hautz et al., 2010; Hinds & Lee, 2008; Nolker & Zhou, 2005). Further, the importance of a clear understanding of the underlying community network structure and the user roles is highlighted to successfully manage such online communities (Hinds & Lee, 2008). It seems crucial to study users’ behavior in various settings and scenarios (Zhao & Zhu, 2014).

We are keen on exploring the heterogeneity of users and distinct user types of an initiative that uses the crowdsourcing approach to seek social innovation in international development.

Buskens (2014) states that people who contribute to open development are not driven by greed or competition. Instead they follow “intrinsic human needs to make useful and meaningful contributions, share openly and collaborate freely” (Buskens, 2014, p. 341). The acknowledgment of others’ human intentionality and a shared purpose might engender a meaningful dialogue and interaction among actors (Buskens, 2014).

In research around social innovation, contributors are often deemed as social entrepreneurs who are defined as agents of social innovation who follow social goals instead of sole personal interests (Lettice & Parekh, 2010). They are instrumental for the enacting of social innovation (Phills, Deiglmeier, & Miller, 2008), have tremendous intrinsic motivation to do social good and are driven by altruism (Martin, 2007).

In light of the literature around social innovation and international development, we expect high collaboration activities and close relations among participants, following the mutual goal of contributing and solving social challenges.

2.4. Team structure in crowdsourcing communities

Crowdsourcing initiatives increasingly incorporate web functions that allow the formation of teams within the crowdsourcing community (Dissanayake, Zhang, & Gu, 2015). Users often form groups to enhance their chances (Rokicki, Zerr, & Siersdorfer, 2015), reach a common goal (Dissanayake et al., 2015) and gain competitive advantage (Rokicki et al., 2015). Literature suggests that teams can achieve synergy effects and outperform individuals when tackling difficult tasks (Cooper & Kagel, 2005). Performing as a team can foster motivation and reduce the required time to completion due to a distribution of work. A diverse team in terms of expertise and skills is able to complete difficult tasks (Rokicki et al., 2015).

During a crowdsourcing contest of Netflix for example, participants were asked to find an effective algorithm for movie recommendations. Various participants combined their algorithms, formed groups and in this way increased their performance.

First researchers have started to investigate teams in online communities and applied SNA to understand the dynamics of teams and the performance of teams (e.g. Balkundi & Harrison, 2006; Benefield et al., 2016; Dissanayake et al., 2014). As suitable framework for their examination Benefield et al. (2016) used the theory of group social capital: “The configuration of a group’s member’s social relationships within the social structure of the groups itself […]” (Oh et al., 2004, p. 861).

In a meta-study Balkundi and Harrison (2006) examined the relation of network structure and team performance. Teams that are central within the community possess a high density and a central leader tends to perform better. In the field of massively multiplayer online game, Benefield et al. (2016) state that team member attributes and intragroup connections (density) are predictors for team effectiveness. However, they reveal when team’s social, task or exchange network ties are too sparse or too dense, the level of achievement decreases. Rokicki et al. (2015) studied various team competition strategies for crowdsourcing. They show substantial performance boosts for team-based scenarios. Dissanayake et al. (2014) point out that team performance in crowdsourcing contests is higher when members with high task-related skills are not centralized in the network. In a further study, they reveal that teams benefit, if their members have higher task-related skills or more connection ties with the other members. Team leaders and team experts are the extreme roles in the team with the greatest social capital and intellectual capital, respectively (Dissanayake et al., 2015).

Understanding how the configuration of teams and the social structure within teams influences the performance of teams is often discussed as needed future research in existing literature (Balkundi & Harrison, 2006; Benefield et al., 2016; Oh et al., 2004). Especially, in the field of crowdsourcing there is a lack of evidence in literature about the team performance compared to individual performance (Dissanayake et al., 2015) and understanding what makes teams successful is of rising interest (Dissanayake et al., 2014). As the investigated crowdsourcing initiative allows users to either submit and improve their ideas by themselves or form teams to collaboratively develop ideas, we are able to examine which individual user roles are most prominent and most valuable in teams.

3. Methodology

For the purpose of this exploratory research, a case study method is appropriate as there is a need to examine the proposed research interest with the help of a contemporary event (Benbasat, Goldstein, & Mead, 1987; Yin, 1994). Further, the phenomenon of interest can only be studied within its natural setting (Yin, 1994). This research analyzes a crowdsourcing initiative that focuses on fostering social innovation for a development project. The initiative was hosted by the platform openIDEO. openIDEO is a global community working together to design solutions for the world’s biggest challenges (https://openideo.com/). The case study setting is described in detail below.

3.1. Case study setting

The community at openIDEO consists of more than 17,000 users from over 170 countries. The platform has already conducted over 30 challenges to foster social innovations in different fields. The community of openIDEO can be considered as a collaborative community (Boudreau & Lakhani, 2013). OpenIDEO only provides the platform and its community and acts as a facilitator to various challenges. Given the similar design of all initiatives hosted on openIDEO and the focus on social innovation and development projects throughout all challenges this investigated initiative is representative for the whole openIDEO platform in terms of dynamics and user behavior.

In this study, we analyzed a challenge hosted by the Amplify program, which was initiated by the UK Department for International Development (DFID). The goal of this program is to end extreme poverty in developing countries with the help of social innovations. The program runs 10 challenges over 5 years on the platform openIDEO. The challenge addressed in this study is focusing on the central question: “How might we make low-income urban areas safer and more empowering for women and girls?” This topic of interest represents a fitting case to the area of ICT4D as gender issues can be seen as a central point of discussion in our understanding of ICTs in developing countries (Walsham, 2017). The challenge is divided into successive phases with clear assignments of tasks in each phase. First, there was the so-called research phase with the aim to motivate all participants to share inspirations, stories, tools and successful examples on the challenge topic. Based on these insights the “idea phase” followed and participants were asked to propose solutions to the given problem. Best ideas were then selected via an “applause phase” by the community and experts to advance to the “refinement phase” where the community collaboratively refined those ideas. An “evaluation phase” followed to select the final ideas with the view of having the most feasible ones funded.

The main part of the challenge was conducted within 22 weeks from February 2014 to July 2014. During this time period community members were able to comment and applaud research contributions and ideas. As soon as an idea was created the idea creator was able to invite other participants to his/her team to further work on the idea collaboratively. Furthermore they were able to update their ideas, and write stories about the impact of their ideas.

3.2. Data collection and analysis

The data about the investigated case was retrieved from the platforms server log files that record every activity taking place on the website. A digital file was generated that includes all data available on the crowdsourcing initiative. Users were able to comment on ideas in every phase of the challenge. Each comment was assigned to a specific user ID which enabled this study to figure out exactly who a commenter was and who the receiver of the comment was.

In total, 4057 users followed the investigated Amplify Challenge whereby 7646 comments were written. Within the idea phase, 450 individuals submitted a total of 575 ideas, from which 197 ideas contain team members to work on the idea; 52 ideas were selected for the refinement phase including 39 team ideas; 15 ideas were awarded as “final ideas” out of which 11 ideas were processed by teams; and 3 ideas received funding. All funded ideas were elaborated by teams.

We applied SNA and network measures (in-degree, out-degree, betweenness, and reciprocity) to visualize and interpret the network structure of the community. Based on individual user network measures and contribution quantity of users, we were able to conduct a cluster analysis and detect distinct user roles. In addition, we conducted an analysis of variance (ANOVA) to compare the quality of submitted ideas across the identified user roles and compare the structural position of these user roles. A Mann–Whitney U test was performed to investigate differences of team structures. We divided the teams in teams that reached the refinement phase (high-quality ideas) and teams that did not reach the refinement phase.

Social network analysis: With the help of an SNA, the interaction between participants can be grasped. The Amplify Challenge can be considered to be a social network as actor-to-actor relationships exist based on comments written on ideas. While most sociological methods exclude the individual from context and therefore constrain to single actors in a network, the SNA allows a researcher to analyze whole social systems. The big advantage here is that SNA is able to focus on traditional analysis at an individual basis, and at the same time considering information about the relationship across network members including the social context (Friemel, 2008). Applying SNA enables the identification of different user roles present in an online community, their distinct traits and influences on the whole community or other individuals (Gleave, Welser, Lento, & Smith, 2009) and their structural position in the community (Hutter, Hautz, Füller, Mueller, & Matzler, 2011). This type of analysis allowed us to calculate specific actor-based network measures that describe the interaction behavior of users in the network. The software UCINET 6.556 was used to calculate all measures relating to SNA, and to visualize the presented sociograms.

Cluster analysis: In a next step we applied a cluster analysis based on measures describing interaction behavior retrieved from SNA (in-degree and out-degree) and contribution behavior measured with the number of contributions of individual users. This inductive technique helps develop empirical groupings of persons, which can then serve as a basis for further analysis (Punj & Stewart, 1983). Key properties of clusters are external isolation and internal cohesion (Cormack, 1971). External isolation means that objects in one cluster have to be in proper distance with objects of another cluster. Internal cohesion refers to the need of similarity of objects within the same cluster. Further, cluster analysis minimizes the variation within and maximizes the variation between groups (Füller et al., 2014). Consequently the identified set of cluster solutions needs to be interpreted by a researcher (Aldenderfer & Blashfield, 1984).

In our case this method enabled the differentiation between different groups of actors which can be considered as grouping of user roles based on commenting and contribution behavior within the community.

Mann–Whitney U test: To gain insights on the impact and importance of different user roles in teams a Mann–Whitney U test was applied. This test allows to test differences between two groups on variables with no normal distribution (Weiner & Craighead, 2010). Accordingly the test is referred to as the nonparametric version of the parametric t-test. In this study the method tends to be more appropriate than a t-test as the data does not meet parametric assumptions. In our case the test allows for exploring the differences between teams in terms of user role densities.

3.3. Measures of individual interaction and contribution behavior

In this study we decided to use the measures in-degree, out-degree and contributions to conduct a cluster analysis. These measures were chosen to follow existing well-established research in online communities using the same measures to identify user roles (e.g. Füller et al., 2014; Hautz et al., 2010; Koch et al., 2013). In addition to existing research this study investigates the established clusters with a third measure derived from SNA described below.

SNA represents a valuable method to identify user roles as the derived social network methods can be used as a practical diagnostic and monitoring tool for community behavior (Hinds & Lee, 2008). To gain a complex understanding of how users behave the measures were divided into two separate types of behavior.

Interaction behavior: As users on the platform are able to either write comments or receive comments two different measures are used to capture commenting behavior, namely in-degree and out-degree. With the help of in-degree and out-degree centrality, popularity or activeness of a user can be determined (Kratzer & Lettl, 2008). In-degree is a measure that represents all ingoing relations of a user. Out-degree is a measure that indicates all outgoing relations of a user.

Contribution behavior: As an indicator for submission behavior we used the aggregated measure contribution, as it best describes the users’ direct contribution to the challenge. Contribution is consisting of the number of contributions submitted within the research phase, number of ideas submitted within the ideas phase and number of stories written within the impact phase.

After deriving at the final cluster solution further comparisons of clusters are conducted along the measure of betweenness and the quality of submitted ideas. Betweenness is a strong measure indicating the role of a user in the network as it describes the extent to which a particular individual lies between various other individuals in the network (Borgatti, Everett, & Johnson, 2013). We considered ideas voted into the refinement phase as high-quality ideas.

3.4. Measures of team structure

This research further elaborated measures to investigate on different team structures regarding user roles. As this study identifies user roles based on the cluster analysis, the structure of teams can be assessed indicating the user roles present in a team. To do so, Blau’s index of heterogeneity, considering different user roles, was calculated. Blau’s index can be used for categorical variables and is calculated by , where p defines the proportion of the group in the specific demographic category in each of the i categories (Blau, 1977). The Blau index can obtain any value between 0 and 1, 0 indicating a homogeneous group all of the same demographic category and 1 indicating a completely heterogeneous group with each actor being of a different demographic category. With the help of this index the heterogeneity of teams in respect to the user roles present can be determined.

To investigate the team structure in detail ratios were calculated indicating the share of each user role in a team. This measure is calculated by dividing the amount of users being of the same user role in a team by the total number of team members.

4. Results

4.1. Descriptive statistics and SNA

Figure 1 visualizes the sociogram of the social network based on the commenting behavior of users throughout the whole challenge.

Figure 1. Overall network of the amplify community.

The dyad reciprocity within the network is 44%. A dyad can be described as a present connection between two actors (Borgatti et al., 2013). The measure is computed by comparing the number of actual reciprocal dyads compared to the number of total dyads. A reciprocal relation is established when a relation between two actors is bilateral.

To identify different user roles measures of commenting behavior and submission behavior need to be considered. Table 1 shows descriptive statistics of the measures used as indicators for the participation behavior of users; 1027 users had an out-degree above 0. An out-degree above 0 indicates that those users commented at least once another user. The average out-degree of all users, indicating the average number of posts by a user is 1.67. The in-degree centrality reveals that 932 users received at least one comment by another user. As the value of written comments and received comments remains the same, on average each user received a total of 1.67 comments. On average each participant submitted 0.33 contributions. The median of zero indicates that a large proportion of users did neither write nor receive a comment throughout the challenge (2756). This large amount of passive users within a crowdsourcing initiative of the described size is a phenomenon known in crowdsourcing literature and in line with previous research (Füller et al., 2014; Koch et al., 2013; Kozinets, 1999).

Table 1. Descriptive statistics.

	n	Mean	Median	Std. Dev.	Min.	Max.
In-degree	4057	1.669	0.000	14.2800	0	737
Out-degree	4057	1.669	0.000	17.0491	0	862
Contribution	4057	0.33	0.000	1.100	0	23

Focusing on a team perspective the analyzed data reveals that 197 ideas were elaborated in teams with a minimum of 2 and a maximum of 32 team members. On average each team consists of 4.77 team members. Out of the 197 team ideas 167 ideas were updated at least once. The average number of updates is 8.62. In contrast, 378 ideas were processed by individuals from which 203 received updates. On average these ideas were updated 4.29 times.

4.2. Cluster analysis

Descriptive analysis of the measures presented in Table 1 reveals that the average user is not representative for the community of the investigated case, as seen in the standard deviation of each measure. This is in line with existing research that supports the need of identifying and assigning user roles to understand user behavior in online communities (Füller et al., 2014; Hutter et al., 2011; Koch et al., 2013; Panzarasa, Opsahl, & Carley, 2009).

Therefore we applied a cluster analysis based on the three measures identified representing commenting behavior (in-degree and out-degree) and submission behavior (contribution). Values have been standardized. In a first step we filtered community managers and a fake account created by openIDEO to preserve the content from deleted user profiles. In addition only users with either an out-degree above zero or contribution above zero were included in the cluster analysis as they are the users who actively participated. We identified two individual users who are superior in all three measures compared to the other participants. Those users were also removed from the dataset and analyzed separately in order to foster the stability of the cluster solution.

In our research, we combined a hierarchical clustering method with a non-hierarchical clustering technique (Punj & Stewart, 1983). As a starting point, we conducted a hierarchical clustering using the Ward minimum variance method based on squared Euclidian distances (Ward, 1963) to identify the number of clusters the dataset should be divided into. Interpreting the results with the help of the elbow method revealed that more than one large jump in the coefficient exist, which is evidence for more than one natural set of clusters (Ketchen & Shook, 1996). This finding serves as a starting point for the k-means non-hierarchical clustering method. For each possible case we conducted a k-means clustering analysis searching for the best cluster size, by considering the number of iterations needed to create each cluster solution (Lloyd’s algorithm) until convergence conditions are met (Kanungo et al., 2002). Out of six possible cluster solutions a four cluster solution required the lowest number of iterations. Based on the iterations needed for each cluster solution, the supporting literature and the usability for interpretation a four cluster solution was found to be most relevant, presented in Figure 2.

Figure 2. Four cluster solution.

Labeling the different clusters was done by interpreting each cluster in detail. Table 2 shows mean values of the three measures used for each cluster solution and an overview about the distribution of users across the four clusters.

Table 2. Statistical indicators of cluster solutions.

	Collaborator	Contributor	Allrounder	Passive user	Total
Variable	Mean	Mean	Mean	Mean
In-degree	42.462	36.167	10.343	2.157
Out-degree	44.038	34.500	8.990	1.985
Contributions	2.27	14.33	4.40	0.56
Team contributions	1.08	2.5	0.39	0.10
Frequency	26	6	105	1164	1301
Percentage	2.0	0.5	8.1	89.5	100.0

Note: Team contributions = contributions where teams were formed after submission.

To analyze differences in these four user roles and their interaction behavior in the following each cluster will be analyzed in detail with the help of network measures and the visualization of the egocentric networks of the user roles.

Collaborator: This type of user is characterized by a very high level of commenting behavior and a very low level of contribution behavior. On average the collaborator has an in-degree of 42.46 and an out-degree of 44.04 indicating that this user is very involved in commenting and in a dialogue with other users in general, as they receive a high amount of response. At the same time the collaborator only contributes at a very low level of own contributions (n = 2.27) instead he is focusing on the ideas of others. 1.08 contributions from collaborators formed teams which means that 47.6% of contributions submitted by a collaborator were refined in teams. Figure 3 shows the egocentric network of a typical collaborator.

Figure 3. Collaborator (ID 24711), 3 contributions, in-degree 55, out-degree 47.

Contributor: The contributor is characterized by a high level of commenting behavior and a high level of contribution behavior. In detail this type of users has on average an in-degree of 36.17 and an out-degree of 34.5. Again both commenting behavior measures are around the same value which indicates an equal distribution of ingoing and outgoing relations of a user. The egocentric network of a contributor is visualized in Figure 4. In contrast to the collaborator the contributor has a high level of contributions submitted. On average each contributor is responsible for 14.33 contributions out of which 2.5 contributions were improved in team work, representing 17.45%.

Figure 4. Contributor (ID 37383), 23 contributions, in-degree 43, out-degree 31.

Allrounder: This type of user is classified with a moderate level of commenting behavior. On average an allrounder has 10.34 ingoing relations and 8.99 outgoing relations. The allrounder contributes on average 4.4 times. Compared to the previously described user roles the allrounder is low in commenting behavior. Contribution behavior positions this user above the collaborator with twice as many contributions, but way below the contributor. Only 8.9% of contributions submitted by an allrounder included team members. The commenting behavior is presented in Figure 5.

Figure 5. Allrounder (ID 39299), 6 contributions, in-degree 10, out-degree 10.

Passive User: The least interactive user based on commenting behavior and submitted contributions is the passive user. On average this user type has an in-degree of 2.16 and an out-degree of 1.99. This very low commenting behavior is complemented by 0.56 contributions submitted from which 17.86% contributions were refined in teams.

In addition to these four cluster solutions, we identified two high performing individuals who are superior in commenting behavior and contribution behavior. These users outperform any of the clusters and therefore they have to be approached separately. In our research these users are referred to as Stars.

User 23241 has an out-degree of 391 and an in-degree of 220 and submitted 10 contributions. Considering his superior level of commenting behavior his importance to the interaction within the community is clearly given. His dense interaction behavior is shown in the egocentric network in Figure 6.

Figure 6. Star (ID 23241), 10 contributions, in-degree 220, out-degree 391.

User 36885 can also be perceived as Star, as he has an in-degree of 125, an out-degree of 144 and submitted 9 contributions. These values implicate that this user contributes to the community with both, his interaction behavior and contribution behavior with a strong focus on interaction (Figure 7).

Figure 7. Star (ID 36885), 9 contributions, in-degree 125, out-degree 144.

4.3. Comparing quality of contribution across user roles

In a next step we compared the quality of submitted contributions across the identified user roles. Out of 575 ideas, 52 were selected for the refinement phase. A dichotomous variable was calculated (“1” if a submitted idea reached the refinement phase;“0” if the submitted idea did not reach the refinement phase). An ANOVA revealed that the collaborators differ significantly in terms of quality of ideas submitted compared to the other three user roles. As Table 3 shows the collaborator submitted most ideas which managed to get to the refinement phase with a mean value of 0.46. This means the collaborator submits ideas with the highest potential to be of high quality. With a probability of 46% an idea submitted by a collaborator is elected to the refinement phase.

Table 3. Idea quality and betweenness among user roles.

	Collaborator		Contributor		Allrounder		Passive user
Variable	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Total	F-value
Idea quality: refinement phase	0.46	0.508	0.17	0.408	0.04	0.192	0.03	0.168		47.507***
Structural position: betweenness	18,265.95	17,708.01	7430.90	3832.72	1749.53	3293.32	333.47	1398.56		326.342***
n	26		105		105		1164		1301
Percentage	2.0		0.5		8.1		89.5		100

Notes: Refinement phase: 0 = lowest score; 1 = best score. *p < .05; **p < .01; ***p < .001.

4.4. Comparing structural position across user roles

As a last step we study how the identified user roles differ in terms of structural position. We therefore compare the network measure betweenness across the four user roles. Due to the fact that users with a high degree do not necessarily have to play an important intermediary role this measure is crucial to get a deep understanding of the network (Scott, 2012).

ANOVA revealed that the betweenness differs significantly across all four user roles. All means and standard deviations can be seen in Table 3. A high betweenness, as seen on the collaborator, indicates a strong dependency of others on the observed user who can be seen as gatekeeper (Scott, 2012).

4.5. Team structure

As users were able to form teams to work on ideas this paper also investigates on how specific team structures influence the quality of ideas. In a first step Blau’s index of heterogeneity was calculated based on the four cluster solution of user roles. In addition, ratios were computed describing the share of each user role in each team (R_Collaborator, R_Contributor, R_Allrounder, R_Passive User). For example, a team of four individuals out of which two are defined as collaborators receives a collaborator ratio (R_Collaborator) of 0.5. A Mann–Whitney U test was performed to test for differences between ideas of high-quality compared to low-quality ideas. Idea quality was again defined by reaching the refinement phase. Blau’s index of heterogeneity and the four team structure ratios were used as independent variables. The test indicated significant differences on one variable. The variable R_Collaborator is significantly higher (p = .006) on ideas that reached the refinement phase. On average a team accountable for a high-quality idea includes 13% collaborators (R_Collaborator = 0.13) compared to teams responsible for low-quality ideas including 7% collaborators (R_Collaborator = 0.07) (Table 4).

Table 4. Team structure.

	Not elected to refinement phase		Elected to refinement phase		p
Variable	Mean	Std. Dev.	Mean	Std. Dev.
Blau’s Index	0.43	0.20	0.42	0.24
R_Collaborator	0.07	0.14	0.13	0.17	**
R_Contributor	0.04	0.11	0.01	0.04
R_Allrounder	0.18	0.27	0.09	0.16
R_Passive User	0.45	0.28	0.50	0.32
n	158		39

Note: *p < .05; **p < .01.

5. Discussion

The case we investigated represents a best practice example of an ICT-enabled project for international development. Different to the often mentioned open source and tracking platforms like Ushahidi, openIDEO represents a concept that aims to generate social innovations for the developing countries. Not only the platform itself can be considered as innovative, but the content that is generated collaboratively by a large group of people from all over the world. Research in ICT4D lacks research about the application of crowdsourcing (Hellström, 2016). Researchers agree that the behavior in crowdsourcing initiatives is influenced by the context and purpose of the initiative (Guo et al., 2017; Hautz et al., 2010; Hinds & Lee, 2008; Nolker & Zhou, 2005; Zhao & Zhu, 2014). Current research on community structure and user roles is limited to specific initiatives in the fields of innovation contest communities for new product development (Füller et al., 2014, Guo et al., 2017), open source projects (Toral et al., 2010), open government (Koch et al., 2013), and virtual communities of consumption (Kozinets, 1999). By investigating the interaction behavior within an ICT-enabled project for international development we provide insights about the community structure and dynamics of such crowdsourcing communities. Further, research on team performance in crowdsourcing is scarce (Dissanayake et al., 2015) and future research on team structure and performance wanted (Balkundi & Harrison, 2006, Benefield et al., 2016, Oh et al., 2004). This study sheds light on team performance and how the constellation of teams influences team success.

On a community perspective, findings reveal that a large number of participants are passive users. This reflects findings of previous research on crowdsourcing communities (Füller et al., 2014; Koch et al., 2013; Kozinets, 1999). In addition this research discovered a high level of interaction between all active users. All users who actively engage in the community show a very intense interaction behavior, expressed by high levels of in-degree and out-degree. Saliently, the in-degree and out-degree are of similar level. This indicates a high collaboration between users and reciprocal conversations, as users not only write a high amount of comments, but in return also receive roughly the same amount of comments. The finding is supported by a high dyad reciprocity of the network of 44% compared to the dyad reciprocity in an innovation contest of around 10% (Kathan, Füller, & Hutter, 2013). Hence it can be assumed that the social purpose leads to a high collaboration in the community, which is in line with literature stating that doing something good is supposed to minimize competition (Buskens, 2014), but should lead to open collaboration and meaningful dialogue (Buskens, 2014). As social entrepreneurs are supposed to be driven by altruism and the urge to achieve social goals rather than focusing on personal gain (Lettice & Parekh, 2010), it can be argued that the overall communication pattern in the investigated community is an indicator for the behavioral pattern of social entrepreneurs.

Several studies have analyzed the interaction behavior of individuals and user roles in online communities (Arazy & Nov, 2010; Cross, Parker, & Borgatti, 2002; Füller et al., 2014; Hutter et al., 2011; Koch et al., 2013; Toral et al., 2010) and similarities between different community settings were found (Füller et al., 2014). The question arises whether an online crowdsourcing community focusing on social innovation for development projects appears to have different interaction behavior and user roles present. A key contribution of this study is the identification of four distinct user roles within the investigated community, namely collaborators, contributors, allrounders, passive users. Those user roles display significant difference in terms of interaction behavior and contribution behavior. Findings show that a large number of participants are passive users. This reflects findings of previous research on crowdsourcing communities (Füller et al., 2014; Guo et al., 2017; Koch et al., 2013; Kozinets, 1999). It can be assumed that this type of user role is characteristic for online communities independent from context. Some identified user roles in this study show similarities with previously identified user roles in literature (Füller et al., 2014; Koch et al., 2013; Kozinets, 1999) whereas others can only be found in the given context of open development projects.

The collaborators have been identified as the most interactive user role. This type of user seems to be able to integrate his/her collected knowledge into his/her ideas, as he submits ideas with the highest potential to be of high quality. In addition, this type of user is able to transfer knowledge between lots of other users giving this user a gatekeeper position in the network. Without such gatekeepers the community loses a lot of knowledge and may have a deficit in collective intelligence (Welser et al., 2011). A similar user role could not be found in existing literature. We assume the collaborators unique behavioral patterns strongly relate to the social context of the community. Based on the definition of social entrepreneurs, we presume a high number of social entrepreneurs in the cluster of the collaborator. The contributor attracts attention in being good in all disciplines. They are of special value for the community as a high amount of submitted ideas increases the probability of finding an appropriate solution (Osborn, 1953). The contributors can be compared to the user role insiders, identified by Kozinets (1999), and masters, as described by Füller et al. (2014). The relatively large group of allrounders interact and submit ideas at a moderate level with a balanced ingoing and outgoing commenting behavior. However, the majority of users are represented by passive users. Furthermore we discovered star users within the community, who are fairly superior in all disciplines. But also here the pattern of both sided interaction can be seen. We could not find a user type focused only on contributions as mentioned in Füller et al. (2014). Most active users participate in a dialogue within the community of our study.

This study further examines the configuration of teams, the teams’ performance and compares teams to the performance of submissions of individuals. Although only 34% of submissions formed teams, 75% of the ideas that reached the refinement phase are team submissions. These results are coherent with previous studies (e.g. Rokicki et al., 2015) and indicate that teams have the potential to outperform individuals. Social problems are considered to be of multifaceted nature and high difficulty. Hence, we refer this finding to the statement of Mohammed and Ringseis (2001) who claim that teams are particularly formed to solve issues and tasks that are too complex to solve individually. These teams often exploit different types of expertise, experiences and perspectives to a common task (Mohammed & Ringseis, 2001). We ascribe the teams the willingness to improve the ideas as a team, as team ideas were edited on average 8.62 times, compared to 4.29 times for individuals.

When calculating the Blau index of heterogeneity to show the team diversity in terms of user roles, we could not find evidence that higher diversity in terms of the number of different user roles within a team leads to a high team performance.

To detect which user roles seem necessary to form successful teams we formed the ratio number of a certain user role divided by the total number of team members. We compared teams with high-quality ideas with teams that could not reach the refinement phase. Our findings reveal a significant difference of the relative number of collaborators. Put differently, teams that reached the refinement phase possess a higher proportion of collaborators. Identifying collaborators as valuable team members leads to several assumptions: Collaborators tend to work in teams. Around half of the ideas collaborators contributed, teams were formed. Providing feedback, cooperation and communication are main teamwork skill dimensions (Baker & Salas, 1992). The high values of their communication measures (in-degree, out-degree) suggest collaborators are team players. Considering teamwork a prerequisite for team success the status of collaborators becomes comprehensible. Due to the findings that collaborators are high in betweenness, we assume they gain access to diverse information and resources that lead to a performance advantage (Benefield et al., 2016). Perspectives and knowledge of external members may positively influence the whole team.

6. Theoretical implications

Our research contributes to literature about how ICT can enable individuals’ innovativeness (Thompson, 2008). We have shown that a crowdsourcing platform can act as a tool to encourage participation and share inspiration to generate social innovation for developing countries (Heeks, 2008). Crowdsourcing as tool has the capability to gather a crowd to find new solutions and constitutes an appropriate tool to facilitate a bottom-up approach. Further we add insights to the linkage between crowdsourcing and social innovation (Hoang & Antoncic, 2003) in the context of open development (Bentley & Chib, 2016).

While Zhao and Zhu (2014) highlight the importance to study participants’ behavior in certain contexts and scenarios, this research contributes to existing literature by extending the current research on community structures in crowdsourcing initiatives to the field of open development and social innovation. We complement research on crowdsourcing by introducing crowdsourcing as a tool for ICT4D and analyzing the community structure of a best practice crowdsourcing initiative for seeking social innovation for development.

Our results have important theoretical implications as they indeed show that the behavior of users in the investigated context differs significantly in terms of interaction behavior compared to other crowdsourcing initiatives. The presented findings contribute to a better understanding of the dynamics and structures of crowdsourcing communities that search for social innovation. We contribute to the theory of crowdsourcing by illustrating that context and purpose of crowdsourcing initiatives may impact the behavior (Nolker & Zhou, 2005) and type of users (Hautz et al., 2010).

Our findings contribute to literature of teams in crowdsourcing and the impact of team social structure on team performance. We provide insights to team management in crowdsourcing and support the theory of segmenting team member roles by their skills (Davenport, Thomas, & Cantrell, 2002). Further, we link team performance to the importance of highly interactive participants present within teams. Understanding how different types of participants influence teams and team performance contributes to existing literature on team performance (Balkundi & Harrison, 2006, Benefield et al., 2016, Rokicki et al., 2015) and shows the road for further studies of this topic.

7. Managerial implications

Managerial implications of this study are manifold. The results can aid designers and organizers to appropriately structure crowdsourcing initiatives that permit and intensify dialogue. As Kane et al. (2012) state, platform designers have substantial control over different mechanisms influencing the way users are able to communicate and interact (Kane et al., 2012). Such design decisions are increasingly important for organizations to employ different platform designs for different purposes since varying technical features may be more beneficial to cultivating networks with different objectives (Kane et al., 2012). Community managers can utilize the information to adjust their moderation strategies and strengthen bonds between users and encourage participation. For example, collaborators may be more interested in the interaction possibilities of the platform to engage in the community itself whereas contributors focus more on the idea generation process and the submission of ideas. These different behavioral patterns could be addressed by community managers with targeted incentive structures. Such incentives could include stimuli to reward not only best ideas but also most supportive behavior to attract collaborators and foster such behavior. In contrast, the behavior of contributors may still be fostered by rewarding the best ideas but also by rewarding users with the highest number of quality submissions. Managers could use targeted stimuli as described to foster the behavior of collaborators and contributors and may even encourage allrounders or passive users to denser interaction and submission behavior. Passive users are of high importance for crowdsourcing initiatives as they are most relevant to reach a critical mass and promote awareness about an initiative (Füller et al., 2014). However, a large number of passive users may represent a problem as this group of users provides little content (Preece, Nonnecke, & Andrews, 2004). Managers may address this problem by providing structures that make it interest to contribute for participants while at the same time be aware of the importance of many participants in a crowdsourcing initiative, especially in the case of open development projects. Based on the finding that groups are able to outperform individuals managers could adapt a collaborative platform design and provide incentives to encourage team formation and team work.

8. Limitations and future research

We note several limitations in this study that point to potential new directions of future research. With over 4000 observed participants this study can be considered as the current best practice example. However, the generalizability of user roles and their contextual behavior still needs to be tested by analyzing other crowdsourcing initiatives in a similar setting. The research focused on an exploratory design with emphasis on interaction and contribution behavior which may be complemented by additional measures indicating the growth of user roles throughout the crowdsourcing initiative in future research. Also, this paper used a quantitative research approach accomplished with an SNA. Future studies need to include qualitative content analysis regarding comments and submitted ideas to strengthen and further develop the presented user roles.

On a team perspective future research should focus on the investigation of teams as units and calculate centrality measures not only for individuals but also for teams as unit and the communication between teams could give hints about efficient teams in crowdsourcing. Also the dialogue and content of the dialogue within teams could reveal interesting patterns about the differences between teams. As shown in this study, research on the influence of different types of team members on team performance is of rising interest. Future research could take a closer look on team constellations and influencing factors of team communication on team success. Also a specific focus on team versus individual performance would be interesting for the research community in this specific field.

Overall, an interesting field of future research could be the investigation on the actual motives and personalities that are underlying different user types and how such motives might influence the interaction and contribution behavior of individuals in different settings of crowdsourcing initiatives.

9. Conclusion

In this exploratory study, we examined the community structures of an open development model in form of a crowdsourcing initiative which aims to find innovation to enhance conditions for women and girls in developing countries. The investigated community at openIDEO represents a social network defined by actor-to-actor relationships. By using network measures (in-degree, out-degree, betweenness, and reciprocity) and users’ contribution quantity and quality this study elaborated insights in the network structure and users’ roles and behaviors on the platform.

We consider crowdsourcing as a promising open model to integrate large networks into the innovation process. We have shown that different user roles with different interaction and contribution behavior exist within crowdsourcing communities in the specific context of international development.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Simon Fuger is a PhD candidate at the Innsbruck University School of Management. He holds a master's degree in Strategic Management. His research interest is in the field of open innovation, social innovation and crowdsourcing.

Robert Schimpf is a PhD candidate at the Innsbruck University School of Management and manager of the University Incubator "InnCubator". He holds a master s degree in Strategic Management. His research interest is in the field of open innovation, crowdsourcing and online innovation communities.

Johann Füller is Professor for Innovation and Entrepreneurship at the Innsbruck University School of Management. He is Fellow at the at the Crowd Innovation Lab/NASA Tournament Lab at Harvard University and CEO of Hyve AG, an innovation and community company. In line with his research focus, he regularly gives guest lectures about co-creation, online branding, creative consumer behavior, online marketing, open innovation, and the utilization of online communities.

Katja Hutter is Professor for Marketing and Innovation at the University of Salzburg and an associate of the Crowd Innovation Lab/NASA Tournament Lab at the Harvard University. Her research topics are anchored in the fields of marketing and innovation. She is interested in incentive schemes and consumer interaction behavior in online communities to generate insights for innovation activities that resolves around customers and their needs.