Social mechanisms in crowdsourcing contests: a literature review

ABSTRACT

Crowdsourcing contests allow organisations to engage with an external workforce. Over the years, the phenomenon has attracted considerable research interest. In the present review, we synthesise the crowdsourcing contest literature by adopting the social mechanism lens. We begin by observing that stakeholders in crowdsourcing contests range from individuals (solvers) to large-scale organisations (seekers). Given that such vastly different entities interact during a crowdsourcing contest, it is expected that their behaviour, too, can have a varying range of predictors, such as individual and organisational factors. However, prior reviews on Crowdsourcing contests and crowdsourcing, in general, haven't explored the phenomenon's multi-layered nature. In addressing this gap, we synthesise 127 scholarly articles and identify underlying social mechanisms that explain key behavioural outcomes of seekers and solvers. Our review makes two specific contributions. First, we determine three distinct tensions that emerge from the key design decisions that might be at odds with the central principle of crowdsourcing contests: broadcast search for solutions from a long-tail of solvers. Second, we provide three recommendations for future research that, we believe, could provide a richer understanding of the seeker and solver behaviour.

KEYWORDS:

• Behaviour
• crowdsourcing contests
• social mechanisms
• literature review
• seeker behaviour
• solver behaviour

Acknowledgement

The authors acknowledge the infrastructure support provided by their institutes FORE School of Management, New Delhi, India and Indian Institute of Management, Ahmedabad, India.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Notes

1 The searches were executed in the following manner: for each keyword pair (say, x AND y), we looked at articles that included x as well as y in the abstract. We did not combine the two keywords as a search query, allowing us to locate articles in which keywords in a given pair do not appear consecutively.

2 There are a few second-order behavioural outcomes as well. For instance, Gatzweiler, Blazevic, and Piller (2017) examined the precise constributions in idea contests to show that solvers can post ‘deviant content’, which can either have beneficial outcomes such as more discussion and idea refinement (positive deviant content) or lead to malacious protest and ridicule (negative deviant content). Faullant and Dolfus (2017) revealed the presence of negative engagement between solvers (e.g. bullying and reciprocal voting). Lampel and Jha (2017) examined ‘performance delta’ (change in solver's two subsequent design) in robotics competiton. Lastly, Koh (2019) studied the extent to which solvers rely on seeker-provided information. However, in this study, we emphasise on the first-order behavioural outcomes as these outcomes cover the majority of the prior work on CCs. That being said, we do believe such second-order behavioural outcomes are certainly pertinent and can be covered as part of a future review of CC literature.

3 The notion of cultural distance is separate from the cultural mechanisms discussed in 5.1.1. While the cultural tightness, uncertainty avoidance, and performance orientation aspects of solver's culture are not measured relative to that of the seeker's national culture, cultural distance is, by definition, a dyadic measure.

4 This explanation mirrors that provided by the competition-related mechanism (Section 5.1.3). If a high reputation solver enters early in a CC, they expect their presence to lower the winning probabilities of potentially competing solvers. As a result, such competitors may choose to not join the given CC.

5 We do not claim that solvers are exclusively driven by intrinsic motivations. There is large body of work showing that solvers are driven by extrinsic as well as internalised motivations (Alencar and Gama 2018). These motivations, however, can be addressed through a seeker intervention (e.g. offering monetary returns or reputation gains). In contrast, intrinsic motivations are largely psychological and not reliant on external interventions (Acar 2019). Given this distinction, we consider extrinsic and internalised motivations separately as part of our discussion on the effects of incentive structure on solver behaviour (Section 5.4).

6 That being said, there is some evidence to suggest that seeker-driven interventions can influence intrinsic motivations. For instance, affording greater autonomy and offering tasks that require the solver to undertake avariety of activities positively affects solver's intrinsic motivation (Zheng, Li, and Hou 2011; Lee et al. 2015; Garcia Martinez 2017; Liu, Liu, and Xiao 2020). However, by and large, intrinsic motivations are considered as independent of external stimuli.

7 At the same time, however, prior performance may also have a hindering effect on subsequent performance, especially if the task at hand is unstructured (e.g. idea contest). For instance, Bayus (2013) showed that past success might create a mental fixation with a particular approach or a solution, which may become a mental anchor for the focal solver (Bayus 2013, 230). As a result, past success may result in the solver providing related, and hence, less diverse solutions.

8 There is some contrasting evidence as well. For instance, Wang, Khasraghi, and Schneider (2019) showed that community activity (number of votes received for knowledge sharing in CC platform) does not related to continued participation (Wang, Khasraghi, and Schneider 2019, 141), although authors do hypothesize that receiving votes from other solvers could leads to greater sense of community (Wang, Khasraghi, and Schneider 2019, 139).

9 See Stol and Fitzgerald (2014) for a detailed narrative of the process, which a seeker can use to organize a CC.

10 See Fayard, Gkeredakis, and Levina (2016) for a cultural argument explaining the seekers’ adoption of crowdsourcing.

11 Also characterised as local versus distant knowledge environments (Jespersen 2018).

12 Also, see Afuah and Tucci (2012) for a conceptual argument backing this claim.

13 This was a real-life challenge broadcast on a CC platform InnoCentive. See https://bit.ly/3bZUO9T

14 Similar argument was presented by Bayus (2013) who showed that past success narrows the solver's idea horizon, and leads to more repetitive and incremental idea generation (see Section 5.1.6). However, Gillier et al. (2018) showed that the mental fixation mechanism is not just limited to solver's past success. It can also stem from suggestive formulation of CC task.

15 No provision of in-content feedback makes sense when a solver can only enter a single solution in a given CC. In such settings, a solver cannot use feedback for improving their output. For instance, TopCoder, a leading CC platform hosts programming contests in which, generally, a solver can only enter one solution (Boudreau, Lacetera, and Lakhani 2011). As a result, the platform does not provide any in-contest feedback for programming CCs.

16 In a related argument, La Toza et al. (2015) showed that providing public feedback also affects solver behaviour such that solvers respond not only to the feedback that their solutions have received but also the feedback that solutions of competing solvers have received. Such information spillover from the feedback signal is one of the underlying logics for the benefits of early entry (see Section 5.1.4).

17 This is a rather unique work given it points out a growing trend of mobilizing crowd on an industrial scale with a physical workplace.

18 While the focus of the discussion in this section is on how reward can shift/interact with other motivations, prior work also shows that reward can have an ever more far-reaching implications. For example, Koh (2019) showed that solver's use of seeker-provided information could be predicated on the reward amount.

19 Acar (2018) shows that in creative contests, either having a high monetary reward or having no monetary reward outperforms having a low monetary reward. The reason that high rewards to better than no reward, which, in turn, does better than the low reward is likely twofold. First, with the availability of reward, extrinsic motivations may crowd-out intrinsic motivations. Second, the presence of a reward may, by design, attract solvers who are extrinsically motivated. On both these fronts, the low reward may have limitations as they may not sufficient for ‘extrinsic to take over intrinsic’ (Acar 2018, 185). Therefore, the monetary reward may have a non-linear influence on solver behaviour (Acar 2018, 186).

20 In a subsequent study, Mack and Landau (2020) provide a more radical finding wherein they report extrinsic motivations to be the sole class of motivations driving the quality of output in idea generation CCs.

21 There is limited insight on predictors of IP selection in CCs. In a recent study, Piazza et al. (2019) showed that the IP regime selection is contingent on the extent to which the seeker may need to depend on the solver. For instance, when the problem is well-delineated (when the evaluation parameter are well-articulated and pre-specified in the RFP), seeker may have little incentive to continue in a joint activity with the solver, and hence, prefer a unilateral IP arrangement (e.g. patent acquisition).

22 This arrangement is not the same as ‘all-pay auctions’ characterisation of CCs (Liu et al. 2014). Although such actions require all the participating bidders (solvers) pay the cost of participation (efforts required to develop a solution), losing bidders may not lose the rights over their bids (solutions)

23 Although this study is not conducted in CC settings, we do consider its implications to be of relevance given that solvers in CCs are known to behave strategically to increase their chances of winning (see Yang, Adamic, and Ackerman 2008).

24 Kaggle, for instance, announces key changes made to the platform. The log of such changes along with the related discussion can be found here: https://www.kaggle.com/product-feedback