Fostering Creative Performance of Platform Crowdworkers: The Digital Feedback Dilemma

ABSTRACT

With crowdsourcing increasingly contributing to organizations’ innovative performance, it becomes more and more important for them to cultivate the creativity of their crowdsourcing communities. While digital feedback is the main, if not the only, two-way channel of communication between the platforms and the crowdworkers, little is yet known about how to use digital feedback to manage and foster the creative performance of crowdworkers. This study examines how the provision and nature of feedback, provided virtually through online interfaces, influence creative performance. We argue that the alleged positive relationship between the creative self-efficacy of crowdworkers and creative performance is conditional upon the joint effect of digital feedback valence and the degree to which crowdworkers focus on learning as achievement outcomes. We conducted a two-stage experimental study with 298 participants in a crowdsourcing setting. The results show that feedback provided in virtual settings, irrespective of whether the feedback is positive or negative, can be perceived as surveillance and thus hurt the creative performance of crowdworkers with high creative self-efficacy but low mastery goal orientation. However, the results also show that when receiving negative feedback, community members who have high creative self-efficacy and mastery goal orientation try harder in subsequent creative tasks. Accordingly, we advocate for nurturing platform cultures that emphasize both confidence in the contributor’s own competence and the abilities to learn and develop.

KEY WORDSAND PHRASES:

• Creative performance
• creative self-efficacy
• crowdsourcing
• digital culture
• digital feedback
• mastery goal orientation
• virtual settings

Introduction

The large-scale interconnectivity afforded by internet-based technologies has transformed the ways in which organizational tasks are performed [22]. One of the most significant outcomes of digitalization is the emergence of a global workforce organized and connected via digital labor platforms through which outsourcers obtain ideas, services, and content from the online community [13]. This type of work is termed crowdsourcing and crowdwork in the literature [13]. Crowdsourcing includes a variety of types of practice, including contract-based micro-work and competition-type idea contests that primarily reward contest winners [35]. In our study, we focus on the former, as this type of crowdwork represents the majority of crowdsourcing market. This type of crowdsourcing opens new opportunities for organizations to outsource specific tasks to the public instead of a designated agent. In doing so, organizations are able to draw on knowledge from diverse external sources to solve internal problems that benefit from diverse idea-generation and problem-solving [57]. More and more people are engaging in such types of online participation in organizational tasks or crowdwork, either to make a living or to earn additional income through digitally mediated labor platforms [13].

From an outsourcer’s perspective, this source of open innovation presents an opportunity to engage with a wider pool of creative labor to solve creative-oriented activities. The efforts of organizations to tap into the potential of open innovation entail a number of key challenges—for example, the challenge of sustaining quality participation in crowdsourcing (cf. [15]) through appropriate incentives [48]. Crowdsourcing platforms often attempt to incentivize crowdworkers with particular structural regimes, which, if they prove effective, result in positive reviews for community members [58]. However, satisfying crowdsourcers can be challenging for crowdworkers, since crowdworkers are often not given proper feedback on their work compared with those working in more traditional work settings [5]. While the provision of appropriate feedback may help facilitate continuous improvement, poor quality feedback or lack of feedback likely perpetuates unsatisfactory work, causing perceived unfairness among workers and even ethical concerns [13].

Understanding how to improve the creativity of community members and how to produce innovative products has become imperative for organizations to stay competitive in the market [5]. An important pathway to increase the creativity of online communities may lie in tailoring feedback according to crowdworkers’ creative self-efficacy, that is, their belief in their ability to produce creative outcomes [63]. Feedback is important for community members because they usually perform their work online often having limited interaction with their outsourcers. Thus, even the limited digital performance feedback they receive on their tasks could be a major social stimulus for their creative performance [64]. In particular, the literature suggests that creative self-efficacy may help creative performance depending on an individual’s social environment. However, individuals with high creative efficacy may respond differently to feedback given on their creative tasks [55]. These individuals may, for example, view such feedback as a learning opportunity or as a mean of control.

Schörpf et al. [58] identified various elements employed on crowdsourcing platforms that encourage both creative and feedback-related behaviors, typically including upgrades, rewards for completing a certain amount of projects, or online recognition for receiving reviews. These feedback mechanisms are thought to foster creative growth and additional participation, but they can also potentially hinder these outcomes (e.g., with negative reviews). In other words, while feedback is typically intended to foster improvement in performance, it may have unintended negative emotional and performance-related outcomes. The outcomes of feedback may not necessarily be caused by the feedback valence (i.e., positive vs. negative). Indeed, research indicates that negative feedback on a person’s creativity-standard gap may help them identify problems with their creative performance and stimulate them to reassess and improve their creative performance [38]. The different ways in which individuals respond to positive and negative feedback thus merit further examination. In this study, we first address the role of crowdworkers’ creative self-efficacy as it pertains to their creative performance in response to digital feedback. We argue that crowdworkers with lower creative self-efficacy may respond more strongly to digital feedback because their confidence in judging their work is lower [55]. In addition, the valence of the feedback (i.e., negative or positive) also plays an important role, yet conditional. That is, it is not necessary that negative feedback may drive negative response. In some conditions, negative feedback helps individuals to improve their work. In this paper, we argue that the response of crowdworkers to the valence of digital feedback (i.e., negative or positive) may depend to a great extent on their creative self-efficacy to carry out tasks successfully in their particular circumstances, which is in line with feedback intervention theory [41]. Feedback intervention theory maintains that positive feedback may increase task motivation, whereas negative feedback draws attention to potential creativity-standard gaps. We argue that this is especially the case for crowdworkers with low creative self-efficacy. More specifically, we propose that the supposed positive relationship between creative self-efficacy and creative performance may be negative depending on the performance feedback valence a crowdworker receives.

We further propose that this relationship must be understood in conjunction with a crowdworker’s mastery goal orientation, that is, their focus on learning as an achievement outcome [29]. We argue that crowdworkers with low creative self-efficacy will respond more positively to negative digital performance feedback on their subsequent creative tasks if they score high on mastery goal-orientation compared with crowdworkers with high creative self-efficacy who score low on mastery goal-orientation. In other words, the focus of crowdworkers on learning can inform the feedback intervention mechanism for those who have less confidence in judging their creative work but are eager to learn for future improvement. Digital feedback can decrease uncertainty and improve credibility among a platform’s constituencies [31]. However, crowdworkers may fear potential negative consequences from such feedback in terms of their subsequent payment and performance rating on the platform [49]. Accordingly, we look at the role and impact of feedback within the crowdsourcing process, that is, situations in which task requesters specifically intervene in the process by providing digital feedback of various kinds and examining how the provision and nature of feedback influences creative performance.

The intended contributions of the present study are threefold. First, in light of the rapid growth of crowdsourcing arrangements as a way for organizations to augment their businesses by accessing talent to fill critical skill gaps [41], our study extends the current understanding of how to foster the creative performance of online communities through predictive interaction between personal and contextual factors—namely, creative self-efficacy, digital performance feedback valence, and mastery goal orientation. Second, while some studies suggest that computer-mediated feedback compared with human-mediated feedback can provide more positive and consistent outcomes (e.g., [40]), the current study adds to the body of literature on feedback provided through digital channels suggesting that the feedback dilemma (i.e., instead of improving performance feedback can hinder performance) is also present in the digital environment. Third, monitoring and improving worker performance is a key role performed by human resource management. Among other job design elements, feedback is one of the important aspects that is designed and managed by human resource function. Against this background, some attention has been devoted to crowdworkers and contingent knowledge workers [8, 51]; the role of human resources practices, such as feedback design, is still understudied in the gig economy compared with more traditional corporate settings [21, 41]. Combined with research on machine-mediated feedback, our research provides both theoretical and practical insights into how to manage a platform-mediated community that, to some extent, parallels certain human resource management tasks associated with a more conventional organizational workforce.

Theory and Hypotheses

Creative Crowd Labor as an Important Source for Innovation and the Role of Digital Feedback

Much attention has been paid over the past decade to the ongoing fourth industrial revolution that is set to change work, work practices, and workplaces [21]. This transformation of work allows large-scale projects to be broken down into small work packages that can be distributed among a remote workforce [41]. These work packages distributed through various digital labor platforms involve different types of work, ranging from small micro-tasks, such as sorting images, to complex professional services, such as improving technology and organizational design. These tasks are also increasingly creative oriented and innovative focused [2].

The growth of crowdsourcing can be attributed to clients seeking to obtain three key benefits: lower costs (e.g., financial and time costs), greater flexibility (e.g., finding talents to fill seasonal work), and access to a wider pool of skills (i.e., high quality workers [13]). As an important subset of crowdworking, moreover, crowdsourcing opens new opportunities for organizations to outsource specific tasks to the public instead of a designated agent. In doing so, organizations are able to draw on knowledge from diverse external sources to solve internal problems that benefit from diverse idea-generation and problem-solving [57]. However, these positive aspects of crowdsourcing also come with challenges. For example, while technology—or, in this case, a digital labor platform—offers crowdworkers resources for finding and accomplishing tasks, technology can also make it difficult for crowdworkers to be fairly assessed for their work [1]. Unlike workers in more traditional work settings, crowdworkers interact with “software” rather than humans, often excluding opportunities for discussion, negotiation, and feedback on completed work [37]. Evaluation and eventual payment for delivered task work depend on the subjective assessment of the requestor. This is particularly true for creative performance, where an individual’s creative work (e.g., logo designing) is assessed and compensated on the basis of the crowdsourcer’s opinion [13]. This approach appears to constitute unfair treatment of crowdworkers, as communications between crowdsourcers and crowdworkers are not interactive but rather one-sided, with crowdsourcers setting expectations for the end product or end service. This leads to crowdworkers missing opportunities for feedback and for learning from their “mistakes” and improving future performance. One example of the negative impact of the absence of such a feedback process is denial of payment on the part of the crowdsourcer. Once payment is denied, the crowdworker’s ratings may fall. Similarly, a requester’s reputation may be seriously threatened on community-organized online forums [13]. Crowdworkers and crowdsourcers are thus both vulnerable to the lack of digital feedback systems in place. Such nonresponsiveness and/or false responsiveness opens an important avenue for research on how to unpack the “black box” [37] around fair feedback mechanisms on crowd platforms. Fair performance treatment is especially important in crowdsourcing creativity (i.e., generating creative ideas through crowds), where humans are assumed to take responsibility for both generating and evaluating new ideas.

Creative Self-Efficacy and Creative Performance

Individual creativity, in the sense of the generation of novel and useful ideas [4], is a primary driver of organizational innovation [3]. As the global knowledge economy continues to escalate in pace and scope, individual creativity is increasingly at the heart of organizations’ abilities to remain competitive in the marketplace [9]. However, fostering individual creativity can be challenging and complex [33], particularly when it comes to fostering the creative performance of crowdworkers, as their interactions with task requesters are, to a great extent, ambiguous and limited [48].

Comprehensive reviews of creativity, such as those undertaken by George [30]; Shalley, Zhou, and Oldham [59]; and Zhou and Hoever [67], have pointed to two major types of predictors for creative performance: individual factors and contextual factors. In a recent study conducted by Liu et al. [46], involving an extensive meta-analysis of 191 samples, the researchers found that creative self-efficacy as an individual factor is a primary motivational antecedent to creative performances. Similar to general job efficacy, creative self-efficacy refers to an individual’s belief in their ability to succeed in specific situations or to accomplish a task [10]. Unlike general self-efficacy, however, creative self-efficacy does not reflect an individual’s belief in their capability across domains but rather their specific belief in their expertise as required for creative performance [4, 62]. As such, the extent to which individuals endeavor to allocate resources to use their expertise will largely influence their creative outcomes [4].

The greater an individual’s belief in their creative skills, expertise, and knowledge, the more likely they will be to endeavor to apply them [11, 63]. The topic of motivation has been prompted in crowdwork research, because motivation is a multifaceted and impactful element of this type of work (e.g., [71]). We argue that individual beliefs are a salient source of motivation for crowdworkers in determining the level of effort they will invest in their tasks. This idea reflects the probability that, with the lack of social stimulus available in the online crowdwork environment, crowdworkers will rely even more on attributes such as creative self-efficacy to approach their tasks.

Creative self-efficacy reflects an individual’s intrinsic motivation to engage in creative activities [32]. The relationship between creative self-efficacy and creative performance is context dependent [55], influenced by the larger social context in which behavior is enacted. Creativity depends on the individual’s willingness to proactively engage with creative challenges, as well as to persist when faced with challenges and obstacles or roadblocks. Despite an important role of supportive team context, which enables information exchange as a result of peer collaboration and facilitates the relationship between creative self-efficacy and creativity [55], prevailing evidence makes creative self-efficacy as a key factor of generating new and potentially useful ideas [62]. We can thus generally expect that the extent to which creatively self-efficacious individuals endeavor to allocate resources to use their creative expertise will positively influence their creative outcomes.

We therefore posit the following hypothesis:

Hypothesis 1: The creative self-efficacy of crowdworkers is positively related to their creative performance.

The Moderating Role of Digital Performance Feedback Valence

Individuals respond differently depending on the social contexts in which they are situated [11]. Among other contextual factors that may influence the relationship between creative self-efficacy and creative performance, we postulate that digital performance feedback serves as a primary motivational stimulus for crowdworkers working through the mediation of online platforms. The nature of work in the crowdsourcing environment is isolated compared with traditional work settings; the potential for close manager–employee relationships, through feedback and development processes, for example, is likely to be significantly lower [21]. Furthermore, the internet makes it possible for individuals to create ideas in isolation from one another and for their ideas to be subsequently aggregated. Although we acknowledge that certain boundaryless cases may exist, such as crowdworkers actively organizing and reaching out to others in the crowdworking community, these are comparatively small in number. The current reality for most crowdworking setups is that information about the objectives and outcomes of specific task assignments is limited (at least compared with more traditional organizational setups for creativity) and more guidance on how best to use technology to promote organizational goals and individual improvement is needed [21]. As such, the digital performance feedback received by crowdworkers on task assignments is one of the few and limited social cues available to them [30].

Research on feedback points in different directions regarding the effects of feedback on subsequent creative performance [38]. Feedback valence that captures either negative or positive psychological value has been found to produce equivocal outcomes. The idea that feedback, regardless of its direction or nature, can cause null, positive, or negative creative performance further supports perplexing empirical evidence [41, 63]. Moreover, recent research strengthens those equivocal findings, showing that both positive and negative implicit feedback improves the quality of task recommendation systems for crowdsourcing and helps crowdworkers gain new skills [45].

A key advantage of feedback is that it reveals gaps between what a creator intends and what others perceive in their creation, thus serving a critical aspect of creative performance [26]. Acknowledging gaps in creative performance helps performers generate more diverse ideas and iterate higher quality solutions [24]. We argue that positive digital performance feedback valence is a crucial differentiator in terms of creative performance for crowdworkers with low creative self-efficacy, on the basis that individuals not intrinsically inclined to believe they can do well on a creative task ultimately tend to increase their objective creative performances when positive feedback is given. Increases in creative self-efficacy correspond with increases in creative performance when individuals are given feedback over time [63], mostly because feedback interventions can increase both learning and intrinsic motivation [41], because feedback provides opportunities for crowdworkers to make constructive improvements and to excel in their creative skills by learning from “mistakes,” which is unlikely to happen when feedback is inappropriate or not even provided.

For individuals low in creative self-efficacy, positive feedback may strengthen their beliefs in their creative abilities, in turn driving better creative performances [63]. A study by Pajares and Johnson [50] supported the positive influence of both positive-outcome expectancy and self-efficacy on writing performance, which is similar to digital work in being a remote and isolated task. The role of positive feedback for individuals with low creative self-efficacy should thus be particularly valuable in digitally mediated setups, which tend to be more isolated than traditional work settings.

In other words, these moderating effects should be further enhanced in the digital work environment. This logic can be further supported by the classical theory of “rare and important effects” [21] and the theory of “black swan effects” [60], which declare that what rarely comes, and as a surprise, has a robust effect. Accordingly, crowdworkers may interpret their received positive digital feedback regarding their work as surprising (because they are not very confident in their creativity) and as having pronounced significance (because feedback or any constructive interaction concerning their work and performances rarely occurs in a digital setting).

Thus, we posit the following:

Hypothesis 2: Creative self-efficacy and digital performance feedback valence interact in influencing crowdworkers’ creative performance. Crowdworkers with lower levels of creative self-efficacy will exhibit higher levels of creative performance when given positive as opposed to negative digital performance feedback.

The Three-Way Interaction among Creative Self-Efficacy, Digital Performance Feedback Valence, and Mastery Goal Orientation

Our previously posited hypothesis emphasized the beneficial role of positive digital feedback in stimulating creative performance for crowdworkers with low self-efficacy. However, the relationship between creative self-efficacy and creative performance is complex [63]. We argue that how crowdworkers with high versus low creative self-efficacy may respond to negative versus positive digital feedback may depend on their mastery goal orientation. Mastery goal orientation refers to individuals’ perceptual-cognitive frameworks that focus on developing competence and skills when responding to achievement situations [66], such as, in our case, the accomplishment of creative tasks. Research on creative self-efficacy and learning orientation demonstrate that while creative self-efficacy is concerned with one’s confidence in his or her own creative competences, mastery orientation, grounded in an incremental concept of ability, leads to one’s focus on competence development [33]. Therefore, the conjunction of the two beliefs would provide four scenarios, depicted in Figure 1, such that (1) I don’t have the creative competence and such competence cannot be developed; (2) I have the creative competence, there is nothing more I can learn; (3) I don’t have the creative competence but I can develop it as I learn more; and (4) I have the creative competence and I can develop it even more.

Figure 1. Creative Self-Efficacy and Mastery Goal Orientation Matrix

More specifically, we argue that the degree of mastery goal orientation of a crowdworker may trigger that individual to engage in two different psychological processes derived from feedback intervention theory: task-motivation processes and meta-processes [34]. Crowdworkers with high mastery goal orientation are likely to seek learning opportunities based on the digital feedback received and invest effort accordingly in generating better strategies for their creative performance (i.e., task-motivation processes [41]), which leads to self-enhancement and improvement of their future performance [4]. However, for those with low mastery goal orientation, they tend to focus on displaying, instead of developing, their current knowledge and creative competence and trying something new or more novel can be seen as risky because it may fail [66]. Previous studies on creativity have demonstrated that individuals’ creativity decreases when they know their work is to be judged [59], reflecting the feedback dilemma [6]. We consider, therefore, that digital feedback can remind individuals that their work is being judged. Instead of focusing on the tasks, individuals with low mastery goal orientation are more likely to seek to protect their self-image. This psychological state in which their ego or self-image is affected by digital performance feedback is referred to as meta-processes. One reason for this is that when an individual’s work is being evaluated, the person is exposed to the risk of ego cost incurred through receiving negative feedback and being criticized [6]. Such perceived cost often prevents individuals from seeking feedback that provides information about their success or failure in their tasks. This is particularly true for individuals with low mastery goal orientation, as they tend to see their abilities as less malleable, resulting in the forming of personal accounts of the achievement or failure indicated from the feedback [27]. For these individuals, feedback is likely to be perceived as a judgment rather than a learning opportunity [47]. When an individual feels judged, their mastery goal orientation will likely determine how they cope with the fear of being deemed as failing to meet specific standards, that is, whether they respond by reducing their novel ideas to lower potential risks or by striving to improve [16]. This can also be reflected in results from research on person-versus-machine-mediated feedback such that a person-mediated feedback can impose stronger feelings of judgment and subsequently hinder crowdworkers’ performance [40].

Feedback intervention theory argues that task-learning processes may be activated by motivational task processes (in our study, high creative self-efficacy) when feedback is negative. Negative feedback informs crowdworkers that their creative performance should be improved by adopting more novel and useful strategies [30]. Consequently, those with high mastery goal orientation may engage in task-learning processes and perceive negative feedback as an opportunity to master their skills and invest more effort in responding to creativity-standard gaps, as illustrated in scenario 4 in Figure 1. On this basis, we argue that positive digital performance feedback can potentially lead crowdworkers with high creative self-efficacy to reduce potential risks by staying safe, which will hinder their creative performance if their mastery goal orientation is low. Conversely, negative performance feedback will stimulate crowdworkers with high creative self-efficacy and high mastery goal orientation to invest more effort in their tasks. However, for crowdworkers with low creative self-efficacy but high mastery goal orientation (scenario 3), positive feedback is likely to help them feel safe to explore, instead of engaging in meta-processing triggered by negative feedback. On the other hand, the lack of feedback is likely to result in low creativity from crowdworkers who have low mastery goal orientation and low creative self-efficacy (scenario 1). It is because they are likely to expect little of themselves to perform creatively and are not likely to be interested to explore opportunities to do so [47]. Last but not least, lower creativity can also be expected from crowdworkers with low mastery goal orientation but high creative self-efficacy (scenario 2) when they receive digital feedback regardless its valence (negative or positive), as it reminds them that their work is being judged and thus triggers the fear of maintaining their egos [59]. Indeed, research on feedback has shown that in certain circumstances receivers report reduced motivation when feedback is given than when it is not [40]. Given that feedback likely directs the focus of crowdworkers away from meta-processes (self-image threatened by negative feedback) toward task-learning processes, feedback will improve creative performance for those with both high creative self-efficacy and mastery goal orientation, as illustrated in Figure 1.

The preceding propositions are based on two sets of arguments. The first of these is that individuals with low mastery goal orientation tend not to value the learning aspects of the feedback they receive [54]. We argue that this negative tendency toward learning is particularly salient in the digital environment, which has few channels of communication that do not enable multiple cues from other social support [37]. Second, individuals with low mastery goal orientation and high creative self-efficacy may see their creative abilities as less malleable, possibly regarding their creative talents as given rather than learned [36]. As such, they may not see the value of investing more effort, even if they receive positive feedback on their creative performance, since any effort to try something new or creative entails a certain level of risk. Accordingly, crowdworkers with high creative self-efficacy but low mastery goal orientation may feel they have more to lose if they fail and may opt for less novel ideas as a way of preserving their expectations [14].

However, when mastery goal orientation is high, negative digital feedback may serve as encouragement for crowdworkers with high creative self-efficacy, influencing them to try harder, compared with crowdworkers with low creative self-efficacy and thus less motivated to close the creativity-standard gap by improving their current creative performance. This difference in individual motivation arises, because individuals with high creative self-efficacy tend to be more confident in their intrinsic abilities to perform creative tasks [63]. Individuals with high mastery goal orientation often believe they can improve their abilities by trying harder [53], thus making it more likely that they will invest greater effort. We therefore posit the third hypothesis:

Hypothesis 3: Mastery goal orientation will moderate the joint condition of the creative self-efficacy and digital performance feedback valence in influencing creative performance. Digital performance feedback (whether positive or negative) will have a negative effect on creative performance for individuals who exhibit high levels of creative self-efficacy and low levels of mastery goal orientation. However, negative feedback will have a positive effect on creative performance for crowdworkers who exhibit high creative self-efficacy and high levels of mastery goal orientation.

Figure 2 shows our conceptual model with the three hypotheses.

Figure 2. The Conceptual Model with Hypotheses

Methods

Sample

We conducted this study using Amazon Mechanical Turk, a crowdworking platform specializing in tasks such as image classification, translation, and training machine-learning algorithms. The task took approximately 15 minutes to complete. Of the 298 working professionals who participated in the study, 57.5 percent were male, and about 52.5 percent were younger than 45 years of age. All were based in the United States. The majority had acquired a bachelor’s degree (50.5 percent), and their main fields of employment were in the service industry (31.4 percent), technology (28 percent), education (12.1 percent), healthcare (10 percent), and finance (8.8 percent). A total of 47.7 percent had more than nine years of work experience.

Measurements

We used a structured questionnaire with Likert-type five-point scales, from 5 (strongly agree) to 1 (strongly disagree), to measure all the constructs in this study aside from the control variables.

Creative Self-Efficacy

We used Tierney and Farmer’s [62] three-item measure of creative self-efficacy. Sample items included “I have confidence in my ability to solve problems creatively” and “I feel that I am good at generating novel ideas” (α = .93).

Goal Orientation

We assessed goal orientation using nine items adapted from VandeWalle, Cron, and Slocum [64]. Five of these items measured mastery goal orientation (e.g., “I am willing to select a challenging work assignment that I can learn a lot from”). Three assessed the dimension of “proving oneself” as this relates to performance goal orientation (e.g., “I prefer to work on projects where I can prove my ability to others”), and one item measured the “avoidance” dimension of performance goal orientation (e.g., “I’m concerned with showing that I can perform better than my co-workers can”; α = .90).

Control Variables

We controlled for demographic variables such as age, gender, education, and work experience, consistent with the findings of previous studies that have demonstrated these variables might have potential influences on motivational processes [53]. In addition, we controlled for crowdworkers’ creative performances on task 1.

Design and Procedure

We used a 3 × 2 × 2 (no feedback/positive feedback/negative feedback × low/high creative self-efficacy × low/high mastery goal orientation) between-subjects factorial design. Using a two-stage experimental design (the first creative task, feedback manipulation, and the second creative task), we randomly assigned the 298 participants recruited from the crowdworking platform to one of the three conditions of feedback manipulation. Later, within each condition, we split the participants into two groups according to their creative self-efficacy and mastery goal orientation using the split-means method.

We used an online questionnaire to examine the interactions between creative self-efficacy, digital performance feedback valence, and mastery goal orientation. We conducted an online questionnaire in September 2016. We told the participants that the research aimed to generate ideas for a new platform that would be a worker-friendly environment that could serve as an alternative to currently available platforms.

The first part of the survey included the first assigned task and evaluated the participants’ creative self-efficacy. For the first assigned task, participants were asked to think carefully about solutions. This task came with the following instructions:

You are tasked to design a framework to further improve the platform for workers. You need to come up with novel and potentially useful ideas.

Before you develop the details of this framework for the meeting, you have to spend a few minutes coming up with your own list of options.

Using the text below, please take at least five minutes to think carefully and write down a list of ideas for building a great platform for ONLINE (tailored to the participants, such as MTurker) workers.

We then gave the participants positive or negative digital performance feedback immediately after they had completed the first task. The stated positive and negative feedback were worded as follows:

Your ideas of the previous task have now been reviewed. Your creativity score is at THE TOP 20% among those who have taken this test.

Your ideas of the previous task have now been reviewed. Your creativity score is at THE BOTTOM 20% among those who have taken this test.

The participants who received negative digital performance feedback were debriefed after the experiment was completed with the following information:

This was a role-play scenario with instructions for research purposes. The evaluations were made up, and you did a tremendous job. You will receive an additional bonus of $1. Thank you for helping us out. Science thanks you!

These debriefing measures were taken as precautionary steps to prevent negative feedback from misleading participants and subsequently impacting their actual performance in the future or even reducing their confidence and performance in general [13]. The next part of the survey included a second assigned task with a self-assessment of their mastery goal orientation and demographic information. The second task, which all participants performed, involved their selecting and describing the most creative ideas they listed in the first task. This allowed us to assess the ideas that each participant invented and selected as their most creative separately from the other ideas they generated. In this way, we could control for participants’ creative performance on task 1 before the manipulations. The tasks were constructed following the logic applied by Ritter, Van Baaren, and Dijksterhuis [56], and the following instructions were provided: “Please take a few minutes to choose one of the most creative ideas among your list and describe it in detail below. We will evaluate your creativity both in novelty and usefulness.”

Two participants failed to provide descriptions of their ideas despite answering all other mandatory questions and were omitted from further analyses, thus resulting in the final subset of 298. Two independent raters (experts and evaluators in the field of creativity), who were blind to the manipulations and the purpose of the study, assessed each individual’s creative performance on task 2 on a scale from 1 (not at all creative) to 7 (very creative). Generally, the ideas generated for platform improvement can be classified as follows: suggestions relating to pay (e.g., increasing remuneration or using bitcoin), nonmonetary rewards (e.g., advancement, skill development), software (e.g., mobile friendly), platform reputation (e.g., efforts related to increasing perceptions of trust and social media marketing), interface and graphics (e.g., holograms or colors), and social collaboration (e.g., chat rooms or videoconferences among the crowdworkers).

The following suggestion is an example of an idea rated high in creativity:

Christmas Parties and Summer Cook-Outs, letting the employees know they are valued and that the workplace is not meant to be a place of stress. At these events, some ideas that would make them more fun include hosting games, having a vote like in school to a “King and Queen” of the event, and choosing themes for each event like a Christmas Luau or Summer Snow-In.

The following is an example of an idea rated low in creativity:

Hire more chat, phone, and email support to assist online workers, or Payscale distributed based on skill experience, the higher the level, the more payout for tasks.

The two raters’ reliability, ICC(2) = .81, and agreement (average deviation = .47) were within conventional guidelines [43]. We averaged their ratings into a measure of overall creative performance for tasks 1 and 2.

Results

The means and standard deviations of the focal variables for each condition are presented in Tables 1 and 2. The bivariate correlations shown in Table 1 indicate that male participants demonstrated higher creative performance for task 1 (r = −0.12, p < 0.05) in which they were asked to make a list of creative ideas to improve crowdsourcing platforms. However, no gender difference was observed for the creative performance of task 2 (r = 0.02, p > 0.10), in which participants were asked to select one of the most creative ideas listed in task 1 and to develop the idea in detail. In addition, participants with longer work experience reported higher creative self-efficacy (r = 0.17, p < 0.01); in other words, they were more confident in their creative performance. However, work experience was not significantly correlated with either task 1 (r = −0.03, p > 0.10) or task 2 creative performance (r = 0.04, p > 0.10).

Table 1. Means, Standard Deviations, Alpha Reliabilities, and Correlations among Variables

Variables	M	SD	α	1	2	3	4	5	6	7
Age	3.21	0.76	N/A	—
Education	1.64	0.62	N/A	0.06	—
Gender	1.43	0.49	N/A	0.13*	−0.01	—
Work tenure	3.01	1.10	N/A	0.66**	0.01	0.09	—
Task 1 creative performance	3.28	1.26	N/A	−0.02	0.07	−0.12*	−0.03	—
Mastery goal orientation	3.71	0.70	0.90	−0.05	−0.04	0.04	0.01	0.01	—
Creative self-efficacy	4.08	0.82	0.93	0.08	−0.04	−0.04	0.17**	0.00	0.51**	—
Task 2 creative performance	3.07	1.13	N/A	0.01	−0.03	0.02	−0.04	0.11	0.00	−0.01

Note: N = 298. Age was placed into five classes: 1 = younger than 18, 2 = 18–24, 3 = 25–34, 4 = 35–54, and 5 = 55 and older. Education was placed into five classes: 1 = junior high school diploma, 2 = senior high school diploma, 3 = bachelor’s degree, 4 = master’s degree, 5 = doctorate degree. 1 = female, 2 = male.

*p < 0.05. **p < 0.01.

Table 2. Means and Standard Deviations of Task 2 Creative Performance by Feedback Manipulation Conditions

Conditions	Sample size	M	SD
No feedback	98	2.80	1.17
Negative digital feedback	101	2.79	1.02
Positive digital feedback	99	3.59	1.01

Manipulation Check

In terms of manipulation checks for feedback valence (negative feedback and positive feedback vs. no feedback), we assessed its effectiveness using a five-point scale ranging from 1 (not at all) to 5 (extremely) with a single question: “To what extent do you think the committee liked your idea?” The results from the analysis of variance demonstrated that participants in the positive feedback condition were rated highest on average in comparison with the negative-feedback and no-feedback control conditions, F(2, 298) = 18.607, p < 0.01. In comparison with the no-feedback condition M_{[no feedback]} = 2.81, participants in the negative-feedback condition (M_{[negative feedback]} = 2.79) rated the extent to which they thought the committee liked their ideas significantly lower, t(119) = .15, p = 0.88. The effectiveness of the feedback manipulations was therefore deemed satisfactory.

Testing of Hypotheses

We performed the analysis of variance to test our hypotheses, controlling for age, gender, education, work tenure, and participants’ creative performance on task 1. Hypothesis 1 posited that creative self-efficacy is positively related to creative performance (task 2). The results showed that the direct effect of creative self-efficacy on task 2 creative performance was not significant, F(1, 298) = 0.261, p > 0.10, η² = 0.001). Hypothesis 1 is therefore not supported.

Hypothesis 2 posited that digital performance feedback valence moderates the positive relationship between creative self-efficacy and creative performance so that positive digital feedback, compared with negative digital feedback, helps crowdworkers with low creative self-efficacy to perform creative tasks more than it helps crowdworkers with high creative self-efficacy. The results indicated that creative performance in the control group (i.e., the no-feedback condition) was statistically insignificant among crowdworkers with higher creative self-efficacy (M_{[creative performance]} = 2.82, SD = 1.23) and crowdworkers with lower creative self-efficacy (M_{[creative performance]} = 3.27, SD = 1.07), t(96) = −.12, p > 0.10.

However, for crowdworkers with low creative self-efficacy the creative performance was, as predicted, poorer among crowdworkers who received negative feedback (M_{[creative performance]} = 2.56, SD = .89) than among those who received positive feedback (M_{[creative performance]} = 3.20, SD = .95), t = −2.16, p < 0.05. For crowdworkers with high creative self-efficacy, creative task performance was statistically insignificant for those who received negative feedback (M_{[creative performance]} = 3.10, SD = .96) compared with those who received positive feedback (M_{[creative performance]} = 3.59, SD = 1.06), t(99) = −1.78, p > 0.05. As shown in Figure 3, however, creative performance was generally higher among crowdworkers who received positive digital feedback regardless of whether they had high or low creative self-efficacy. The two-way interaction of creative self-efficacy and feedback valence marginally predicted creative performance, F(2, 298) = 2.41, p < 0.10, η² = .014). Hypothesis 2 was therefore supported.

Figure 3. The Two-Way Interaction between Creative Self-Efficacy and Feedback Valence in Predicting Creative Performance

Hypothesis 3 suggested that feedback valence and crowdworkers’ mastery goal orientations would jointly influence the relationship between creative self-efficacy and creative performance. The results indicated that, as predicted, participants with high mastery goal orientation and who received negative digital feedback performed better when they also had high creative self-efficacy (M_{[creative performance]} = 3.03, SD = 1.07) than when their creative self-efficacy was low (M_{[creative performance]} = 2.36, SD = 0.91), t = −2.82, p < 0.01, as shown in Figure 4. For crowdworkers with high mastery goal orientation who received positive digital feedback, creative performance was not statistically different (p > 0.10) between crowdworkers with low creative self-efficacy (M_{[creative performance]} = 3.78, SD = 1.02) and those with high creative self-efficacy (M_{[creative performance]} = 3.59, SD = 1.05). In the low mastery goal orientation scenario, as expected, crowdworkers with high creative self-efficacy performed worse (M_{[creative performance]} = 2.56, SD = 0.72) compared with crowdworkers with low creative self-efficacy (M_{[creative performance]} = 3.57, SD = 0.98) when positive digital feedback was given (t = 2.01, p < 0.10). We also found a similar pattern in the negative digital feedback condition: creative performance was better among crowdworkers with low mastery goal orientation and low creative self-efficacy (M_{[creative performance]} = 3.20, SD = 1.02) than among participants who had low mastery goal orientation but high creative self-efficacy (M_{[creative performance]} = 2.59, SD = 0.69), t = 1.94, p < 0.01. For the control group (no feedback), we found no significant difference in creative performance between crowdworkers with high versus low creative self-efficacy when their mastery goal orientation was high (low creative self-efficacy: M_{[creative performance]} = 2.85, SD = 1.10; high creative self-efficacy: M_{[creative performance]} = 2.71, SD = 1.22), t = 0.48, p > 0.10, and when their mastery goal orientation was low (low creative self-efficacy: M_{[creative performance]} = 2.74, SD = 1.19; high creative self-efficacy: M_{[creative performance]} = 3.60, SD = 0.89), t = −1.54, p > 0.13. The three-way interaction among creative self-efficacy, digital performance feedback valence, and mastery goal orientation yielded a significant prediction regarding creative performance, F(2, 298) = 4.50, p < 0.05, η² = .026, supporting Hypothesis 3.

Figure 4. The Three-Way Interaction between Creative Self-Efficacy, Feedback Valence, and Mastery Goal Orientation in Predicting Creative Performance

Discussion

Our study extends the current understanding of the relationship between creative self-efficacy and creative performance in the digital crowdwork setting. We investigated the intertwined roles of digital performance feedback and individual goal orientations in an attempt to explain the conditions under which the relationship between creative self-efficacy and creative performance may change for crowdworkers. Our findings reveal how the interactions between tasking organizations and online collaborators may subsequently hinder or foster crowdworkers’ creative performance.

The results of our hypothesized relationship between creative self-efficacy and creative performance were similar to those of previous studies conducted in other sectors in that creative self-efficacy did not yield a significant direct effect on crowdworkers’ creative performances. Our results further support other findings that creative self-efficacy alone may not necessarily predict creative performance [55], even for online crowd labor. The different ways in which crowdworkers may capitalize on their beliefs in their creative abilities depend on the extent to which they see learning as an achievement and on the situations (i.e., digital feedback) they encounter. There are two main possible explanations for this finding. First, a crowdworker may not engage in task-process when not given feedback, simply because when feedback is absent, individuals may not see opportunities in it for professional advancement, thus diminishing their motivation to display their creative potentials. Second, creativity can be subjective, and many crowdworkers may not be willing to risk being too creative for fear of harming their performance rating on the platforms. This can be particularly so in the context of online task crowdwork, as in our sample. This classification of crowdwork is to a great extent impersonal [35]. Manager–worker interaction is replaced with microlevel task control, so-called computer control, and that the impersonal rating system may set very restrictive expectations of what outputs would be better rated [27], which hinders creativity. In a regular organizational setting, feedback could be communicated via richer media channels that allow for feedback loop. The lack of such richer communication channels can make the negative aspects of feedback more robust in the context of crowdwork.

Our results also indicate that when crowdworkers’ mastery goal orientation is low but their creative self-efficacy is high, digital performance feedback may stimulate performance apprehension regardless of whether such feedback is positive or negative, resulting in reduced creative performance. This finding illustrates the feedback dilemma [6] as, on one hand, digital performance feedback may serve as a means for future improvement if crowdworkers value the learning opportunities arising from the feedback [7]. On the other hand, digital performance feedback may serve as a reminder that their work is being evaluated, which is particularly harmful to creative performance [6]. Our findings suggest that crowdworkers who have high creative self-efficacy but do not value learning highly (i.e., those with low mastery goal orientation) are more likely to experience digital feedback as an evaluation reminder, thus hindering their creative performance. This finding was further strengthened when we compared the feedback (both negative and positive) groups with the no-feedback control group. In the no-feedback control group, crowdworkers with low mastery goal orientations relied on their creative self-efficacy to drive their creative performance, that is, they relied on what they believed was right based on their previous successes and exerted these creative qualities in their tasks on the platform.

This finding has important theoretical and practical implications. First, creative self-efficacy can either improve or hinder creative performance. In our study, crowdworkers with high creative self-efficacy but low mastery goal orientation were more prone to performance apprehension stemming from digital feedback when performing creative tasks. To foster creative performance, the question is therefore not of how to stimulate crowdworkers’ creative self-efficacy but rather of identifying the conditions under which crowdworkers’ creative self-efficacies can be stimulated and employed. It has been found that creativity is present during all phases of a project but varies in terms of how it is brought about and presented. Moreover, specific leadership skills for encouraging creativity have been shown to emerge at different phases [19]. This finding echoes that of other recent studies (e.g., [55]), challenging the one-size-fits-all assumption and encouraging future theory development in the creative self-efficacy literature to address other potential alternating conditions.

Second, this finding reveals the salience of creative self-efficacy as an internal resource in self-assessing creative tasks [7]. In particular, in the no-feedback control condition, crowdworkers with high mastery goal orientations did not benefit from their creative self-efficacy compared with those low in mastery goal orientation in terms of driving their creative performance. This finding implies that, when assessing their creative tasks, crowdworkers with high mastery goal orientation may rely greatly on external resources such as feedback and less on internal resources such as creative self-efficacy. Although these findings may at first glance seem to contrast with those of studies that have established a connection between general self-efficacy and mastery goal orientation [49], they are less surprising when compared with the extant research on mastery goal orientation in creative tasks. Namely, the relationship between mastery goal orientation and creativity is heavily contingent upon situational cues such as feedback (cf. [18, 33]). These situational cues help individuals to interpret the significance of a creative assignment and understand how best to approach it and other future tasks in terms of capitalizing on their mastery goal orientations.

Another key finding of the current study is that negative digital feedback might function as a motivator for crowdworkers with high mastery goal orientation and high creative self-efficacy to try harder when performing creative tasks. In general, negative feedback has been considered harmful to creative performance [59], primarily on the basis that individuals receiving negative feedback tend to respond to negativity with negativity [44] by losing confidence in their competence [64] and becoming unmotivated to carry out their tasks further [63]. However, our finding suggests that if crowdworkers have strong beliefs in their abilities, combined with high mastery goal orientation and the tendency to look out for frequent learning opportunities, negative feedback might make them want to try harder in their creative tasks. In this sense, feedback (positive or negative) appears to be beneficial in terms of crowdworkers’ task outcomes compared to conditions where no feedback is given on performed tasks. This is a particularly important contribution to crowdwork literature, since crowdworkers are exposed to a greater amount of rejections of a type not seen in more traditional work settings, including unfair rejections, payment refusals and blocks without feedback, which hinders their opportunities to improve performance and pursue new tasks in the future [13].

This finding on negative feedback has two important implications. First, negative feedback may not necessarily be entirely a bad thing. The dynamic function of negative feedback has important practical implications, because managers sometimes need to deliver bad news or evaluate a worker’s poor performance, which points to the importance of stimulating a learning culture within a team or organization and particularly within an online collaborative environment [33]. Most platform providers do not invest in supporting workers in areas such as training and collaboration, but these traditional efforts are still needed and should be provided in new and digitally-accommodating ways [69]. Companies that have attempted to embrace a more balanced two-way communication loop with crowdworkers have found it to be well received [23]. This finding also highlights the importance of applying the Pygmalion effect by expressing confidence in crowdworkers’ future creative performances [62] to buffer potential negative responses. Second, the findings generally support the idea that individual creativity relies not only on talent but also on effort. A key implication for creativity research is that, at least in crowdsourcing environments, a person’s focus on achieving learning and development outcomes through their tasks is more relevant for the stimulation of creative performance than the presence of abilities alone [33]. This is particularly true when crowdworkers have the predispositions needed to respond adequately to negative feedback and invest further effort in their development.

Limitations, Future Research Directions, and Conclusions

This study should be considered in light of its limitations. One limitation of this research is related to the small number of participants in some experimental cells, particularly when we used the split-means approach on the constructs measured (i.e., mastery goal orientation and creative self-efficacy) and when we delineated the participants into groups based on these mean scores. Given that we investigated complex three-way interaction scenarios, it is only natural that some conditions denoted in the combinations of our independent focal variables would not entail a large number of participants. Despite this limitation, our study adds knowledge to the field in terms of understanding these interactions. Future research should nonetheless focus on manipulating all of our studied independent factors to achieve a more balanced sample size in each of the investigated cells.

We also focused entirely on one aspect of goal orientations—the approach dimension. Future researchers could look into the performance-avoidance goal orientation and examine how this interacts with feedback types and creative self-efficacy to influence creative performance. The role of supervisors, as well as their perceived support and fairness [17]—albeit from a distance and via digital means—should also be further investigated. An exploration of the interactive role of personal (creative self-efficacy and goal orientation) and contextual factors (feedback) in crowdworkers’ creative performance is a relevant approach to take when attempting to understand and capitalize on the creative potential of digital labor. It would be advantageous to invest further in understanding these processes in the context of crowdwork. Taken together, the results of our study highlight that the relationship between creative self-efficacy and crowdworkers’ creativity can be complex. While there seems to be no one-size-fits-all practice in terms of how to foster crowdworkers’ creative performance, we observe that overall, crowdworkers benefit from having high levels of mastery goal orientation. In particular, crowdworkers with high creative self-efficacy are less likely to respond to digital feedback positively if they have low mastery goal orientation. On the other hand, for crowdworkers with low creative self-efficacy, receiving positive digital feedback helps their creative performance regardless of whether they have high or low levels of mastery goal orientation. Accordingly, task requesters and crowdsourcing platforms are recommended to ensure a positive learning environment to stimulate high master goal orientation with positive feedback to encourage crowdworkers to engage in creative work.

Additional information

Funding

This work was supported by the Research Council of Norway under the Toppforsk project “Future Ways of Working in the Digital Economy” (Project numbers: 275347) and the Slovenian Research Agency (Project numbers: J5-9329, J5-2555, and P5-0410)

Notes on contributors

Sut I Wong

Sut I Wong ([email protected]) is a professor at BI Norwegian Business School, Department of Communication and Culture. Her research interests center on leadership, employee proactivity, team innovation and performance. Her current research focuses on the role of leadership in better engaging employees in organizational change, such as digitalization, as well as on how ideal leadership models may influence the interaction between leaders and their followers during change. Her work has been published in such journals as Journal of Management, The Leadership Quarterly, Human Resource Management, Human Resource Management Journal, and Journal of Occupational and Organizational Psychology.

Aldijana Bunjak

Aldijana Bunjak ([email protected]; corresponding author) is postdoctoral research fellow at the IFPM ‑ Institute for Leadership and Human Resource Management at the University of St. Gallen, Switzerland. She received her Ph.D. from the University of Ljubljana, Faculty of Economics, Slovenia. Her research focuses on cognitive processes, mindfulness at work, leadership, and digitalization at work.

Matej Černe

Matej Černe ([email protected]) is an associate professor at the University of Ljubljana, School of Economics and Business, Slovenia. His research interests include non-technological innovations, creativity, organizational behavior and psychology, leadership, and multi-level issues in management. His work has been published in such journals as Academy of Management Journal, Human Resource Management, Journal of Organizational Behavior, and The Leadership Quarterly.

Christian Fieseler

Christian Fieseler([email protected]) is a professor of communication management at BI Norwegian Business School and a director of the Nordic Centre for Internet and Society. He received his Ph.D. in management and economics from the University of St. Gallen, Switzerland. His research is focused on the question of how individuals and organizations adapt to the shift brought by new, social media, and how to design participative and inclusive spaces in this new media regime. He has worked extensively on technology and new working modes in projects with the European Union and the Norwegian Research Council.