How do ideas gain legitimacy in internal crowdsourcing idea development? Exploring the effects of feedback on idea selection

Figures & data

Figure 1. Research model.

Table 1. Variables and measurement.

Type	Item	Second-order item	Measurement	Supported references
Dependent variable	Idea Acceptance		Whether the idea is accepted for interest, action, or implementation, where a value of 1 represents that the idea has been accepted, and a value of 0 means that the idea has been rejected	(Beretta 2019, Chan et al., 2021)
Dependent variable in idea development, Independent variable in idea selection	Idea Revision	Idea change	The number of editions after the first comment	(Chen et al., 2020)
Independent variable (IV)	Feedback sentiment	Positive sentiment	The proportion of positive words	(Beretta, 2019)
		Negative sentiment	The proportion of negative words	(Beretta, 2019)
	Feedback diversity	Comment entropy	The topic distribution – Sharon entropy of all comments contributed to each idea.	(Jost, 2006; Masisi et al., 2008)
	Feedback amount	Total comment length	The number of words in comments contributed to each idea	(Beretta, 2019; Chan et al., 2021; Ogink & Dong, 2019)
Moderator	Feedback provider	Manager	The comment numbers from manager	(Chan et al., 2021; Chen et al., 2020;Wooten & Ulrich, 2017)
		Expert	The expertise of feedback providers, measured based on the topic distribution and the overlap degree between the individual’s contributions and the commented ideas (see formula (5) and (6))
Control variables	Idea	Idea length	Number of words in idea description	(Beretta, 2019; Chen et al., 2020)
		Idea sentiment	Sentiment of ideas is calculated by the number of positive and negative words in an idea description	(Beretta, 2019; Chan et al., 2021)
		Idea relatedness	Whether idea is related to other ideas or not	(Beretta, 2019; Chen et al.,2020)
	Idea attention	Feedback	Yes: Idea has received feedback No: Idea has not received any feedback	(Chan et al., 2021; Wooten & Ulrich, 2017)
	Feedback timeliness	First comment speed	Based on the time to the first feedback. The first comment speed refers to the inverse of the time to first feedback
		Time to feedback	Average time to each feedback comment	(Chen et al., 2020)

Table 2. Examples of topics.

Topic 1	Topic 2	Topic 3	Topic 4	Topic 5	Topic 6	Topic i
traffic	save	network	code	system	ericsson	… …
perform	cost	use	design	use	servic	… …
system	time	node	correct	model	prototyp	… …
load	will	instal	new	base	potenti	… …
reduc	per	server	block	approach	global	… …

Table 3. Matrix correlation of all variables.

	1	2	3	4	5	6	7	8	9	10	11	12	13
1 Idea sentiment	1.00
2 Idea length	−0.09***	1.00
3 Idea relatedness	−0.01	0.03***	1.00
4 Idea attention	−0.04***	0.06***	0.07***	1.00
5 First comment speed	−0.01	−0.01	0.00	0.16***	1.00
6 Time to feedback	0.04***	−0.06***	−0.06***	−0.99***	−0.14***	1.00
7 Positive sentiment	−0.03**	0.06***	0.05***	0.53***	0.13***	−0.53***	1.00
8 Negative sentiment	−0.03**	0.04***	0.03***	0.33***	0.04***	−0.33***	0.25***	1.00
9 Feedback diversity	0.00	0.01	0.00	0.00	−0.08***	0.00	−0.43***	−0.04***	1.00
10 Feedback amount	−0.04	0.07***	0.07***	0.57***	0.09***	−0.57***	0.78***	0.50***	−0.20***	1.00
11 Manager	−0.02	0.00	0.04**	0.36***	0.03***	−0.37***	0.23***	0.17***	0.01***	0.29***	1.00
12 Expertise	−0.01	0.05***	0.02*	0.19***	0.04***	−0.19***	0.18***	0.13***	−0.04	0.24***	0.32***	1.00
13 Idea revision	−0.02	0.12***	0.06***	0.17***	0.06***	−0.16***	0.17***	0.11***	0.01***	0.21***	0.08***	0.06***	1.00
14 Idea acceptance	0.00	0.00	−0.01	0.10***	0.03**	−0.10***	0.06***	0.03***	0.02	0.12***	0.09***	0.00	0.04***	1.00
Feedback or not

Signif. codes: 0.001 ‘***’ 0.01 ‘**’ 0.05 ‘*’.

Table 4. Linear regression results for idea development.

	Variables	Model1	Model2	Model3	Model4	Model 5	Model6
Idea	Idea length	1.60e-05***	1.54e-05***	1.25e-05***	1.24e-05***	1.28e-05***	1.36e-05***
	Idea sentiment	−3.80e-04	−1.87e-04	−2.46e-04	−2.58e-04	−2.72e-04	−2.49e-04
	Idea relatedness	3.73e-05***	3.52e-03***	1.12e-03***	3.34e-02***	1.64e-03***	1.04e-03***
Idea attention	Yes/No feedback	1.48e-01**	1.52e-01*	1.5e-01	1.37e-01	1.36e-01	1.51e-01*
Feedback timeliness	First comment speed	2.38e-04***	4.14e-04***	4.12e-04***	4.67e-04***	4.70e-04***	4.54e-04***
	Time to feedback	2.53e-10	3.93e-10	4.19e-10	4.43e-10	4.39e-10	4.14e-10
Feedback sentiment	Positive sentiment(H2a)		1.80e-02***	2.83e-02***	9.38e-03*	9.28e-03*	1.90e-03
	Negative sentiment(H2b)		7.09e-02***	3.75e-02*	−1.56e-02	−1.38e-02	−2.75e-02
Feedback diversity	Comment entropy (H2d)			6.34e-02***	6.31e-02***	6.28e-02***	5.54e-02***
Feedback amount	Total comment length(H2c)				1.21e-03***	1.29e-03***	7.77e-03**
Feedback provider	Manager					1.93e-02	1.08e-01*
	Expert					−3.00e-04***	−2.75e-02
Moderation effects of feedback provider	Total comment length: Positive sentiment						1.16e-04***
	Total comment length: Negative sentiment						−2.54e-04
	Manager: total comment length						−5.40e.03**
	Expert: total comment length						3.41e-05
	Manager: positive sentiment						3.95e-02***
	Manager: negative sentiment(H4c)						−2.32e-02
	Expert: positive sentiment						−1.40e-03**
	Expert: negative sentiment(H4d)						−3.50e-03
	Manager: positive sentiment: Total comment length (H4a)						−1.53e-04***
	Manager: negative sentiment: Total comment length						6.33e-05
	Expert: positive sentiment: Total comment length (H4b)						1.72–06***
	Expert: negative sentiment: Total comment length						1.48e-05
	R2	0.043	0.053	0.055	0.063	0.066	0.073

Significance levels (p): 0.001 ‘***’ 0.01 ‘**’ 0.05 ‘*’.

Figure 2. Moderation effects of managers.

Figure 3. Moderation effects of experts.

Table 5. Logistic regression results for idea selection.

	Variables	Model1	Model2	Model3	Model4	Model5	Model6	Model7
Idea	Idea length	−1.88e-05	−2.58e-05	−2.58e-05	−2.58e-05	−2.11e-05	−2.27–05	−2.22–05
	Idea sentiment	3.18e-04	2.30e-04	2.18e-04	2.19e-04	4.93e-04	4.17e-04	4.61e-04
	Idea relatedness	−1.04e −02	−9.44e −03	−8.16e −04	−8.14e −04	−7.53e-03	−1.97e-02	−2.52e-02
Idea attention	Yes/No feedback	1.58e + 00**	1.58e + 00***	1.58e + 00***	1.58e + 00***	1.49e + 00**	1.51e + 00**	1.53e + 00**
Feedback timeliness	First comment speed	−7.18e-03**	−8.16e-03**	−8.28e-03**	−8.42e-03**	−7.05e-03***	−6.68e-03***	−6.53e-03***
	Time to feedback	2.55e-09	2.66e-09	2.61e-09	2.73e-09	2.94e-09	3.02e-09	3.02e-09
Idea revision	Idea change		2.62e-01**	2.55e-01**	2.56e-01**	2.09e-01**	1.83e-01**	1.75e-01*
Feedback sentiment	Positive sentiment (H1a)			1.89e-02	1.87e-02	−9.31e-02	−1.06e-01	−9.61e-02
	Negative sentiment (H1b)			−1.27e-01	−1.28e-01	−4.68e-01***	−4.32e-03**	−3.92e-02*
Feedback amount	Total comment length (H1c)					6.99–03***	7.92e-03***	7.46e-03***
Feedback diversity	Comment entropy(H1d)				−1.43e-03	5.66e-02	−4.77e-03	9.53e-02
Feedback providers	Manager						3.20e-01**	4.40e-01**
	Expert						−4.05e-02***	−7.81e-02***
Moderation effects of feedback provider	Manager: positive sentiment (H5a)							−2.41e-02
	Manager: negative sentiment (H5c)							1.59e-02
	Expert: positive sentiment (H5b)							4.84e-03***
	Expert: negative sentiment (H5d)							−1.04e-03
	R2	0.030	0.034	0.034	0.035	0.049	0.059	0.062

Significance levels (p): 0.001 ‘***’ 0.01 ‘**’ 0.05 ‘*’.

Figure 4. The moderation effects of expertise on feedback sentiment.

Table 6. Mediation test results.

Relationship	Mediator	Moderator	Effect	Estimate	95% CI Lower	95% CI Upper	p-value
Feedback sentiment (Positive)→ Idea acceptance(H3a)	Idea revision	Manager	Indirect	5.90e-04	3.66e-05	0.01	0.03**
			Direct	2.96e-03	−2.47e-03	0.01	0.25
			Total	3.56e-03	−1.79e-03	0.01	0.18
		Expertise	Indirect	5.72e-04	2.44e-05	0.01	0.04**
			Direct	2.06e-03	−2.92e-03	0.01	0.40
			Total	2.64e-03	−2.32e-03	0.01	0.29
Feedback sentiment (Negative)→ Idea acceptance(H3b)	Idea revision	Manager	Indirect	1.72e-03	8.48e-06	0.01	0.046**
			Direct	−7.39e-03	−4.54e-02	0.02	0.61
			Total	−5.6e-03	−4.37e-02		0.69
		Expertise	Indirect	0.001	−0.0001	0.01	0.09*
			Direct	−0.007	−0.004	0.03	0.60
			Total	−0.0058	−0.037	0.03	0.68
Feedback amount→ Idea acceptance(H3c)	Idea revision	Manager	Indirect	2.52e-05	−5.58e-06	0.00	0.058*
			Direct	3.76e-04	2.97e-04	0.01	<2e-16 ***
			Total	4.01e-04	1.99e-04	0.00	<2e-16 ***
		Expert	Indirect	2.54e-05	−3.28e-06	0.00	0.086*
			Direct	3.50e-04	1.38e-04	0.01	<2e-16 ***
			Total	3.75e-04	1.67e-04	0.01	<2e-16 ***
Feedback diversity→ Idea acceptance(H3d)	Idea revision	//	Indirect	1.3e-03	−3.2e-04	0.00	0.098*
			Direct	8.1e-03	−0.013	0.06	0.44
			Total effect	9.0e-04	−0.012	0.06	0.37

Significance levels (p): 0.001 ‘***’ 0.01 ‘**’ 0.05 ‘*’.

Beretta, M. (2019). Idea selection in web‐enabled ideation systems. Journal of Product Innovation Management, 36(3), 5–23. https://doi.org/10.1111/jpim.12439

 Google Scholar

Chan, K. W., Li, S. Y., Ni, J., & Zhu, J. J. (2021). What feedback matters? The role of experience in motivating crowdsourcing innovation. Production and Operations Management, 30(1), 103–126.

 Web of Science ®Google Scholar

Chen, Q., Magnusson, M., & Björk, J. (2020). Collective firm-internal online idea development: Exploring the impact of feedback timeliness and knowledge overlap. European Journal of Innovation Management, 23(1), 13–39. https://doi.org/10.1108/EJIM-02-2018-0045

 Web of Science ®Google Scholar

Jost, L. (2006). Entropy and diversity. Oikos, 113(2), 363–375. https://doi.org/10.1111/j.2006.0030-1299.14714.x

 Web of Science ®Google Scholar

Masisi, L., Nelwamondo, V., & Marwala, T. (2008, November). The use of entropy to measure structural diversity. 2008 IEEE International Conference on Computational Cybernetics, Stara Lesna, Slovakia (pp. 41–45). IEEE.

 Google Scholar

Ogink, T., & Dong, J. Q. (2019). Stimulating innovation by user feedback on social media: The case of an online user innovation community. Technological Forecasting and Social Change, 144, 295–302. https://doi.org/10.1016/j.techfore.2017.07.029

 Web of Science ®Google Scholar

Wooten, J., & Ulrich, K. (2017). Idea generation and the role of feedback: Evidence from field experiments with innovation tournaments. Production and Operations Management, 26(1), 80–99. https://doi.org/10.1111/poms.12613

 Web of Science ®Google Scholar