Agile work practices: measurement and mechanisms

Figures & data

Table 1. A conceptual analysis of agile work practices.

	Potential underlying mechanisms
Agile work practices	Temporal linkages	Managerial control	Team processes	Work design
Sprints	Entrainment	Concertive control instead of centralized bureaucracy	Emergent taskwork planning	Vertical loading (e.g., autonomy)
Iterative development	Probing the future
Stand-up meetings	Managing the present		Emergent teamwork planning	Open feedback channels
Retrospective meetings	Learning from the past

Table 2. Overview of sample and participant characteristics.

Sample	Teams	Employees	Teamsize	Team response	Demographics	Work content	Team experience	Agile transformation
A. Convenience	Not applicable	163	Not applicable	Not applicable	Mostly women (66%), who were between 35 and 55 years old (50%), and held a Bachelor degree or higher (78%). Most participants were German native speakers, while 36 completed the survey in English.	Various sectors, primarily logistics (16%), IT (14%), construction (10%).	71% of the participants had been a member of their team for more than 6 months.	39% of the participants described their team as an “agile” or “rather agile” team.
B. AgileOrg 2019 teams	114	476	M = 9.51, SD = 2.41	Min. = 2, Max. = 9, Median = 40% per team.	Mostly men (78%), who were between 35 and 55 years old (55%), and held a Bachelor degree or higher (64%).	87 delivery teams (n = 365); 27 support teams (n = 111).	92% of the participants had been a member of their team for more than 6 months.	42% of the teams were in an advanced phase of agile transformation.
C. AgileOrg 2019–2020 longitudinal	71	302	M = 12.20 SD = 2.10	Min. = 2, Max. = 9, Median = 40% per team	Mostly men (71%), who were between 35 and 55 years old (51%), and held a Bachelor degree or higher (66%).	63 delivery teams (n = 260); 8 support teams (n = 42).	96% of the participants had been a member of their team for more than 6 months.	75% of the teams were in an advanced phase of agile transformation.
D. AgileOrg 2020 new teams	155	657	M = 11.54 SD = 2.51	Min. = 2, Max. = 11, Median = 43% per team	Mostly men (65%), who were between 35 and 55 years old (46%), and held a Bachelor degree or higher (65%).	126 delivery teams (n = 521); 29 support teams (n = 136).	88% of the participants had been a member of their team for more than 6 months.	48% of the teams were in an advanced phase of agile transformation.

Note. The transformation at AgileOrg is split up in four phases resembling the forming-storming-norming-performing model by (Tuckman, 1965) and the teams’ progress in the transformation is registered in the company’s HR system. We recoded the ordinal agile transformation variable into a binary dummy variable representing the beginning (forming-storming) and advanced (norming-performing) phases of the transformation at AgileOrg. This allowed for more easily interpretable known-group comparisons.

Table 3. Items and factor loadings of the APWI in Sample A.

Item and factor loading		1	2	3	4
ID1	We develop a prototype/pilot before expanding our plans.				.55
ID2	We experiment with different ideas before settling on an approach.				.62
ID3	We refine our initial ideas successively.				.63
ID4	We quickly adapt our approach to changing requirements.				.56
ID5	We regularly ask customers/clients for feedback on the progress of our work.				.63
ID6	We frequently check whether the demands of our customers/clients have changed.				.76
ID7	When we start working on an assignment, we often do not know what the final product will look like.				.59
SI1	We plan our work activities in short sequences.			.64
SI2	We limit the duration of our work cycles to less than 1 month.			.73
SI3	We try to reduce uncertainties by keeping work cycles short.			.83
SI4	We try to increase flexibility by keeping work cycles short.			.74
SI5	We plan our tasks in short cycles (e.g., weekly).			.64
DS1	We have a short meeting to discuss new developments in our tasks.	.87
DS2	We have a short meeting to discuss impediments that hinder us from completing our tasks.	.84
DS3	We have a short meeting to discuss the progress of our work.	.88
DS4	We have a short meeting to discuss difficulties in our tasks.	.90
DS5	We have a short meeting to inform each other about what we are working on.	.81
DS6	We have a short meeting to inform each other about relevant issues.	.84
DS7	We have a short meeting to discuss changes in our tasks.	.92
RR1	We take our time to appreciate each other for our efforts.		.72
RR2	We take our time to talk about what went well in the team.		.90
RR3	We take our time to discuss our work processes.		.79
RR4	We take our time to critically reflect on our work activities.		.79
RR5	We take our time to improve the social dynamics of the team.		.78
RR6	We take our time to discuss ways to improve team performance.		.61
RR7	We take our time to discuss feedback from our customers/clients.		.53		.24

Note. The boldfaced items were used from Phase 2 onward. Response format agile taskwork items: 1 (fully disagree), 2 (disagree), 3 (slightly disagree), 4 (indecisive), 5 (slightly agree), 6 (agree), 7 (fully agree). Response format agile teamwork items: 1 (almost never), 2 (few times a year), 3 (once a month), 4 (few times a month), 5 (once a week), 6 (few times a week), 7 (daily).

Table 4. Pearson product-moment correlations of variables measured in Sample A.

Variable	N	M	SD	1	2	3	4	5	6	7	8	9	10	11	12	14	15	16
1. Iterative development	163	3.50	0.76
2. Sprints	163	3.24	0.94	.51**
3. Stand-up meetings	163	3.20	1.09	.39**	.50**
4. Retrospective meetings	163	2.78	0.96	.31**	.34**	.49**
5. Agile team	150	3.16	1.22	.50**	.49**	.41**	.43**
6. Scrum	150	2.95	2.17	.34**	.52**	.45**	.37**	.62**
7. Kanban	150	3.01	2.03	.26**	.25**	.25**	.21**	.37**	.45**
8. Extreme Programming	150	1.65	1.30	.21**	.28**	.23**	.23**	.40**	.42**	.29**
9. Design Thinking	150	2.67	1.85	.44**	.22**	.23**	.23**	.39**	.26**	.36**	.38**
10. Lean Development	150	2.30	1.71	.38**	.28**	.25**	.23**	.42**	.34**	.31**	.53**	.51**
11. Waterfall Development	150	2.09	1.54	.24**	.09	.18*	.07	.26**	.26**	.13	.37**	.29**	.42**
12. Task interdependence	163	3.74	0.78	.21**	.15	.15	.04	.05	−.07	.06	−.13	.01	−.03	.10
14. Service/production team	163	0.31	0.47	−.03	.15	.10	.09	.08	.08	.04	.08	−.07	.06	−.04	−.09
15. Consulting team	163	0.28	0.45	.00	−.10	−.12	−.10	−.02	−.03	−.01	.04	.14	.05	.04	−.11	−.36**
16. Project (time-limited) team	163	0.37	0.48	.18*	.10	.09	−.01	.09	.03	−.03	.07	.06	.08	.09	.18*	−.49**	−.31**
17. Management team	163	0.12	0.33	−.08	−.12	−.04	.04	−.08	−.06	.00	−.13	−.08	−.05	−.04	.05	−.25**	−.19*	−.13

Note. Variables 14 to 17 are coded in a binary way (0 = no, 1 = yes), N = sample size, M = mean, SD = standard deviation, ** p < .01, * p < .05

Table 5. Fit of confirmatory factor models in Sample B.

Model	N	df	AIC	BIC	Δχ^2	CFI	TLI	RMSEA	SRMR
1. Within-team	476	293	30,553	30,903	859.23***	.875	.861	.064	.054
2. Between-team	114	293	5784	6014	631.16***	.874	.861	.101	.091
3. Multilevel	476	586	32,374	32,952	1010.93***	.924	.916	.041	w = .061; b = .149
4. Rev. within-team	476	98	19,198	19,423	218.19***	.953	.942	.051	.047
5. Rev. between-team	114	98	3537	3685	156.69***	.963	.955	.072	.072
6. Rev. multilevel	476	196	20,760	21,137	260.14**	.981	.977	.027	w = .050; b = .079

Note. *** p < .001, ** p < .01. Rev. = revised models after retaining only items with high between-team factor loadings (from Model 4 onward).

Table 6. Internal consistency, group mean stability, median within-team agreement, and test-retest reliability.

	Reliability	α	ωh	ICC2	rwgj	rt+1year
Scale	Agile taskwork long (12 items)	.91	.83	.80	.95
	Agile taskwork (8 items)	.90	.83	.79	.94	.72
	Agile teamwork long (14 items)	.94	.87	.79	.98
	Agile teamwork (8 items)	.90	.85	.77	.96	.58
Finalsubscales	Iterative development (4 items)	.84	.87	.66	.90	.62
	Sprints (4 items)	.95	.95	.80	.88	.73
	Stand-up meetings (4 items)	.93	.94	.75	.94	.56
	Retrospective meetings (4 items)	.91	.90	.64	.93	.43

Note. The items used in the long and short versions are shown in Table 3. The test-retest reliability was calculated using the data of Sample C, all other values are based on data of Sample B.

Table 7. Higher-order factor models and measurement invariance tests in Sample B and D.

Model and sample			χ^2	df	CFI	RMSEA	ΔCFI	ΔRMSEA	Δχ^2
A. Higher-order models	Sample B and D combined
Hypothesized second-order	n = 1133		567.36***	99	.954	.065
One second-order			583.64***	100	.952	.065	−.002	.000	16.28***
Uncorrelated factors			1206.78***	104	.891	.097	−.061	.032	623.14***
B. Samples	Sample B (2019)	Sample D (2020)
Configural	n = 476	n = 657	747.09***	198	.946	.070
Metric			772.91***	212	.945	.068	−.001	−.002	25.82*
Scalar			837.45***	222	.940	.070	−.005	.002	64.54***
C. Team types	Delivery teams	Support teams
Configural	n = 886	n = 247	663.96***	198	.953	.064
Metric			682.54***	212	.953	.063	.000	−.001	18.59
Scalar			733.82***	222	.949	.064	−.004	.001	51.27***
D. Agile transformation	Beginning	Advanced
Configural	n = 580	n = 553	686.69***	198	.951	.066
Metric			704.52***	212	.951	.064	.000	−.002	17.83
Scalar			728.12***	222	.950	.063	−.001	−.001	23.60**

Note. According to Chen (2007), ΔCFI ≤ −.01 supplemented by ΔRMSEA ≥ .015 indicate non-invariance, *** p < .001, ** p < .01, * p < .05

Figure 1. Observed group differences on the AWPI scale scores based on Welch’s t-tests.

Note. We report Cohen’s d as a standardized measure of effect size, *** p < .001, ** p < .01, * p < .05, ns = not significant.

Table 8. Pearson product-moment correlations of variables measured in Sample D.

Variable	N	M	SD	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19
1. Teamsize	155	11.54	2.51
2. Team type	155	.19	.39	−.04
3. Agile transformation	155	.48	.50	.22**	.27**
4. Agile taskwork	155	4.6	.84	−.06	−.16*	.04
5. Agile teamwork	155	4.61	.72	−.07	−.02	.04	.60**
6. Iterative development	155	4.53	.70	−.13	−.10	−.02	.72**	.38**
7. Sprints	155	4.67	1.27	−.01	−.16*	.06	.92**	.58**	.40**
8. Stand-up meetings	155	5.75	.91	−.03	−.04	.06	.55**	.91**	.30**	.56**
9. Retrospective meetings	155	3.46	.72	−.09	.01	.01	.51**	.85**	.39**	.46**	.56**
10. Bureaucracy	155	2.19	.64	−.08	.05	−.02	−.21**	−.17*	−.29**	−.11	−.09	−.22**
11. Taskwork planning	155	5.19	.64	−.05	−.03	−.06	.47**	.48**	.55**	.32**	.37**	.49**	−.40**
12. Teamwork planning	155	5.04	.70	−.08	.05	.03	.34**	.38**	.44**	.20*	.30**	.38**	−.26**	.66**
13. Autonomy	155	5.56	.63	−.06	.09	−.07	.19*	.12	.32**	.07	.03	.20*	−.49**	.33**	.32**
14. Feedback	155	4.59	.73	−.12	.04	.04	.40**	.49**	.43**	.30**	.33**	.55**	−.36**	.56**	.57**	.39**
15. AC agile taskwork	66	4.55	1.26	.14	.01	.25*	.53**	.35**	.23	.55**	.27*	.37**	−.05	.16	−.01	−.23	.17
16. AC agile teamwork	66	4.68	.89	.24	.07	.14	.33**	.31*	.13	.35**	.27*	.27*	−.17	.17	−.15	−.37**	.04	.58**
17. AC iterative development	66	4.45	1.09	.05	.00	.21	.35**	.15	.25*	.32**	.12	.14	−.18	.23	.14	−.08	.19	.79**	.43**
18. AC sprints	66	4.66	1.79	.17	.01	.24	.54**	.41**	.18	.59**	.30*	.44**	.04	.09	−.09	−.27*	.13	.93**	.56**	.51**
19. AC stand-up meetings	66	5.73	1.13	.30*	−.03	.16	.30*	.32**	.04	.36**	.33**	.22	−.1	.07	−.24*	−.40**	.03	.53**	.88**	.40**	.51**
20. AC retrospective meetings	66	3.62	.97	.10	.16	.07	.25*	.20	.18	.23	.12	.24*	−.19	.24*	0	−.22	.05	.45**	.83**	.33**	.44**	.46**

Note. Variable 15 to 20 represent agile coach (AC) ratings available for 66 teams. Dotted cells represent the heterotrait-monomethod correlations (i.e., discriminant validity), grey cells represent monotrait-heteromethod correlations (i.e., convergent validity). Cells inside the double-lined box represent heterotrait-heteromethod correlations. Team type is a binary dummy variable (0 = delivery, 1 = support), as is the agile transformation variable (0 = beginning, 1 = advanced), N = sample size, M = mean, SD = standard deviation, ** p < .01, * p < .05.

Table 9. Multilevel regression coefficients.

Model and level of analysis			Taskwork planning			Teamwork planning			Team autonomy			Team feedback
			γ	SE	p	γ	SE	p	γ	SE	p	γ	SE	p
A	L1:	Intercept	3.05	.30	.000	3.16	.35	.000	4.91	.31	.000	2.19	.34	.000
		Agile taskwork (w)	.47	.05	.000	.40	.05	.000	.14	.05	.004	.27	.05	.000
		Agile teamwork (w)	.30	.05	.000	.21	.06	.000	−.01	.05	.863	.33	.06	.000
	L2:	Agile taskwork (b)	.24	.07	.001	.18	.08	.017	.18	.07	.010	.19	.08	.013
		Agile teamwork (b)	.24	.08	.003	.22	.09	.014	−.04	.08	.657	.32	.09	.000
		Team type (0 = delivery team)	.09	.11	.457	.15	.13	.257	.22	.12	.065	.15	.13	.228
		Transformation (0 = beginning)	−.11	.09	.209	−.03	.10	.751	−.10	.09	.277	.01	.10	.924
B	L1:	Intercept	3.05	.30	.000	3.16	.35	.000	4.91	.31	.000	2.19	.34	.000
		Agile taskwork (raw)	.47	.05	.000	.40	.05	.000	.14	.05	.004	.27	.05	.000
		Agile teamwork (raw)	.30	.05	.000	.21	.06	.000	−.01	.05	.863	.33	.06	.000
	L2:	Agile taskwork (b)	−.23	.08	.005	−.22	.09	.017	.04	.08	.661	−.08	.09	.384
		Agile teamwork (b)	−.07	.09	.488	.01	.10	.901	−.03	.10	.784	−.01	.11	.929
		Team type (0 = delivery team)	.09	.11	.457	.15	.13	.257	.22	.12	.065	.15	.13	.228
		Transformation (0 = beginning)	−.11	.09	.209	−.03	.10	.751	−.10	.09	.277	.01	.10	.924
C	L2:	Intercept	3.15	.44	.000	3.41	.49	.000	4.92	.44	.000	2.05	.49	.000
		Agile taskwork (b)	.21	.11	.051	.12	.12	.295	.09	.11	.419	.17	.12	.158
		Agile teamwork (b)	.24	.12	.042	.23	.13	.085	.06	.12	.643	.38	.13	.005
		Team type (0 = delivery team)	−.74	.69	.283	−1.14	.78	.145	.60	.71	.395	−.34	.77	.659
		Transformation (0 = beginning)	.27	.62	.661	.21	.70	.764	−.29	.63	.643	.66	.69	.337
		Agile taskwork * type	.32	.16	.054	.02	.18	.897	.05	.17	.775	.15	.18	.406
		Agile taskwork * transformation	−.09	.14	.518	.09	.16	.593	.15	.14	.316	−.02	.16	.879
		Agile teamwork * type	−.13	.20	.529	.26	.23	.252	−.13	.21	.534	−.04	.22	.862
		Agile teamwork * transformation	.01	.17	.942	−.14	.19	.468	−.10	.17	.550	−.11	.19	.547

Note. Within-team relationships are denoted by L1, between-team relationships are denoted by L2 coefficients. The L2 predictors were aggregated (b), the L1 variables were centred to the group-mean (w) in Model A or included as uncentered scores (raw) in Model B. Following Bliese et al. (2018), the L2 coefficients in Model B represent a formal test of whether regression slopes significantly differ across levels of analysis.

Tuckman, B. (1965). Developmental sequence in small groups. Psychological Bulletin, 63(6), 384–399. https://doi.org/10.1037/h0022100

 PubMed Web of Science ®Google Scholar

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. https://doi.org/10.1080/10705510701301834

 Web of Science ®Google Scholar

Bliese, P. D., Maltarich, M. A., & Hendricks, J. L. (2018). Back to basics with mixed-effects models: Nine take-away points. Journal of Business and Psychology, 33(1), 1–23. https://doi.org/10.1007/s10869-017-9491-z

 Web of Science ®Google Scholar