Agile work practices: measurement and mechanisms

ABSTRACT

Organizations increasingly follow agile management frameworks (e.g., Scrum), to implement practices that aim to enable continuous change. Currently, it is unclear how agile work practices (AWPs) are best conceptualized and measured. The present study draws from the taskwork-teamwork distinction to develop a new theoretical framework and measurement instrument of AWPs. We outline potential mechanisms of AWPs in terms of (a) temporality, (b) managerial control, (c) team processes, and (d) work design. Based on this framework, we validate measures of agile practices with data collected from 269 different teams, including multisource and multiwave data (n = 1664 observations). We first establish the factorial validity, internal consistency, test-retest reliability, and measurement invariance of the instrument. Subsequently, we show that AWPs diverge from centralized bureaucracy and converge with measures of emergent team planning, autonomy, and feedback. The pattern of relationships with variables in the nomological network supports the taskwork-teamwork model. Results of multilevel regression analyses indicate that the use of AWPs is associated with favorable team planning behaviors and enriched work design experiences. By disentangling the AWP concept from software development and popular management frameworks, this study broadens the scope of research on agility. 

KEYWORDS:

• Agile work practices
• scale development
• team processes
• work design
• bureaucracy
• agility
• agile practices
• agile teams

In an increasingly complex world, organizations need structures to facilitate continuous change (Brown & Eisenhardt, 1997; Gilson et al., 2005; Grass et al., 2020; Ohly et al., 2006). Agile management frameworks promise to create such structures by establishing a range of practices that have their origins in software development (Beck et al., 2001), but are now increasingly used in other work areas (Annosi et al., 2020). Consider, for instance, a team that is tasked with the development of a new marketing campaign. Based on the logic of agile management (Sutherland, 2014), this team may split up the advertisement development process in sprints or iterative work cycles of about four weeks (Van Oorschot et al., 2018). By the end of each sprint, the team aims to have developed and tested a new feature of the advertisement through low cost experiments (e.g., A/B testing; Ghosh, 2021). Throughout the agile development process, the team constantly adapts the campaign to changing requirements and reflects on whether their approach still meets the needs of end-users (Grass et al., 2020; Liu et al., 2019).

This example indicates that agile management tries to enhance the speed of decision-making, foster team reflexivity, and enable co-creation with the end-users of products (Koch & Schermuly, 2021). Scholars and practitioners agree that these are important characteristics for work in the 21^st century (Petermann & Zacher, 2021) – not only in software development contexts – but more broadly, as work becomes more digitalized, knowledge-intensive, and socially interdependent (Cooper, 2021; Hewett & Shantz, 2021; Mergel et al., 2021; Schippers et al., 2015; Vough et al., 2017). However, there currently exists some confusion about the concept of agile working. In executive surveys, managers have associated the term “agile” with chaos and a loss of managerial control (Girod & Králik, 2021). In contrast, employees have reported about heightened peer control and rule pressure due to agile working (Khanagha et al., 2022; Stray et al., 2020). Finally, researchers in our field likely relate the term “agile” to behavioural constructs such as flexibility (He et al., 2014), adaptivity (Maynard et al., 2015), or proactivity (Potočnik & Anderson, 2016). In other words, current perspectives on agile working are fragmented, disconnected, and perhaps even paradoxical (Prange, 2021).

One of the reasons for the state of research is the lack of theory on the concept of agile working (Niederman et al., 2018). Moreover, organizational psychologists have raised concerns about existing measures of agile work practices and called for “a psychometric validation study to develop a valid measurement concept that can be used consistently across different studies” (Rietze & Zacher, 2022, p. 19). To develop valid measures, we need to be more specific about the nature of the phenomenon. Agile practices are usually implemented in team-based organizations and represent a set of routines that are collectively practiced by team members (Junker et al., 2021; Grass et al., 2020). Building on this observation, we examine agile work practices from perspectives of the organizational routine (Feldman & Pentland, 2003), work teams (Mathieu et al., 2019), and work design literature (Parker, 2014). Rather than investigating the abundance of practices that have been proposed by management consultants, we focus on practices that (a) have been implemented in a broad range of organizations (Tripp & Armstrong, 2018), (b) enable continuous change according to theory (Brown & Eisenhardt, 1997), and (c) can be examined through the lens of the established taskwork-teamwork continuum (Crawford & LePine, 2013; Fisher, 2014; Marks et al., 2001).

Agile taskwork is characterized by short-term goal cycles (i.e., sprints) and an iterative approach to task accomplishment (Ghosh & Wu, 2021; Liu et al., 2019). Iterative development has been argued to be the core indicator of agility (So, 2010), and refers to the practice of developing work outputs in repeated cycles through continuous improvement and testing of different elements or prototypes of the product. Agile teamwork is characterized by frequent check-in meetings to enable goal monitoring (also known as “stand-up meetings”) and extensive after-action-reviews to foster team reflexivity (also known as “retrospective meetings”; Hummel et al., 2015; Stray et al., 2020). These goal monitoring and reflection meetings facilitate agile taskwork (Ghosh & Wu, 2021). Thus, agile taskwork and agile teamwork are complementary, yet conceptually distinct manifestations of agile working. Owing to their broader applicability (Tripp & Armstrong, 2018), we refer to these activities as Agile Work Practices (AWPs), and distinguish them from domain-specific agile software development practices (Venkatesh et al., 2020).

Currently, agile working is mostly associated with software development, and the scientific literature on this topic is dominated by researchers in the field of information systems (Dingsøyr et al., 2012; Hoda et al., 2017). The here-validated Agile Work Practices Instrument (AWPI) may inspire management scholars and organizational psychologists to conduct research on agile working. Until now, management scholars tend to refrain from developing theory on agile working itself and mostly utilize agile team contexts for investigating related phenomena, such as prioritizing (Kremser & Blagoev, 2021), group pressure (Khanagha et al., 2022), and peer reactions to proactive behaviour (Twemlow et al. 2022). We hope the AWPI provides a basis for more specific research on the concept of agile working.

The AWPI can also be used in practice, to monitor the progress of agile transformation programmes (Girod & Králik, 2021), and as a feedback instrument for team development (Mathieu & Rapp, 2009). By this date, a whole industry emerged around agile management certifications (PMI & AgileAlliance, 2017), and a quick search shows that only in the United States more than 30-thousand open vacancies for agile coaches or agile product owners are visible on popular job platforms (e.g., see indeed.com, May 2022). The present study aims to give organizational psychologists a voice in the conversation on agile working and may encourage them to contribute to this conversation with their knowledge of team planning and work design (Parker et al., 2019). Thus, by validating a new measurement instrument and by proposing a framework that may explain the underlying mechanisms of AWPs, this study also contributes to improving the practice of agile working in organizations.

Theoretical background

Origins of agile working

The use of the term “agile” in a work context dates back to the agile manifesto, which is a document that lists a set of values and principles formulated by a group of software developers at the beginning of the new millennium (Beck et al., 2001). The agile manifesto was written as an antidote to the prevailing Waterfall paradigm of software development. Waterfall development emphasized the adherence to detailed and long-term strategic plans, which did not fit well with the dynamics of the volatile software development industry (Maruping et al., 2009). The proponents of the agile manifesto favoured a lighter planning approach and called for organizing work around self-managing teams that frequently reflect on whether plans still fit with external demands (e.g., customer preferences). The agile manifesto further contributed to the emergence of commercial project management frameworks and certifications such as Scrum (Sutherland, 2014) and Kanban (Anderson, 2010).

These management frameworks, which have also been labelled as “agile methods” (So, 2010), represent a holistic set of practices that aim to enable teams to work in the spirit of the agile manifesto. Thus, the agile manifesto is seen as the underlying philosophy behind agile methods and practices. In addition, we define agile working as the enactment of this philosophy in terms of actual work behaviour (see also Junker et al., 2021). Similar distinctions between management philosophies, practices, and behaviour have been made in scholarly writings on total quality management (TQM) in the 1990s (Dean & Bowen, 1994) and more recently in reviews on lean development (Balzer et al., 2019; Niepce & Molleman, 1998). In contrast to TQM and lean, which are traditionally associated with the manufacturing industry, agile practices are traditionally associated with new product development (NPD) and the software development sector (Grass et al., 2020; Khanagha et al., 2022; Rietze & Zacher, 2022). Next, we argue that AWPs are broader in scope than originally thought.

Conceptualizing agile work practices (AWPs) for organizational research

Here we disentangle agile practices from commercial project management frameworks and propose that these practices can also be used implicitly, without the adherence to the rules of Scrum (Schwaber & Sutherland, 2017) or Kanban (Anderson, 2010). Moreover, we focus on practices that can be implemented beyond the NPD and software domains, which we refer to as Agile Work Practices (AWPs; Junker et al., 2021). We define AWPs as a cluster of team planning routines, which aims to facilitate change-oriented behaviour. By defining AWPs as routines, we imply that these practices are “a repetitive, recognizable pattern of interdependent actions, involving multiple actors” (Feldman & Pentland, 2003, p. 96). Here, the routine actors are team members who repeatedly perform deliberate planning activities to structure taskwork and teamwork in an agile way (i.e., facilitating change-oriented action; cf., Doeze Jager-van Vliet et al., 2022; Doz, 2020; Grass et al., 2020; Petermann & Zacher, 2021).

By defining AWPs as a routine cluster, we emphasize that there exist interrelations and complementarities between the specific agile practices (see, Kremser & Schreyögg, 2016). For instance, iterative development often implies shorter goal cycles and the need for more frequent agile meetings to enable goal monitoring and reflection (Ghosh & Wu, 2021). Teams implement these routines by (a) following prescriptions of agile frameworks (i.e., importation), (b) tailoring prescribed routines (i.e., evolution), or by (c) inventing routines that follow a similar organizing principle (i.e., creation; cf., Deken et al., 2016; Gersick & Hackman, 1990). Instead of determining the what of routines (i.e., ostensive aspects), AWPs shape the how of routines (i.e., performative aspects; Feldman & Pentland, 2003). Hence, the AWP concept is not tied to a specific routine content such as the hiring, software coding, or academic writing routine. Instead, we argue that each of these activities can be planned in an agile way.

By conceptualizing AWPs as a cluster of team planning routines, we can account for the aforementioned paradoxical perspectives on agility (Girod & Králik, 2021; Khanagha et al., 2022; Lewis et al., 2014; Prange, 2021). As a cluster of routines, AWPs offer teams control and stability (Gersick & Hackman, 1990), yet – at the same time – the structure provided by AWPs may facilitate change and flexibility (Brown & Eisenhardt, 1997; Deken et al., 2016; Dönmez & Grote, 2018; Grote et al., 2018). Therefore, conceptualizing AWPs as routines offers a broad theoretical basis for the concept that allows for the integration of macro (e.g., Monteiro & Adler, 2021), meso (e.g., Mathieu et al., 2019), and micro-level streams of organizational research (e.g., Potočnik & Anderson, 2016). Next, we introduce various mechanisms that may explain how AWPs facilitate continuous change (see, Table 1).

Table 1. A conceptual analysis of agile work practices.

	Potential underlying mechanisms
Agile work practices	Temporal linkages	Managerial control	Team processes	Work design
Sprints	Entrainment	Concertive control instead of centralized bureaucracy	Emergent taskwork planning	Vertical loading (e.g., autonomy)
Iterative development	Probing the future
Stand-up meetings	Managing the present		Emergent teamwork planning	Open feedback channels
Retrospective meetings	Learning from the past

How AWPs facilitate continuous change

In the following, we relate the AWP concept to the seminal work of Brown and Eisenhardt (1997), who investigated how routines may enable continuous change. Through case comparisons of six organizations in the computer industry, they developed the concept of links in time, defined as “explicit organizational practices that address past, present, and future time horizons and the transitions between them” (Brown & Eisenhardt, 1997, p. 29). Here, we introduce four AWPs that may facilitate continuous change by creating distinct links in time.

Sprints. Agile teams plan their tasks in relatively short work cycles or intervals, also known as sprints (Van Oorschot et al., 2018). The importance of short-term goal setting for enabling continuous change was already noticed by Brown and Eisenhardt (1997). They concluded that successful firms relied on shorter planning cycles to “become entrained to the rhythm of the environment” (p. 25). Entrainment is a biological concept (Ancona et al., 2001) that describes how systems tend to synchronize over time (e.g., bodily changes across the day-night cycle). By shortening their goal cycles, agile teams become entrained to their volatile environment. Agile work environments are characterized by frequently changing customer requests and sudden resource constraints, which require dynamic prioritizing (Kremser & Blagoev, 2021). Sprints may also promote entrainment within the team, because individuals are better able to coordinate their behaviours when they work towards specific performance goals (Nahrgang et al., 2013). Thus, sprints fulfill an important coordinative function for timing work activities, which may facilitate continuous change.

Iterative development. Agile teams tend to approach their goals in a way that is commonly referred to as iterative development (Ghosh & Wu, 2021; Tripp et al., 2016). For instance, if we had to develop a new mobile application in an agile way, we first would come up with a prototype or minimum viable product (Lee & Geum, 2021). Subsequently, we would engage in user-testing to obtain feedback on this preliminary product. Then, we would continue to release new features of the product and try to obtain additional feedback by means of other low-cost experiments (e.g., A/B testing; Ghosh, 2021). This iterative approach allows for incorporating changes in user-needs throughout the whole development process. Thus, iterative development may create the ability to continuously “probe into the future with a variety of low-cost experiments” (Brown & Eisenhardt, 1997, p. 25). More generally, iterative development builds on the logic of effectuation because this practice creates conditions that allow for affordable loss (e.g., failed prototypes of products; Sarasvathy, 2001). This approach is not only applicable in software development. Iterative development can even be practiced in our very own work as scholars,¹ as described in the book “The lean PhD” (Kirchherr, 2019).

Stand-up meetings. To coordinate team members’ activities, agile frameworks (e.g., Scrum) recommend conducting short daily check-in meetings also known as “standups”. Team interactions during stand-up meetings typically address members’ activities for the day and potential impediments that hinder goal progress (Hummel et al., 2015). Thus, stand-up meetings can be seen as a goal monitoring practice, because these meetings direct members’ attention towards what needs to be done for goal attainment (Rapp et al., 2014). In terms of temporal linkages, stand-up meetings enable teams to manage the present because these meetings potentially create a combination of “clear responsibilities and priorities coupled with extensive communication” (Brown & Eisenhardt, 1997, p. 15). Although there is a debate about how daily standups should be conducted (e.g., what should be discussed, see, Stray et al., 2020), merely having this formal structure may already help teams to coordinate effectively and to integrate knowledge in a flexible manner (Okhuysen & Eisenhardt, 2002).

Retrospective meetings. According to the agile manifesto, teams should reflect on their activities and plans very regularly (Beck et al., 2001). This is commonly practiced in formal retrospective meetings, which take place at the end of a sprint period (i.e., approximately every four weeks; Schwaber & Sutherland, 2017). During these meetings, team members collectively learn from the past by reflecting on their activities. Thus, retrospective meetings may be seen as a type of after-action-reviews, defined as a “systematic technique that turns a recent event into a learning opportunity through a combination of task feedback, reflection, and discussion” (Keiser & Arthur, 2021, p. 1108). This potentially impacts team reflexivity or the capacity of a team to jointly question its own functioning (Schippers et al., 2015). In addition, retrospective meetings can help to establish a psychologically safe work environment (Hennel & Rosenkranz, 2021), and thereby may contribute to team learning (Edmondson, 1999). Finally, as a transition moment between each sprint period, retrospective meetings may create temporal linkages by nudging teams to be “rhythmically moving from the past to the present and forward into the future” (Brown & Eisenhardt, 1997, p. 30).

Commonalities and interrelations. Building on the work of Brown and Eisenhardt (1997), we proposed that the four AWPs function via distinct temporal mechanisms that enable entrainment (i.e., sprints) and create links with the future (i.e., iterative development), the present (i.e., stand-up meetings), and the past (i.e., retrospective meetings). However, these practices also share certain commonalities that we can base on the well-known taskwork-teamwork continuum (Crawford & LePine, 2013; Fisher, 2014; Marks et al., 2001). Similar distinctions have been utilized in conceptualizations of leadership (i.e., task vs. relationship-oriented; Yukl et al., 2002), diversity (i.e., functional vs. social; Gerpott & Lehmann-Willenbrock, 2016), and conflict (i.e., task vs. interpersonal; Shah et al., 2021). Moreover, these dimensions capture the two primary needs of workgroups, namely (1) goal accomplishment (i.e., taskwork), and (2) addressing members’ socio-emotional needs (McGrath et al., 2000; Tuckman, 1965). Validating the second-order factor structure will facilitate an integration of the AWP concept with the broader literature (Abele & Wojciszke, 2014), and with theories of team processes that are based on the taskwork-teamwork dichotomy (Marks et al., 2001).

In the team literature, taskwork generally refers to how teams approach their goals and structure work assignments (Crawford & LePine, 2013; Fisher, 2014). Agile taskwork, in particular, is structured around short goal cycles that allow for developing work outputs in an iterative manner. Hence, sprints and iterative development are AWPs that indicate how agile teams address their need for goal accomplishment (McGrath et al., 2000). Therefore, we predict that these AWPs load on a higher-order taskwork factor. Teamwork refers to how interactions, roles, and responsibilities are organized in groups (Crawford & LePine, 2013; Fisher, 2014). Agile teams rely on frequent goal monitoring and reflection meetings to facilitate group functioning (Ghosh & Wu, 2021). Hence, stand-up and retrospective meetings are AWPs that indicate how agile teams communicate and address members’ socio-emotional needs (Bales, 1950; Tuckman, 1965). Thus, we expect that stand-up and retrospective meetings load on a higher-order teamwork factor. Other scholars have referred to agile teamwork as “social agile practices” (Gupta et al., 2019; Hummel et al., 2015), supporting this classification.

Agile taskwork and agile teamwork are not completely independent constructs. Some empirical overlap is expected because the frequency of agile meetings depends on the length of sprints (Gersick, 1988), or vice versa because these work cycles are structured around meetings (cf., Okhuysen & Waller, 2002). Similarly, agile teamwork may provide input for members’ iterative development activities and these activities are discussed within the context of agile meetings (Twemlow et al., 2022). If we consider teams as complex adaptive systems (CAS; McGrath et al., 2000), interrelations among the four AWPs may be explained by how these practices “interact” with each other to create a social system that adapts to its environment (Ramos-Villagrasa et al., 2018). As customers demand more rapid product development, the duration of the sprint cycles may be shortened, teams rely more on short stand-up meetings instead of extensive retrospective meetings, and iterative development may focus on the quick exploitation of existing prototypes rather than the exploration of novel solutions (cf., Ghosh & Wu, 2021). Thus, relationships among the four AWPs not only occur because they address similar internal needs of the team, but also because of how the team engages with the external environment (e.g., customers, technology, other teams, and so forth).

Although the taskwork-teamwork distinction is a simplification of organizational reality, this dichotomy may help to develop a more coherent understanding of the mechanisms of AWPs. Specifically, we later hypothesize that agile taskwork and teamwork influence distinct team planning and work design mechanisms. Moreover, it should be noted that the taskwork-teamwork distinction is not at conflict with other possible conceptualizations of AWPs, such as more nuanced team process theories (Ishak & Ballard, 2012). Instead, our hypothesized higher-order factor structure provides a tradeoff in terms of parsimony and complexity, and narrow team processes are distinguishable on a taskwork-teamwork continuum (see, Marks et al., 2001, p. 357). For example, from a CAS perspective (Arrow et al., 2000), we may expect that over time certain AWPs cluster because they help teams adjust to environmental demands that require similar taskwork and teamwork processes. This reasoning is consistent with our conceptualization of AWPs as clusters of planning routines that can be distinguished on a taskwork-teamwork continuum (Fisher, 2014; Kremser & Schreyögg, 2016). Therefore, we predict the following factor structure:

Hypothesis 1: The Agile Work Practices Instrument (AWPI) will show a four-factor structure, representing items measuring (1) sprints, (2) iterative development, (3) stand-up meetings, and (4) retrospective meetings.

Hypothesis 2: Agile work practices have two higher-order dimensions, namely: (a) agile taskwork (i.e., sprints and iterative development), and (b) agile teamwork (i.e., stand-up meetings and retrospective meetings).

AWPs imply less centralized bureaucratic control

Agile work practices have been advocated as a new form of planning that potentially reduces organizational bureaucracy (Girod & Králik, 2021; Rigby et al., 2020). Given the breadth of the bureaucracy construct (Monteiro & Adler, 2021), it is necessary to be more specific about which type of bureaucracy is reduced by the implementation of AWPs. When scholars (Mergel et al., 2021) and practitioners (Rigby et al., 2020) refer to agility as the antidote of bureaucracy, they usually mean centralized bureaucracies characterized by high degrees of workflow formalization and centralized decision-making (Adler & Borys, 1996; Langer et al., 2019). Instead of relying on formal hierarchies, agile organizations strive for rapid decision-making within teams (Petermann & Zacher, 2021). This does not imply chaos, because the use of AWPs formalizes parts of the team’s work processes (i.e., by structuring taskwork and teamwork). Moreover, agile teams replace hierarchy with concertive control, which is a form of peer pressure through which teams ensure that each member complies with social norms (Barker, 1993). This shift in managerial control happens because AWPs replace centralized decision-making with locally empowered decision-making by team members (Grass et al., 2020). Given that AWPs partly formalize work processes and contribute to the maintenance of managerial control, we do not see the construct as the polar opposite of bureaucracy – but on a different bureaucratic continuum (Monteiro & Adler, 2021). The bureaucracy construct is relevant to the context of AWPs, yet different in nature because AWPs promote concertive control instead of centralized bureaucracy (see, Khanagha et al., 2022). In sum, we propose the following discriminant validity hypothesis (Campbell & Fiske, 1959):

Hypothesis 3: Agile work practices correlate weakly and negatively with centralized bureaucratic control.

AWPs shape emergent team planning behaviour

Earlier we distinguished agile taskwork (i.e., sprints and iterative development) from agile teamwork (i.e., stand-up and retrospective meetings). As relatively deliberate planning routines (Feldman & Pentland, 2003), agile taskwork and agile teamwork may give rise to more dynamic or fluctuating team processes (Marks et al., 2001). Particularly, we expect that AWPs relate to emergent team planning behaviour (Fisher, 2014), which refers to relatively spontaneous planning activities that happen in the moment as a result of “a naturally occurring team process” (p. 425). Thus, emergent team planning can be distinguished from AWPs, which may primarily manifest “as part of a predesignated or structured team activity” (Fisher, 2014, p. 425). When asked to describe how their team organizes itself, employees commonly refer to certain practices (e.g., AWPs) rather than emergent team planning behaviours. This is because as a type of routines, AWPs are consciously more available to team members than fluctuating behavioural processes (for similar observations, see, Zellmer-Bruhn, 2003, p. 520). As defined and measured by Fisher (2014), emergent team planning focuses on the what, whereas AWPs focus more on the how of planning. Emergent taskwork planning is directed at goals and task-related requirements. Emergent teamwork planning is directed at information-sharing, roles, and relationships. Since the what and the how of team planning are likely interrelated (Mathieu & Rapp, 2009; Mathieu & Schulze, 2006), we expect that AWPs correlate with emergent planning. Thus, building on Hypothesis 2 and the conceptualization of Fisher (2014), we forward hypotheses on the convergent validity of our new instrument (DeVellis, 2016):

Hypothesis 4: Agile taskwork is uniquely and positively associated with emergent taskwork planning (i.e., planning directed at goal and task-requirements).

Hypothesis 5: Agile teamwork is uniquely and positively associated with emergent teamwork planning (i.e., planning directed at interaction and role-requirements).

AWPs have implications for work design

Prior research introduced the AWP concept from a work design perspective (Rietze & Zacher, 2022; Tripp et al., 2016). Broadly defined, work design refers to “the content and organization of one’s work tasks, activities, relationships, and responsibilities” (Parker, 2014, p. 662). Although we are reluctant to equate AWPs with job design as done by Tripp et al. (2016), we recognize that the objective use of these practices has consequences for how employees perceive the design of their job. Agile taskwork may influence work design through vertical loading, that is granting employees more autonomy for self-regulated action (Slocum & Sims, 1980). Indeed, there is evidence that agile taskwork practices, such as iterative development, correlate positively with perceptions of job autonomy (Rietze & Zacher, 2022; Tripp et al., 2016). Agile teamwork may shape work design by opening feedback channels, that is broadening employees’ knowledge of their own functioning and performance (Slocum & Sims, 1980). In line with this assertion, perceptions of agile teamwork have been related to social job resources, such as feedback (Tripp et al., 2016) and peer support (Rietze & Zacher, 2022). A limitation of these prior studies on agile work design is their use of individual-level data for inferring relationships between constructs that are theorized at the team- and job-level, respectively (for why this matters, see, Bakker & Demerouti, 2017; Bliese et al., 2018). Therefore, to increase theory-method fit (Edmondson & McManus, 2007), we formulate and test hypotheses on team-level work design mechanisms of AWPs. Specifically, we predict the following. The more a team practices agile taskwork, the more autonomy is collectively experienced by team members. The more a team practices agile teamwork, the more feedback is collectively experienced by team members. Thus, building on Hypothesis 2 and prior studies on agile work design (Rietze & Zacher, 2022; Tripp et al., 2016), we propose:

Hypothesis 6: Agile taskwork is uniquely and positively associated with team autonomy.

Hypothesis 7: Agile teamwork is uniquely and positively associated with team feedback.

Exploring the contexts in which AWPs are implemented

Over the past ten years, organizations have been trying to implement AWPs in large-scale transformation processes (Rigby et al., 2020). Such transformations have been observed in a wide range of industries, including banking (Barton et al., 2018), retail (Bernstein et al., 2016), and the technology sector (Laanti et al., 2011). Some organizations have used a “big bang approach” and implemented AWPs throughout all departments almost overnight (Denning, 2021), whereas other firms used a more incremental approach and slowly implemented AWPs across different work areas (Girod & Králik, 2021). Thus, the question arises whether (a) the use and (b) the effects of AWPs differ across the phases or stages of an agile transformation process. Similarly, it is still unclear to what extent AWPs can be implemented in different work areas since most agile practices originated in software development (Annosi et al., 2020). With our new instrument, differences in the extent to which AWPs are practiced can be tested by comparing mean scores. Moreover, we can explore whether the potential effects of AWPs hold across contexts, by examining the direction and strength of relationships in the nomological network across different types of teams. We address these empirical issues with the following research questions:

RQ1: How do the mean levels of AWPs vary across (a) different types of teams, and (b) different phases or stages of an agile transformation?

RQ2: How do the relationships between AWPs and variables in the nomological network vary across (a) different types of teams and (b) different phases or stages of an agile transformation?

Methods

Overview of the data sources

The data used in this study were collected as part of a larger research programme on agile working, based on three independent data collection efforts. Table 2 provides an overview of the main demographic characteristics of the different samples used in the present study. Sample A is a heterogenous convenience sample, based on data collected in early 2019 via the authors’ social networks (e.g., LinkedIn and personal contacts). Sample B, C, and D are based on data collected from teams at the IT division of a large German transport and logistics organization. This organization, shortly called “AgileOrg” throughout this manuscript has initiated an agile transformation process in 2017. The transformation happens in all work areas of AgileOrg, including their “delivery teams” (i.e., IT-services, -maintenance, and -consulting) and their “support teams” (i.e., HR, customer relations, finance). Before the transformation, employees worked together in a departmental work structure. This organizational design was successively replaced by a multiteam system composed of self-managing agile teams. The transformation entailed that large departments were split up into teams, following a programme that closely resembled well-known models of group development (i.e., forming-storming-norming-performing). Employees at AgileOrg primarily belong to only one team, which was registered in the organization’s HR system together with a shared team email account, nominal team size, team type, as well as the team’s progress in the agile transformation program. Thus, the context at AgileOrg allowed for an investigation of AWPs in different types of teams (i.e., delivery vs. support) and different phases of their agile transformation (i.e., beginning vs. end).

Table 2. Overview of sample and participant characteristics.

Sample	Teams	Employees	Teamsize	Team response	Demographics	Work content	Team experience	Agile transformation
A. Convenience	Not applicable	163	Not applicable	Not applicable	Mostly women (66%), who were between 35 and 55 years old (50%), and held a Bachelor degree or higher (78%). Most participants were German native speakers, while 36 completed the survey in English.	Various sectors, primarily logistics (16%), IT (14%), construction (10%).	71% of the participants had been a member of their team for more than 6 months.	39% of the participants described their team as an “agile” or “rather agile” team.
B. AgileOrg 2019 teams	114	476	M = 9.51, SD = 2.41	Min. = 2, Max. = 9, Median = 40% per team.	Mostly men (78%), who were between 35 and 55 years old (55%), and held a Bachelor degree or higher (64%).	87 delivery teams (n = 365); 27 support teams (n = 111).	92% of the participants had been a member of their team for more than 6 months.	42% of the teams were in an advanced phase of agile transformation.
C. AgileOrg 2019–2020 longitudinal	71	302	M = 12.20 SD = 2.10	Min. = 2, Max. = 9, Median = 40% per team	Mostly men (71%), who were between 35 and 55 years old (51%), and held a Bachelor degree or higher (66%).	63 delivery teams (n = 260); 8 support teams (n = 42).	96% of the participants had been a member of their team for more than 6 months.	75% of the teams were in an advanced phase of agile transformation.
D. AgileOrg 2020 new teams	155	657	M = 11.54 SD = 2.51	Min. = 2, Max. = 11, Median = 43% per team	Mostly men (65%), who were between 35 and 55 years old (46%), and held a Bachelor degree or higher (65%).	126 delivery teams (n = 521); 29 support teams (n = 136).	88% of the participants had been a member of their team for more than 6 months.	48% of the teams were in an advanced phase of agile transformation.

Note. The transformation at AgileOrg is split up in four phases resembling the forming-storming-norming-performing model by (Tuckman, 1965) and the teams’ progress in the transformation is registered in the company’s HR system. We recoded the ordinal agile transformation variable into a binary dummy variable representing the beginning (forming-storming) and advanced (norming-performing) phases of the transformation at AgileOrg. This allowed for more easily interpretable known-group comparisons.

The data of Sample B were collected in the summer of 2019 and have been partly used in a previous study² (Junker et al., 2021). One year later (i.e., summer 2020), we started another round of data collection at AgileOrg, which allowed us to match teams with data from 2019 (i.e., Sample C), and to contact an independent sample of teams from which no data had been collected previously (i.e., Sample D). Therefore, Sample C is nested within Sample B (i.e., the longitudinal follow-up), while the cases of Sample D are unique. We only retained the data of teams for which we had at least two member responses, as done in prior validation studies of team survey measures (e.g., Fortuin et al., 2021; Mathieu et al., 2020). For parts of Sample D, the present study also includes ratings of the team’s agile coaches (sometimes also called Scrum Masters; e.g., Grass et al., 2020). The agile coaches support the teams in HR-related matters and may therefore provide a valid external perspective on AWPs. Hence, the present study includes multiwave (Sample C) and multisource data (Sample D). It should be noted that at the time of the data collection (2019–2020), our studies were exempt from ethical review at our institute due to their correlational nature. In the following, we outline how the different data sources were used in the present study.

Overview of the validation approach

Following best-practice recommendations (e.g., Cortina et al., 2020; DeVellis, 2016; Johnson et al., 2012), we validate the AWPI in three phases. In Phase 1, we establish the factor structure, internal consistency (Sample A and B), and test-retest reliability of the AWPI (Sample C). In Phase 2, we use the unique AgileOrg cases (Sample B and D) to test the second-order factor structure and measurement invariance of the instrument (Hypothesis 1 and 2). We also explore differences in the mean levels of AWPs across team types and phases of the agile transformation (RQ1). In Phase 3, using the data of Sample D, we test the construct validity of the AWPI according to the proposed nomological network (Hypothesis 3 to 7). We also explore whether relationships in the nomological network vary across contexts (RQ2). Analysis codes and materials are available on the Open Science Framework (https://osf.io/ndtx4).

Results of phase 1: factorial validity and reliability

Rationale and analytic approach

The first phase of any scale validation study is the development of a broad pool of items that can potentially capture variance in the underlying construct (Hinkin, 1995). Before constructing the preliminary AWPI item pool, we conducted interviews with five practitioners in Germany who had substantial experience with agile methodologies (two members of an agile team, two HR experts, and one agile coach). In the interviews, it was inquired how agile practices are enacted in different team contexts (e.g., software development vs. other contexts). The insights were used to adapt items of existing scales (So, 2010; Tripp et al., 2016), which are relatively specific to the software development context and have not been extensively validated. Based on the interviews, we created additional items to capture the full breadth of the AWP construct. This resulted in an item pool of 41 items, which was subsequently reviewed by the interviewees and two experts in organizational psychology who have substantial experience in scale development (one associate and one full professor). Based on their feedback, eight items were discarded because they did not match the proposed definitions of the four AWPs. In the following, we report the analyses that we conducted to refine the instrument and arrive at the final item pool (see, Table 3).

Table 3. Items and factor loadings of the APWI in Sample A.

Item and factor loading		1	2	3	4
ID1	We develop a prototype/pilot before expanding our plans.				.55
ID2	We experiment with different ideas before settling on an approach.				.62
ID3	We refine our initial ideas successively.				.63
ID4	We quickly adapt our approach to changing requirements.				.56
ID5	We regularly ask customers/clients for feedback on the progress of our work.				.63
ID6	We frequently check whether the demands of our customers/clients have changed.				.76
ID7	When we start working on an assignment, we often do not know what the final product will look like.				.59
SI1	We plan our work activities in short sequences.			.64
SI2	We limit the duration of our work cycles to less than 1 month.			.73
SI3	We try to reduce uncertainties by keeping work cycles short.			.83
SI4	We try to increase flexibility by keeping work cycles short.			.74
SI5	We plan our tasks in short cycles (e.g., weekly).			.64
DS1	We have a short meeting to discuss new developments in our tasks.	.87
DS2	We have a short meeting to discuss impediments that hinder us from completing our tasks.	.84
DS3	We have a short meeting to discuss the progress of our work.	.88
DS4	We have a short meeting to discuss difficulties in our tasks.	.90
DS5	We have a short meeting to inform each other about what we are working on.	.81
DS6	We have a short meeting to inform each other about relevant issues.	.84
DS7	We have a short meeting to discuss changes in our tasks.	.92
RR1	We take our time to appreciate each other for our efforts.		.72
RR2	We take our time to talk about what went well in the team.		.90
RR3	We take our time to discuss our work processes.		.79
RR4	We take our time to critically reflect on our work activities.		.79
RR5	We take our time to improve the social dynamics of the team.		.78
RR6	We take our time to discuss ways to improve team performance.		.61
RR7	We take our time to discuss feedback from our customers/clients.		.53		.24

Note. The boldfaced items were used from Phase 2 onward. Response format agile taskwork items: 1 (fully disagree), 2 (disagree), 3 (slightly disagree), 4 (indecisive), 5 (slightly agree), 6 (agree), 7 (fully agree). Response format agile teamwork items: 1 (almost never), 2 (few times a year), 3 (once a month), 4 (few times a month), 5 (once a week), 6 (few times a week), 7 (daily).

Exploratory factor analyses

Using the data of Sample A, we performed parallel analyses and exploratory factor analyses (EFA) to further reduce the item pool. Although the sample is relatively small (N = 163 employees), the Kaiser-Meyer-Olkin value of .90 indicated that our data is suitable for factor analyses due to sufficient commonalities among the items (see also, Fabrigar et al., 1999). Parallel analyses indicated that there are four latent factors present in the data, which accounted for more than half of the variance. Parallel analysis is superior to other methods for factor retention because this procedure identifies the number of eigenvalues that are larger than eigenvalues based on random simulations of the data (Hayton et al., 2004). Subsequently, we performed EFAs with oblique factor rotation to examine the loadings of the four-factor solution. Seven items had substantial cross-loadings (> .25) or loaded on a non-hypothesized factor. These items were subsequently discarded until the EFA solution was “clean” (DeVellis, 2016). The Root Mean Square Error of Approximation (RMSEA) for the final EFA solution was .064 (90% confidence interval [.053; .076]), indicating appropriate model fit (Marsh et al., 2004). The items administered in Sample A are shown together with factor loadings in Table 3 and correspond to the four hypothesized AWP factors, supporting Hypothesis 1. As indicated in Table 2, there were 36 participants (22%) in Sample A who completed the survey in English instead of German. When excluding these participants, a similar factor structure and model fit was obtained (RMSEA = .063).

Correlations with agile methods, team types, and task interdependence

As part of the survey administered in Sample A, we asked participants to indicate whether they regard their team as an agile team on a scale from (1) traditional team to (5) agile team, and which of the four team types according to the typology of Cohen and Bailey (1997) best applied to their team (i.e., service/production team, consulting/advice team, project/time-limited team, or management team). Moreover, we asked participants to report the frequency of using (a) the main agile methods, Scrum and Kanban (So, 2010), (b) the agile software development technique Extreme Programming (Hoda et al., 2017), (c) related frameworks, Design Thinking and Lean Development (Balzer et al., 2019; Liedtka, 2015), and (d) the opposing management framework Waterfall Development (Maruping et al., 2009; Venkatesh et al., 2020). The survey also included measures of task interdependence as introduced by Tims et al. (2013). Shown in Table 4, our new AWPI measures all correlated significantly with participants’ perceptions of team agility (ranging from r = .41 to r = .50), as well as the perceived use of classic agile methods (ranging from r = .21 to r = .52) and related frameworks (ranging from r = .22 to r = .44). Only iterative development (r = .24) and stand-up meetings (r = .18) were weakly associated with Waterfall Development. Iterative development was positively associated with task interdependence (r = .21) and project team membership (r = .18). Taken together, the relatonships with perceived team agility and agile methods provide some initial evidence for the construct validity of our new scales. The moderate correlations also indicate that the use of AWPs is not tied to a single agile framework, team type, or level of task interdependence. The two significant correlations with Waterfall Development imply that iterative development and stand-up meetings are to some extent also used in “non-agile” contexts – despite being known as the most prominent agile practices (Ghosh & Wu, 2021).

Table 4. Pearson product-moment correlations of variables measured in Sample A.

Variable	N	M	SD	1	2	3	4	5	6	7	8	9	10	11	12	14	15	16
1. Iterative development	163	3.50	0.76
2. Sprints	163	3.24	0.94	.51**
3. Stand-up meetings	163	3.20	1.09	.39**	.50**
4. Retrospective meetings	163	2.78	0.96	.31**	.34**	.49**
5. Agile team	150	3.16	1.22	.50**	.49**	.41**	.43**
6. Scrum	150	2.95	2.17	.34**	.52**	.45**	.37**	.62**
7. Kanban	150	3.01	2.03	.26**	.25**	.25**	.21**	.37**	.45**
8. Extreme Programming	150	1.65	1.30	.21**	.28**	.23**	.23**	.40**	.42**	.29**
9. Design Thinking	150	2.67	1.85	.44**	.22**	.23**	.23**	.39**	.26**	.36**	.38**
10. Lean Development	150	2.30	1.71	.38**	.28**	.25**	.23**	.42**	.34**	.31**	.53**	.51**
11. Waterfall Development	150	2.09	1.54	.24**	.09	.18*	.07	.26**	.26**	.13	.37**	.29**	.42**
12. Task interdependence	163	3.74	0.78	.21**	.15	.15	.04	.05	−.07	.06	−.13	.01	−.03	.10
14. Service/production team	163	0.31	0.47	−.03	.15	.10	.09	.08	.08	.04	.08	−.07	.06	−.04	−.09
15. Consulting team	163	0.28	0.45	.00	−.10	−.12	−.10	−.02	−.03	−.01	.04	.14	.05	.04	−.11	−.36**
16. Project (time-limited) team	163	0.37	0.48	.18*	.10	.09	−.01	.09	.03	−.03	.07	.06	.08	.09	.18*	−.49**	−.31**
17. Management team	163	0.12	0.33	−.08	−.12	−.04	.04	−.08	−.06	.00	−.13	−.08	−.05	−.04	.05	−.25**	−.19*	−.13

Note. Variables 14 to 17 are coded in a binary way (0 = no, 1 = yes), N = sample size, M = mean, SD = standard deviation, ** p < .01, * p < .05

Multilevel confirmatory factor analyses

Using the data of Sample B, we attempted to confirm the hypothesized factor structure in multilevel confirmatory factor analyses (ML-CFA), shown in Table 5. We followed the guidelines of Marsh et al. (2004), who propose that model fit is appropriate when the Comparative Fit Index (CFI) and the Tucker-Lewis Index (TLI) are above .90, while the Root Mean Square Error of Approximation (RMSEA) and the Standardized Root Mean Square Residual (SRMR) should be less than .08 for acceptable fit. By examining the factor loadings of the initial ML-CFA (Model 3 in Table 5) and the between-team CFA (Model 2 in Table 5), we identified items with relatively weak factor loadings at the between-team level, which can quickly worsen approximate model fit indices (e.g., the SRMR; Asparouhov & Muthén, 2018). We only retained items with large between-team factor loadings, given that the AWPI theoretically measures a shared team construct. The standardized factor loadings of the retained items at the between-team level were high (mean = .86, range = .56 to .96). The revised models are based on a smaller, yet factorially purer set of items (see bold items in Table 3). This is desirable from psychometric and practical perspectives (DeVellis, 2016). In addition, selecting items based on between-team factor loadings can enhance the predictive validity of collective constructs (Bliese et al., 2019). The multilevel CFAs support Hypothesis 1 because the four-factor structure of the AWPI holds on both levels of analysis, with a reasonable fit for the initial model (Model 3 in Table 5) and a good fit for the revised model (Model 6 in Table 5). Thereby, we established cross-level isomorphism because the variance in the items of the AWPI is partitioned in a similar manner at the between-team and the within-team level (Tay et al., 2014). Thus, individuals’ item responses provide a valid reflection of team-level agile work practices.

Table 5. Fit of confirmatory factor models in Sample B.

Model	N	df	AIC	BIC	Δχ^2	CFI	TLI	RMSEA	SRMR
1. Within-team	476	293	30,553	30,903	859.23***	.875	.861	.064	.054
2. Between-team	114	293	5784	6014	631.16***	.874	.861	.101	.091
3. Multilevel	476	586	32,374	32,952	1010.93***	.924	.916	.041	w = .061; b = .149
4. Rev. within-team	476	98	19,198	19,423	218.19***	.953	.942	.051	.047
5. Rev. between-team	114	98	3537	3685	156.69***	.963	.955	.072	.072
6. Rev. multilevel	476	196	20,760	21,137	260.14**	.981	.977	.027	w = .050; b = .079

Note. *** p < .001, ** p < .01. Rev. = revised models after retaining only items with high between-team factor loadings (from Model 4 onward).

Internal consistency, test-retest reliability, and cross-lagged relationships

As shown in Table 6, both the long version and the shorter revised version of the AWPI scales have adequate internal consistency reliability according to Cronbach’s alpha (assuming strictly parallel items) and McDonald’s omega hierarchical (assuming congeneric items). The ICC₂ values were acceptable, suggesting that sampling more responses per team is unlikely to drastically change the group means (Bliese et al., 2018). The group mean reliability was further underscored by the high levels of within-team agreement (r_wgj), which shows that the AWPI measures a shared team construct rather than idiosyncratic perceptions of individuals. For a sub-sample of teams, we were able to calculate test-retest reliability over a one-year period (i.e., Sample C). It appears that agile teamwork is temporally less stable than agile taskwork, according to test-retest correlations (see, Table 6). Moreover, agile teamwork at T1 had a cross-lagged relationship with agile taskwork at T2 (ß = .22, SE = .11, p = .021), but not vice versa (ß = .02, SE = .08, p = .857). This suggests that agile teamwork is predictive of changes in agile taskwork (changes in terms of rank-ordering; Henk & Castro-Schilo, 2016). In addition, this indicates that agile taskwork and agile teamwork are empirically distinct (Kenny, 1975).

Table 6. Internal consistency, group mean stability, median within-team agreement, and test-retest reliability.

	Reliability	α	ωh	ICC2	rwgj	rt+1year
Scale	Agile taskwork long (12 items)	.91	.83	.80	.95
	Agile taskwork (8 items)	.90	.83	.79	.94	.72
	Agile teamwork long (14 items)	.94	.87	.79	.98
	Agile teamwork (8 items)	.90	.85	.77	.96	.58
Finalsubscales	Iterative development (4 items)	.84	.87	.66	.90	.62
	Sprints (4 items)	.95	.95	.80	.88	.73
	Stand-up meetings (4 items)	.93	.94	.75	.94	.56
	Retrospective meetings (4 items)	.91	.90	.64	.93	.43

Note. The items used in the long and short versions are shown in Table 3. The test-retest reliability was calculated using the data of Sample C, all other values are based on data of Sample B.

Results of phase 2: measurement invariance and mean differences

Rationale and analytic approach

Once we know that our new instrument measures AWPs in a similar way in different sub-groups, we can be more confident about the validity of group comparisons and the applicability of the different scales. Psychometrically, this refers to measurement invariance (MI) and is usually tested in three steps (Vandenberg & Lance, 2000; Van de Schoot et al., 2012). First, we fit the “configural model”, which imposes a similar factor structure but leaves factor loadings and intercepts free to vary. Then, we fit the “metric model”, which constrains factor loadings to equivalence (i.e., relationships with latent variables are the same across groups). Finally, we fit the “scalar model”, which also constrains the intercepts to equivalence (i.e., no systematic response differences between groups are modelled). Before conducting the MI tests, we identify the higher-order factor model which best describes the relationships between the four AWP factors and the 16 selected items from Phase 1 (see bold items in Table 3). Specifically, we compare the hypothesized taskwork-teamwork factor model with (1) a model that posits only one second-order factor, and (2) a model that allows no correlations among the four AWPs, assuming they are completely independent constructs.

Subsequently, we examine the stability of the higher-order factor solution via MI tests. We use individual-level data rather than aggregated team-level data to increase statistical power and obtain a better sample-size-per-parameter ratio, which is necessary for these psychometric tests (Marsh et al., 2004; Molenaar, 2016). This choice is reasonable, since we had established cross-level isomorphism in Phase 1 via multilevel CFAs (see, Tay et al., 2014), and because we are primarily interested in the measurement properties across broad sub-groups. Specifically, we compare the measurement properties across Sample B and D, across members of different types of teams (i.e., delivery vs. support), and across different phases of the transformation at AgileOrg (i.e., beginning vs. advanced phases). Thereby, we provide a more stringent test of Hypotheses 1 and 2. After establishing measurement invariance, we conduct tests of mean differences to answer RQ1. We use the maximum likelihood estimator because cut-off-values for model fit indices are based on simulation studies using this estimator (Chen, 2007; Cheung & Rensvold, 2002; Hu & Bentler, 1999).

Model comparisons

As shown in Table 7a, the hypothesized model with agile taskwork (i.e., sprints and iterative development) and agile teamwork (i.e., stand-up and retrospective meetings) as second-order factors fit the data well. In terms of fit indices, this model provides a slightly better representation of the observed data than the more parsimonious one-factor model and the uncorrelated factor model. Thus, it appears empirically justified to make the taskwork-teamwork distinction in the AWPI. Importantly, the factor loadings and intercepts of the hypothesized second-order factor model are largely invariant across subpopulations, according to the CFI and RMSEA indices (see, Table 7b-d). Although simultaneously constraining all factor loadings (i.e., scalar model) and intercepts (i.e., metric model) to equivalence results in significant chi-square differences, these differences are not large enough to reject the null hypothesis of non-invariance according to methodological guidelines (e.g., Chen, 2007; Cheung & Rensvold, 2002). Thus, we can be reasonably confident that our instrument measured AWPs in similar ways for participants in the two samples (Table 7b), members of delivery and support teams (Table 7c), and at different phases of the agile transformation (Table 7d). Collectively, these findings support Hypotheses 1 and 2.

Table 7. Higher-order factor models and measurement invariance tests in Sample B and D.

Model and sample			χ^2	df	CFI	RMSEA	ΔCFI	ΔRMSEA	Δχ^2
A. Higher-order models	Sample B and D combined
Hypothesized second-order	n = 1133		567.36***	99	.954	.065
One second-order			583.64***	100	.952	.065	−.002	.000	16.28***
Uncorrelated factors			1206.78***	104	.891	.097	−.061	.032	623.14***
B. Samples	Sample B (2019)	Sample D (2020)
Configural	n = 476	n = 657	747.09***	198	.946	.070
Metric			772.91***	212	.945	.068	−.001	−.002	25.82*
Scalar			837.45***	222	.940	.070	−.005	.002	64.54***
C. Team types	Delivery teams	Support teams
Configural	n = 886	n = 247	663.96***	198	.953	.064
Metric			682.54***	212	.953	.063	.000	−.001	18.59
Scalar			733.82***	222	.949	.064	−.004	.001	51.27***
D. Agile transformation	Beginning	Advanced
Configural	n = 580	n = 553	686.69***	198	.951	.066
Metric			704.52***	212	.951	.064	.000	−.002	17.83
Scalar			728.12***	222	.950	.063	−.001	−.001	23.60**

Note. According to Chen (2007), ΔCFI ≤ −.01 supplemented by ΔRMSEA ≥ .015 indicate non-invariance, *** p < .001, ** p < .01, * p < .05

Mean differences

To address RQ1, we tested for differences in the latent means of agile taskwork and agile teamwork between (a) team types and (b) phases of the agile transformation. We released the intercept constraints on the second-order factors, while keeping the constraints on the first-order factor and the items to estimate the latent mean differences in terms of taskwork and teamwork (see Chen et al., 2005). Members of delivery (vs. support) teams scored higher on the latent agile taskwork factor (d = .49, p < .001) and the latent agile teamwork factor (d = .33, p = .002). Similarly, both latent agile taskwork (d = .31, p < .001) and agile teamwork (d = .21, p = .026) were significantly higher at the advanced (vs. beginning) phases of the transformation at AgileOrg. We also conducted Welch’s t-tests on the observed mean differences for the four subscales. As shown in Figure 1, the observed differences mirror the latent mean differences. Noteworthy is that the observed mean differences between team types were not significant for iterative development, as were the observed differences between the agile transformation phases for stand-up meetings and retrospective meetings. Taken together, our answer to RQ1 is (a) members of delivery (vs. support) teams score higher on agile taskwork (particularly, sprints) and agile teamwork, and (b) scores at the advanced (vs. beginning) phases of the agile transformation are particularly higher for agile taskwork and somewhat higher for agile teamwork.

Figure 1. Observed group differences on the AWPI scale scores based on Welch’s t-tests.

Note. We report Cohen’s d as a standardized measure of effect size, *** p < .001, ** p < .01, * p < .05, ns = not significant.

Results of phase 3: nomological network and observer ratings

Rationale and analytic approach

Another way of showing differences between agile taskwork and agile teamwork is to examine their relationships with variables in the nomological network. Thereby, we can test whether agile taskwork and agile teamwork have different functions (Morgeson & Hofmann, 1999), potentially affecting work outcomes via distinct mechanisms (see, Table 1). To this end, the aim of Phase 3 is to investigate relationships with related constructs using the data of Sample D. Thereby, we test the convergent and discriminant validity of our new instrument (Hypotheses 3 to 7). We also examine relationships with agile coach ratings of AWPs to provide a multi-rater perspective on the construct (Campbell & Fiske, 1959). Finally, we explore whether relationships of AWPs with established team planning and work design measures differ between team types and phases of the agile transformation to address RQ2.

Measures

Whenever scales were not available in German, the items were translated by the first author and back-translated by a research assistant (unclarities in translations were resolved). For all scale scores, significant between-team variance was present, as the F-values calculated with a one-way random-effects ANOVA were all statistically significant (p < .001). We further calculated ICC₂ and r_wgj to examine whether analyses at the team-level were justified. Means, standard deviations, and correlations of the scales based on aggregated team-level data are shown in Table 8. Readers may notice that some of the ICC₂ values are below the recommended .60 value for reasonable group-mean reliability. According to Bliese et al. (2018), low ICC₂ values do not harm the validity of team-level analyses but make it harder to detect statistically significant relationships at this level of analysis (i.e., increase type II errors). We refrained from excluding teams with low response rates or low within-group agreement because simulation studies indicated that doing so further reduces statistical power (due to lowering the Level-2 sample size; see, Biemann & Heidemeier, 2012). Instead, we jointly examine ICC₁, ICC₂, and r_wgj statistics to consider whether analyses at the team-level are appropriate.

Table 8. Pearson product-moment correlations of variables measured in Sample D.

Variable	N	M	SD	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19
1. Teamsize	155	11.54	2.51
2. Team type	155	.19	.39	−.04
3. Agile transformation	155	.48	.50	.22**	.27**
4. Agile taskwork	155	4.6	.84	−.06	−.16*	.04
5. Agile teamwork	155	4.61	.72	−.07	−.02	.04	.60**
6. Iterative development	155	4.53	.70	−.13	−.10	−.02	.72**	.38**
7. Sprints	155	4.67	1.27	−.01	−.16*	.06	.92**	.58**	.40**
8. Stand-up meetings	155	5.75	.91	−.03	−.04	.06	.55**	.91**	.30**	.56**
9. Retrospective meetings	155	3.46	.72	−.09	.01	.01	.51**	.85**	.39**	.46**	.56**
10. Bureaucracy	155	2.19	.64	−.08	.05	−.02	−.21**	−.17*	−.29**	−.11	−.09	−.22**
11. Taskwork planning	155	5.19	.64	−.05	−.03	−.06	.47**	.48**	.55**	.32**	.37**	.49**	−.40**
12. Teamwork planning	155	5.04	.70	−.08	.05	.03	.34**	.38**	.44**	.20*	.30**	.38**	−.26**	.66**
13. Autonomy	155	5.56	.63	−.06	.09	−.07	.19*	.12	.32**	.07	.03	.20*	−.49**	.33**	.32**
14. Feedback	155	4.59	.73	−.12	.04	.04	.40**	.49**	.43**	.30**	.33**	.55**	−.36**	.56**	.57**	.39**
15. AC agile taskwork	66	4.55	1.26	.14	.01	.25*	.53**	.35**	.23	.55**	.27*	.37**	−.05	.16	−.01	−.23	.17
16. AC agile teamwork	66	4.68	.89	.24	.07	.14	.33**	.31*	.13	.35**	.27*	.27*	−.17	.17	−.15	−.37**	.04	.58**
17. AC iterative development	66	4.45	1.09	.05	.00	.21	.35**	.15	.25*	.32**	.12	.14	−.18	.23	.14	−.08	.19	.79**	.43**
18. AC sprints	66	4.66	1.79	.17	.01	.24	.54**	.41**	.18	.59**	.30*	.44**	.04	.09	−.09	−.27*	.13	.93**	.56**	.51**
19. AC stand-up meetings	66	5.73	1.13	.30*	−.03	.16	.30*	.32**	.04	.36**	.33**	.22	−.1	.07	−.24*	−.40**	.03	.53**	.88**	.40**	.51**
20. AC retrospective meetings	66	3.62	.97	.10	.16	.07	.25*	.20	.18	.23	.12	.24*	−.19	.24*	0	−.22	.05	.45**	.83**	.33**	.44**	.46**

Note. Variable 15 to 20 represent agile coach (AC) ratings available for 66 teams. Dotted cells represent the heterotrait-monomethod correlations (i.e., discriminant validity), grey cells represent monotrait-heteromethod correlations (i.e., convergent validity). Cells inside the double-lined box represent heterotrait-heteromethod correlations. Team type is a binary dummy variable (0 = delivery, 1 = support), as is the agile transformation variable (0 = beginning, 1 = advanced), N = sample size, M = mean, SD = standard deviation, ** p < .01, * p < .05.

Agile taskwork

Following the results of Phase 2, we measured agile taskwork with the four iterative development (α = .83) and the four sprint (α = .94) items. Internal consistency estimates justified summing the eight items to an overall agile taskwork score (α = .88, ω_h = .85). Intraclass correlations and within-group agreement statistics supported the validity of analyses at the team-level (ICC₁ = .40, ICC₂ = .74, median r_wgj = .94).

Agile teamwork

Given the results of Phase 2, we measured agile teamwork with the four stand-up meeting (α = .92) and the four retrospective meeting (α = .89) items. Internal consistency estimates supported summing the eight items to an overall agile teamwork score (α = .91, ω_h = .88). Intraclass correlations and within-group agreement indices justified the use of team-level analyses (ICC₁ = .34, ICC₂ = .68, median r_wgj = .94).

Bureaucratic control

We used the 3-item centralized bureaucratic environment scale (α = .78) of Langer et al. (2019). An example item of this scale is “Even small matters have to be referred to someone higher up for a final answer” rated from 1 (fully disagree) to 7 (fully agree). Analyses at the team-level were appropriate according to within-group agreement statistics (ICC₁ = .17, ICC₂ = .46, median r_wgj = .89).

Emergent team planning

Following the work of Fisher (2014), we included a 3-item emergent taskwork planning (α = .81) scale (e.g., “My team considers alternative courses of action for completing the task”) and a 3-item emergent teamwork planning scale (α = .81) scale (e.g., “My team clarifies the roles and responsibilities of all members”), rated from 1 (fully disagree) to 7 (fully agree). Within-group agreement indices supported a team-level analysis of Fisher’s measures, although the ICC₂ statistics were low (for taskwork planning: ICC₁ = .13, ICC₂ = .39, r_wgj = .89; for teamwork planning: ICC₁ = .18, ICC₂ = .47, median r_wgj = .88).

Work design

We adapted the German version of Morgeson and Humphrey’s Work Design Questionnaire (WDQ; Stegmann et al., 2010) to measure team work design. Autonomy was measured as a 3-item index (α = .88) with the highest-loading items for autonomy regarding work methods, work scheduling, and decision-making. Feedback from others was measured with the original 3-item (α = .77) scale. We used the referent-shift composition approach (Chan, 1998) to capture team perceptions of work design (e.g., “The team members receive feedback on their performance from different people in the organization”) from 1 (fully disagree) to 7 (fully agree). Within-group agreement statistics justified team-level analyses of work design measures, although the ICC₂ statistics were somewhat low (for autonomy: ICC₁ = .16, ICC₂ = .45, r_wgj = .92; for feedback: ICC₁ = .16, ICC₂ = .45, median r_wgj = .86).

Discriminant validity

We fit a CFA including the final AWPI items and established measures of team planning, work design, and bureaucracy on the individual-level data of Sample D. Again, for the CFAs we use individual-level data, to achieve a reasonable sample-size-per-parameter ratio (Marsh et al., 2004). Results support the distinctiveness of our new measures because the expected nine-factor solution (i.e., four AWPs, taskwork planning, teamwork planning, autonomy, feedback, and bureaucracy) provided an appropriate fit to the data (χ2 = 1208.61, df = 398, CFI = .92, TLI = .91, RMSEA = .06, SRMR = .05). Merging the indicators of any latent variable together on the same factor resulted in a significant deterioration in model fit (Δχ² ≥ 132.14). Model fit deteriorated the least when merging the indicators of Fisher’s (2014) emergent taskwork and emergent teamwork indicators (Δχ² = 132.14, Δdf = 8, p < .001), and the most when merging the sprint and stand-up meeting indicators (Δχ² = 1120.89, Δdf = 8, p < .001). This shows that the item loadings of the four primary AWP factors are independent of other constructs, satisfying discriminant validity (DeVellis, 2016; Voorhees et al., 2016).

Another way of testing discriminant validity is by forming a multitrait-multimethod matrix (MTMM) and by examining correlations with constructs that are relevant to the context but different in nature (Campbell & Fiske, 1959). As shown in Table 8, the monotrait-heteromethod correlations between member-ratings of AWPs and agile coach-ratings of AWPs were all positive and significant, satisfying convergent validity (see grey cells in Table 8). These correlations were larger than the heterotrait-monomethod correlations between AWPs and centralized bureaucratic control (see dotted cells in Table 8), satisfying discriminant validity. At the team-level, the correlations between AWPs and centralized bureaucratic control ranged from r = −.29 for iterative development to r = −.09 for stand-up meetings. The latent individual-level correlations between centralized bureaucratic control and AWPs were all statistically significant and negative (r ≤ −.11, p ≤ .032). This pattern of empirical relationships is consistent with Hypotheses 3, which stated that AWPs are weakly and negatively related to centralized bureaucratic control, supporting discriminant validity (Campbell & Fiske, 1959).

Convergent validity

Hypotheses 4 to 7 proposed that agile taskwork and agile teamwork are differentially related to emergent team planning and work design constructs. Moreover, we specified these relationships at the team-level, such that we expected that AWPs influence collective team planning and work design experiences. To test these relationships, we conduct multilevel regression analyses following the approach of Bliese et al. (2018). Specifically, we separate the relationships of AWPs with the outcomes at the individual and the team-level by including both within-team centred predictors (i.e., individual-level slopes) and the aggregated versions of the predictors (i.e., team-level slopes). Most relevant for testing our hypotheses are the L2 coefficients in Table 9a because they represent the potential team-level effects of AWPs (see, LoPilato & Vandenberg, 2015). We also include a formal test of how much the team-level slopes differ from the individual-level slopes, indicated by the L2 coefficients in Table 9b. Finally, we explore whether relationships vary across team types and phases of the agile transformation by testing for moderated relationships, as shown in Table 9c.

Table 9. Multilevel regression coefficients.

Model and level of analysis			Taskwork planning			Teamwork planning			Team autonomy			Team feedback
			γ	SE	p	γ	SE	p	γ	SE	p	γ	SE	p
A	L1:	Intercept	3.05	.30	.000	3.16	.35	.000	4.91	.31	.000	2.19	.34	.000
		Agile taskwork (w)	.47	.05	.000	.40	.05	.000	.14	.05	.004	.27	.05	.000
		Agile teamwork (w)	.30	.05	.000	.21	.06	.000	−.01	.05	.863	.33	.06	.000
	L2:	Agile taskwork (b)	.24	.07	.001	.18	.08	.017	.18	.07	.010	.19	.08	.013
		Agile teamwork (b)	.24	.08	.003	.22	.09	.014	−.04	.08	.657	.32	.09	.000
		Team type (0 = delivery team)	.09	.11	.457	.15	.13	.257	.22	.12	.065	.15	.13	.228
		Transformation (0 = beginning)	−.11	.09	.209	−.03	.10	.751	−.10	.09	.277	.01	.10	.924
B	L1:	Intercept	3.05	.30	.000	3.16	.35	.000	4.91	.31	.000	2.19	.34	.000
		Agile taskwork (raw)	.47	.05	.000	.40	.05	.000	.14	.05	.004	.27	.05	.000
		Agile teamwork (raw)	.30	.05	.000	.21	.06	.000	−.01	.05	.863	.33	.06	.000
	L2:	Agile taskwork (b)	−.23	.08	.005	−.22	.09	.017	.04	.08	.661	−.08	.09	.384
		Agile teamwork (b)	−.07	.09	.488	.01	.10	.901	−.03	.10	.784	−.01	.11	.929
		Team type (0 = delivery team)	.09	.11	.457	.15	.13	.257	.22	.12	.065	.15	.13	.228
		Transformation (0 = beginning)	−.11	.09	.209	−.03	.10	.751	−.10	.09	.277	.01	.10	.924
C	L2:	Intercept	3.15	.44	.000	3.41	.49	.000	4.92	.44	.000	2.05	.49	.000
		Agile taskwork (b)	.21	.11	.051	.12	.12	.295	.09	.11	.419	.17	.12	.158
		Agile teamwork (b)	.24	.12	.042	.23	.13	.085	.06	.12	.643	.38	.13	.005
		Team type (0 = delivery team)	−.74	.69	.283	−1.14	.78	.145	.60	.71	.395	−.34	.77	.659
		Transformation (0 = beginning)	.27	.62	.661	.21	.70	.764	−.29	.63	.643	.66	.69	.337
		Agile taskwork * type	.32	.16	.054	.02	.18	.897	.05	.17	.775	.15	.18	.406
		Agile taskwork * transformation	−.09	.14	.518	.09	.16	.593	.15	.14	.316	−.02	.16	.879
		Agile teamwork * type	−.13	.20	.529	.26	.23	.252	−.13	.21	.534	−.04	.22	.862
		Agile teamwork * transformation	.01	.17	.942	−.14	.19	.468	−.10	.17	.550	−.11	.19	.547

Note. Within-team relationships are denoted by L1, between-team relationships are denoted by L2 coefficients. The L2 predictors were aggregated (b), the L1 variables were centred to the group-mean (w) in Model A or included as uncentered scores (raw) in Model B. Following Bliese et al. (2018), the L2 coefficients in Model B represent a formal test of whether regression slopes significantly differ across levels of analysis.

Supporting Hypothesis 4, agile taskwork is uniquely and positively related to emergent taskwork planning. Importantly, the L2 slope in Table 9a is significant (γ = .24, p = .001), indicating that agile taskwork relates positively to collective emergent taskwork planning. Shown in Table 9b, the L2 slope is significantly smaller (γ = −.23, p = .005) than the L1 slope (γ = .47, p < .001). According to Bliese et al. (2018) differences in the strength of statistical relationships across levels of analyses can either indicate a shift in the meaning of constructs and/or a change in reliability. Both explanations are plausible because the emergent taskwork planning measure of Fisher (2014) aims to capture a shared team construct (which may differ in meaning from an individual perception), but at the same time had a mediocre group-mean reliability in our sample (ICC₂ = .39). Supporting Hypothesis 5, agile teamwork is uniquely related to emergent teamwork planning (γ = .22, p = .014), and the individual-level slope (γ = .21, p < .001) is not significantly different (γ = .01, p = .901) from the team-level slope. In line with Hypothesis 6, agile taskwork is uniquely related to team autonomy (γ = .18, p = .010), and the individual-level slope (γ = .14, p = .004) is not significantly different (γ = .04, p = .661) from the team-level slope. Finally, supporting Hypothesis 7, agile teamwork related uniquely to feedback at the team-level (γ = .32, p < .001), and the individual-level slope (γ = .33, p < .001) did not differ significantly (γ = −.01, p = .929) from the team-level slope.

Taken together, the results shown in Table 9 support the convergent validity of our new instrument and indicate that AWPs relate in meaningful ways to collective experiences of emergent team planning (Hypothesis 4 and 5) and work design (Hypothesis 6 and 7). Interestingly, the individual-level slopes and team-level slopes of AWPs with the other variables in the nomological network are highly similar (except for emergent taskwork planning). This corroborates our findings from Phase 1, which implied that there exist isomorphic relationships between the individual-level item responses and the latent team-level constructs (Tay et al., 2014). Finally, shown in Table 9c, the slopes of agile taskwork and agile teamwork do not vary depending on team types or phases of the transformation at AgileOrg, as indicated by the lack of statistically significant moderation effects (all p > .05). Therefore, in response to RQ2, we conclude that relationships in the nomological network do not depend on team types or phases of the agile transformation. Hence, AWPs may have similar effects on team planning and work design across contexts.

Discussion

The present study addresses calls for more generalizable theory (Niederman et al., 2018) and broadly applicable measures of agile work practices (AWPs; Junker et al., 2021; Rietze & Zacher, 2022). Building on the routine dynamics (Feldman & Pentland, 2003; Kremser & Schreyögg, 2016), team process (Fisher, 2014; Mathieu et al., 2020), and work design literature (Bakker & Demerouti, 2017; Parker, 2014), we defined AWPs as clusters of team planning routines that aim to facilitate change-oriented action (i.e., agility; Doeze Jager-van Vliet et al., 2022; Doz, 2020; Grass et al., 2020; Petermann & Zacher, 2021). We introduced four AWPs that are (a) practiced across a broad range of organizations (Tripp & Armstrong, 2018), (b) potentially enable continuous change (Brown & Eisenhardt, 1997), and (c) are distinguishable on a taskwork-teamwork continuum (Crawford & LePine, 2013).

Agile taskwork is characterized by the reliance on short-term goal cycles (i.e., sprints) and an iterative approach to task accomplishment. Agile teamwork happens in meetings that aim to enable goal monitoring (i.e., stand-up meetings) and reflexivity (i.e., retrospective meetings). According to agile management frameworks (Anderson, 2010; Rigby et al., 2020; Schwaber & Sutherland, 2017), these practices enhance the speed of decision-making, allow for rapid experimentation, and foster co-creation with customers. These are important capabilities for work in the 21^st century (Girod & Králik, 2021; Lifshitz-Assaf et al., 2021). Whether AWPs indeed create these dynamic capabilities is a matter of empirical investigation. The present study sets the groundwork for such investigations in terms of theory and measurement. In the following, we discuss the evidence for the proposed theoretical framework and summarize the psychometric properties of the Agile Work Practices Instrument (AWPI).

Validity of the measures

Building on established theoretical dichotomies (Bales, 1950; Crawford & LePine, 2013; Fisher, 2014), we set out to validate the AWPI in terms of taskwork and teamwork. Results of exploratory and confirmatory factor analyses (CFA) indicated that the four AWPs are empirically distinct, but ground in two higher-order factors (Hypotheses 1 and 2). Specifically, sprints and iterative development load on a higher-order taskwork factor, whereas stand-up and retrospective meetings load on a higher-order teamwork factor. Despite being theoretically distinct, the two higher-order factors are empirically not independent as shown by the moderate correlation of agile taskwork and agile teamwork (team-level correlations of r = .49 in Sample B, r = .47 in Sample C, and r = .60 in Sample D). Hence, teams that practice agile taskwork tend to simultaneously practice agile teamwork. It should also be noted that the fit of the hypothesized taskwork-teamwork model was only slightly better than the fit of the model with one higher-order factor. Nevertheless, we are confident that the taskwork-teamwork model of AWPs will replicate³ beyond the present study, and we encourage future research to follow this conceptualization because agile taskwork and agile teamwork may function via distinct mechanisms. Similar conceptualizations have been adopted to understand the mechanisms of leadership (Yukl et al., 2002), diversity (Gerpott & Lehmann-Willenbrock, 2016), and conflict (Shah et al., 2021). Thus, an understanding of AWPs in terms of taskwork (goal accomplishment needs) and teamwork (socio-emotional needs) may facilitate an integration of the concept with the broader literature (e.g., Abele & Wojciszke, 2014).

The cross-lagged analyses (Phase 1) indicated that agile teamwork was predictive of changes in agile taskwork over a one-year period (ß = .22), but not vice versa (ß = .02). This provides additional support for the distinctiveness of the two dimensions. If agile taskwork and agile teamwork were the same construct and empirically redundant, the cross-lagged relationship would not be expected because all variance would be “absorbed” by the auto-correlation paths (see, Kenny, 1975). From a theoretical stance, the finding that only agile teamwork exhibited significant cross-lagged effects aligns with the idea that early interpersonal activities during a team’s history can have long-lasting effects on taskwork activities (Tuckman, 1965). Agile teams that dedicate more attention to forming a shared understanding of members’ roles and responsibilities during stand-up and retrospective meetings (i.e., teamwork) may over time be more able to engage in iterative development activities and to work in sprints (i.e., taskwork). Although we did not predict this finding a priori, it aligns with prior research showing that the development of team charters and performance strategies mainly benefits long-term rather than short-term group functioning (Mathieu & Rapp, 2009). In agile teams, such activities happen in the context of stand-up or retrospective meetings (Hummel et al., 2015). Hence, future research on agile teams in organizational settings may be able to replicate past findings that are based on research with students engaged in business simulations (Mathieu & Rapp, 2009) or course projects (Druskat & Kayes, 2000).

The taskwork-teamwork distinction was also supported in the analysis of the nomological network during Phase 3. Findings of CFAs and multilevel regression analyses indicated that AWPs are related but distinct from other bureaucracy, team process, and work design constructs. The four AWPs have in common that they are weakly and negatively associated with centralized bureaucratic control (Hypothesis 3), which may imply that these practices lie on a different bureaucratic continuum (i.e., concertive instead of centralized; Monteiro & Adler, 2021). Providing evidence for convergent validity, we found that agile taskwork related positively to shared experiences of autonomy and emergent taskwork planning. In contrast, the more the teams practiced agile teamwork, the more their members collectively experienced adequate feedback and engaged in teamwork planning behaviour. We emphasize the collective nature of these empirical relationships to highlight that our findings go beyond subjective perceptions. This is because establishing statistical relationships among collective constructs requires a high degree of intersubjectivity (i.e., shared agreement among team members as indicated by r_wgj and ICC₂ statistics; Bliese et al., 2018). Taken together, the findings support Hypotheses 4 to 7, suggesting that agile taskwork and agile teamwork may function via distinct team planning and work design mechanisms (see, Table 1). While agile taskwork relates to emergent taskwork planning behaviour and team autonomy, agile teamwork converges with emergent teamwork planning behaviour and seems to open feedback channels.

Given the complex nature of the AWP construct, it is unsurprising that not all statistical relationships can be explained in terms of the latent taskwork-teamwork factors. More complex relationships among the four AWPs can be explained by examining agile teams as complex adaptive systems (CAS; McGrath et al., 2000; Ramos-Villagrasa et al., 2018). From this perspective, we may see AWPs as a dynamic system resulting from team members’ actions (taskwork) and their interactions (teamwork). This system is constrained by contextual dynamics that affect the group’s developmental trajectories. For instance, the addition of a new team member may require more frequent agile meetings to re-establish roles and responsibilities. At the same time, additional labour may allow for reducing the duration of work cycles by lowering the total workload per team member (Van Oorschot et al., 2018). To validate more complex explanations of the interrelations between AWPs, psychometric network modelling is a potentially suitable approach (Schmittmann et al., 2013). Rather than explaining the covariance of the different AWPs with latent variables (e.g., taskwork-teamwork), this approach would assume that the covariance emerges due to causal “interactions” between the specific practices. To enhance the validity of this approach (Borsboom et al., 2021), it will be necessary to collect multiple waves of data to provide stronger evidence for causal interactions of agile practices in a system. Since psychometric network and latent variable models can be statistically equivalent (Van Bork et al., 2021), researchers may also employ simulation techniques to examine how AWPs function as a system (e.g., Van Oorschot et al., 2018). To this end, the results of survey studies may provide useful starting values for building more complex computational models. Predictions from these models can then be validated using data of real teams (McGrath et al., 2000). Thus, even with the advent of computational modelling, survey measures remain important for studying agile teams as complex adaptive systems.

Applicability of the measures

An integration of the AWP concept in organizational research requires a broadly applicable measurement instrument of AWPs that can be used across work contexts – beyond software development (see Junker et al., 2021). The findings of Phase 1 indicated that the use of AWPs is not restricted to a single agile method, team type, or level of task interdependence. Next, in Phase 2, we showed that the measurement properties of the AWPI are similar for different types of teams and in different phases of their agile transformation. While the AWP measures had similar meanings for broad groups of participants, members of delivery teams had higher mean scores on agile taskwork and agile teamwork than members of support teams (RQ1). Hence, AWPs are used more consistently in contexts where individuals are directly in charge of producing value for external customers (here customers received IT services, maintenance, and consulting). The most pronounced differences between delivery and support teams existed for the dimension of agile taskwork. This may indicate that it was more difficult to plan tasks in short iterative cycles in support teams, who mainly created value within the organization (here HR, finance, and sales teams). Nevertheless, our findings regarding measurement invariance show that (1) the AWPI can be administered in a broad range of work contexts, and (2) observed score differences can be interpreted as “true” differences in the underlying latent AWP construct (Vandenberg & Lance, 2000). Thus, we are confident that the AWPI is a broadly applicable measure, fulfiling validity standards for research in management and organizational psychology (Cortina et al., 2020).

We highlight two important observations regarding the applicability of the construct and the generalizability of our findings that emerged during Phase 3 of this study. First, we found that AWPs are related to more established team planning and work design constructs in similar ways for delivery and support teams. Second, the relationships were not otherwise explained by the agile transformation programme. In other words, although certain teams may use AWPs more frequently, these practices appeared to have similar relationships with work design and team planning across different sub-samples. It should be noted that we did not have specific expectations about this and decided to approach this empirical issue with a research question (RQ2). The present study may be somewhat underpowered to find potential moderated effects of AWPs because all participating teams were from the same organization, which may have reduced heterogeneity in the use of AWPs and their effects. We do not intend to convey that AWPs have causal effects or that these practices can be used in all types of teams to enable continuous change. Yet, we conclude that valid measurement of AWPs beyond classic areas of application is possible (see also Junker et al., 2021; Koch & Schermuly, 2021), and that our measures have similar psychometric properties in different samples. Moreover, certain AWPs can arguably be implemented in almost any type of team. For instance, retrospective meetings represent a type of after-action-review (or debrief), which has been successfully implemented in a broad of different teams (for a meta-analysis, see, Keiser & Arthur, 2021). Thus, the AWP concept, or at least certain practices, seem applicable and measurable in a broader range of work contexts than initially thought.

Limitations and future research

Although our validation study was extensive, some limitations and open questions about the measurement and mechanisms of AWPs remain. First, our theorizing regarding the distinct temporal mechanisms of the four AWPs (see, Table 1) needs to be validated. We based this theorizing on the work of Brown and Eisenhardt (1997) who used an inductive case study approach to investigate practices that enable continuous change by creating “links in time” (p. 29). Hence, the distinct temporal mechanisms of the four AWPs can potentially be tested by administering the AWPI and relating score differences to qualitative observations in terms of temporal patterning (e.g., Kremser & Blagoev, 2021). Another way to validate the temporal mechanisms may be to examine how the use of AWPs impacts perceptions of subjective time (Shipp & Jansen, 2021) and team reflexivity (Schippers et al., 2007). For instance, based on our framework, we may expect that the use of retrospective meetings induces a past focus among team members (Shipp et al., 2009).

Second, the present study was limited in terms of providing evidence for the criterion validity of the AWPI. Although good work design is an important criterion in work and organizational psychology (Parker & Jorritsma, 2021), future research may consider more classic outcomes in terms of well-being and performance. There is some evidence that the use of AWPs relates positively to work engagement (Rietze & Zacher, 2022), psychological empowerment (Koch & Schermuly, 2021), as well as team proactivity and performance (Junker et al., 2021). What currently makes it hard to summarize empirical findings on AWPs is the diversity of ways in which the construct has been operationalized. By administering the AWPI, future research can establish the criterion validity of this new instrument. A consistent use of similar measures would facilitate the integration of the agility literature and enable more valid meta-analytic research.

Third, there were some associations between agile coach ratings and team member ratings that did not align with our predictions. For instance, agile coach ratings of stand-up meetings correlated negatively with member ratings of emergent teamwork planning (r = −.24) and autonomy (r = −.40), opposite to what was expected. It is possible that an overinvolvement of agile coaches in stand-up meetings potentially undermines spontaneous planning activities by members (Fisher, 2014; Ghosh & Wu, 2021), and reduces collective experiences of autonomy (Van Mierlo et al., 2006). Thus, rather than being a methodological artefact, such unanticipated correlations may indicate how much the leaders and members of the same team can diverge in their evaluation of group functioning (see, Bolinger et al., 2020). Hence, future research may investigate why leaders and members diverge in their perceptions of AWPs, beyond the usual methodological arguments that are utilized to explain a lack of convergence in multisource ratings (e.g., common method variance).

Finally, it is necessary to cross-validate our findings based on new data from multiple organizations. Although we had access to a wide range of different teams at AgileOrg, the context of this organization was rather unique. For instance, there was strong management support for the implementation of AWPs, and the organization transitioned to agile working in a stepwise manner (instead of a big-bang approach; see, Denning, 2021). Our findings regarding measurement invariance (Phase 2) increase our confidence that the AWPI will retain its psychometric properties in more diverse samples. Moreover, we constructively replicated the proposed factor structure in an independent sample with other self-report measures of AWPs (see https://osf.io/ndtx4/). Nevertheless, it is possible that relationships with criterion variables differ between organizations or industrial sectors. Thus, a promising avenue for future research will be to investigate which organizational contexts enhance (or undermine) the potential effects of AWPs on well-being, performance, and innovation outcomes.

Practical implications

As reflected in the label of the concept, AWPs originated in practice and refer to tangible routines that are already used by many organizations worldwide (Annosi et al., 2020). Paradoxically, our main practical contribution may thus be that we make the concept of agile working more theoretical, following the statement attributed to Kurt Lewin “nothing is so practical as a good theory” (Lewin, 1945). By integrating the AWP concept with our knowledge of work and organizational psychology, we provide a new framework that practitioners can use to guide the implementation of AWPs and understand their mechanisms. The here developed instrument can also be administered to measure the progress of agile transformation programs. According to our findings, the AWPI is valid in different team contexts and may therefore serve as a valid empirical benchmark for agile transformations.

The theoretical framework that we developed to validate the AWPI may also be used to improve the practice of agile working in organizations. Rather than focusing on the adherence to the rules of agile frameworks (e.g., Scrum), we recommend that practitioners focus on theoretically derived mechanisms of AWPs in terms of (a) temporality, (b) managerial control, (c) team planning, and (d) work design (see, Table 1). Organizations may be able to reap the benefits of AWPs by implementing them in ways that enable teams to create links in time (Brown & Eisenhardt, 1997), reduce centralized bureaucratic control (Monteiro & Adler, 2021), promote emergent team planning (Fisher, 2014), and provide favourable work design (Parker, 2014). Thus, organizations may focus on these evidence-based mechanisms rather than blindly following popular agile management frameworks (Sutherland, 2014). With their knowledge of these mechanisms, work and organizational psychologists can be at the forefront of evidence-based agile transformation programmes.

Conclusion

This study introduces theory and measurement frameworks for research on agile working by providing evidence for the factor structure, reliability, and construct validity of the Agile Work Practices Instrument (AWPI). In line with the taskwork-teamwork dichotomy, we find that AWPs are related but distinct from other constructs in terms of bureaucracy, emergent team planning, and work design. Moreover, our findings show that agile taskwork and agile teamwork may function via distinct team planning and work design mechanisms. We also conclude that the AWPI can be used for research on agility in a range of work contexts and that the concept of agile working is more broadly applicable than commonly thought. Whether the use of AWPs is indeed necessary or sufficient for enabling continuous change is a matter of future empirical research. The present study provides the basis for this research in terms of the measurement and mechanisms of agile work practices.

Data availability

Analysis code and materials are available online (https://osf.io/ndtx4). Data is available on request.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/1359432X.2022.2096439

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Notes

1. Consider, for instance, the practice of uploading draft versions of manuscripts on pre-print servers or for conference presentations. This can be seen as an iterative development practice because it represents a low-cost experiment through which authors engage with the end-users of their work. This may enable scholars to continuously update their work with advancements in the field they may otherwise be unaware of.

2. Please note, that overlap in terms of variables only exists for our measures of agile work practices. The aim of Junker et al. (2021) was to test a theoretical model of agile working in relation to proactive behaviour and performance. The present study uses the data of Junker et al. (2021) only to provide information on the factorial validity and reliability of the AWPI in Sample B.

3. A re-analysis of publicly available data of AWPs collected by other researchers (https://osf.io/xpzbq), constructively replicates our distinction between agile teamwork and agile taskwork in an independent sample with different self-report measures. Details regarding the re-analysis, including information on CFA model fit and the analysis code can be found in the online supplement (https://osf.io/ndtx4/).