655
Views
0
CrossRef citations to date
0
Altmetric
Research Article

Early career and seasoned referees: contrasting motivation, perceptions of organisational support and intention to continue

, , &
Received 07 Feb 2023, Accepted 09 Sep 2023, Published online: 31 Oct 2023

ABSTRACT

Research question:

The aim of this research was to better understand how perceived organisational support relates to a referees’ motivation and intention to continue (ITC).

Research methods:

320 early-career referees (ECRs) and seasoned referees (SRs) were compared across motivation (both onset and continuing dimensions), perceived organisational support and ITC. To analyse the effects of motivation and perceived organisational support on a re-coded dichotomous ITC variable, a series of binary logistic regression analyses were used. Based on tenure, 123 referees were classified as ECRs in their first five years, the remaining 195 were classified as SRs.

Results and findings:

Results indicated that motivation differs between ECRs and SRs. Furthermore, evidence was found for perceived organisational support as a moderator of sport-related motivation. Whereas, perceived organisational support was not found to moderate the intrinsic, extrinsic, and social motivation–ITC relationship in any way.

Implications:

Analyses of onset and continuing motivation subscales yielded new insights, and it was also found that ECRs and SRs are not homogeneous groups when it comes to their motivation, perceived organisational support and how it affects their ITC. Sport managers should purposefully tailor their support for ECRs and SRs.

Introduction

The extent to which referees stay in the role varies (Bernal et al., Citation2012; Forbes & Livingston, Citation2013), but the reasons why are still not well understood. Although researchers have explored continuation amongst referees, a comprehensive understanding is still lacking (Forbes & Livingston, Citation2013). Numerous studies have explored referee motivation, which is an important aspect of the underlying referee psychology (Fowler et al., Citation2019; Hancock et al., Citation2015; Hong et al., Citation2019; Livingston & Forbes, Citation2017). However, the construct is not as useful when conceptualised and operationalised broadly and alone is a weak predictor of a referee’s ongoing engagement (Giel & Breuer, Citation2020; Hoye et al., Citation2008). A narrower focus on motivation is needed at the dimension level because of its varied nature and complex relationship with behaviours (Johansen, Citation2015; Symonds & Russell, Citation2018). Likewise, conceptualising and operationalising onset and continuing motivation separately has value and has precedent elsewhere (Auger et al., Citation2010; Hancock et al., Citation2015; Livingston & Forbes, Citation2017).

The experiences of referees are multifaceted (Ridinger, Tingle, et al., Citation2017), including factors related to the organisation they are connected. Insights on how these organisations influence the referee experience will help sport managers take action to facilitate continued engagement. Studies have found that organisational and managerial factors influence the continuation of referees (Forbes & Livingston, Citation2013; Warner et al., Citation2013). It has increasingly become a priority to better understand referee perceptions of organisational support (e.g. Cuskelly & Hoye, Citation2013; Giel & Breuer, Citation2020; Ridinger, Kim, et al., Citation2017). Organisational Support Theory (OST) underpins the construct and takes into account elements of fairness, supervisor support and job conditions (Eisenberger & Huntington, Citation1986; Rhoades & Eisenberger, Citation2002). OST allows for a focus on the employee-organisation relationship from the employees’ viewpoint (Kurtessis et al., Citation2017; Shanock et al., Citation2019). In contrast to players and coaches within organised sport (Livingston et al., Citation2017), much less is known about referees and their experience being connected to organisations. Perceived organisational support has been found to moderate the motivation-continuation relationship outside of sport contexts (Perryer et al., Citation2010; Treglown et al., Citation2018), so this should be explored as potentially characterising the referee experience.

Furthermore, the experience of early-career referees (i.e. those who have been refereeing for five years or less) is different than their more seasoned colleagues (i.e. six years or more), and it is likely that the nature of that experience affects continuation (Cuskelly & Hoye, Citation2013; MacMahon et al., Citation2014). As per Cuskelly and Hoye (Citation2013), those who have been active for five years or less were referred to as early career referees (ECRs). Whilst experiences have not yet been empirically contrasted by tenure, evidence elsewhere suggests that subgroups within sport referee organisations are treated differently. For example, Kellett and Warner (Citation2011) found that various roles within the Australian Football community (i.e. Line Umpires or Goal Umpires) had different experiences. Therefore, it is important to understand if differences exist due to tenure. This is important because many ECRs do not continue after five years (Cuskelly & Hoye, Citation2004; MacMahon et al., Citation2014), and specific insights of this phenomenon may help to increase tenure.

The aim of this research was to better understand the motivation, perceived organisational support, and intention to continue of rugby union referees and expands the body of referee research in three important ways. First, ECR perceptions of organisational support and motivation were contrasted with those of seasoned referees (SRs). Second, the comparison of motivation across these two types of referees also included distinction between onset motivation (i.e. drivers as one started to referee) and continuing motivation (i.e. drivers of one’s ongoing involvement). Third, testing the moderating effect of perceived organisational support sheds further light on the all-important motivation-ITC relationship. Insights related to these unique design elements will be actionable for those managing referees and will likely increase referees’ tenure. Thus, this study has both practical and theoretical implications.

Conceptual background and review of literature

It is well documented that there is a shortage of referees at the grassroots level (Jacobs et al., Citation2020; Jordan et al., Citation2019; Livingston & Forbes, Citation2017; Ridinger, Citation2015; Zvosec et al., Citation2021), but a coherent plan to rectify this is not yet evident (Forbes & Livingston, Citation2013). The outcome of a referee continuing is often operationalised in related research as their “intention to continue” (Choi & Chiu, Citation2017; Cuskelly & Hoye, Citation2013). This needs to be a focal point, especially for ECRs. The majority of referee research has recently focused on worsening abuse (Dawson et al., Citation2021; Jacobs et al., Citation2020; Webb, Dicks, et al., Citation2019; Webb, Rayner, et al., Citation2019). However, other aspects of the referee experience warrant study, including the support an organisation can provide. While in recent years, scholarly inquiry into the role of refereeing organisations has increased (e.g. Baldwin & Vallance, Citation2016; Giel & Breuer, Citation2020; Kim et al., Citation2022; Livingston & Forbes, Citation2016, Citation2017; Wicker & Frick, Citation2016), it is evident that further research is required to understand how refereeing organisations can support and increase retention.

Intention to continue

Facilitating an environment in which referees continue in their role should be a central focus for managers across all sports. A referee’s stated intention to carry on in their role is, therefore, a vital metric to better understand. An intention is “a measure of the likelihood that a person will engage in a given behaviour [which] may be termed behavioural intention” (Ajzen & Fishbein, Citation1980, p. 5). The value of the construct has been established across numerous contexts because it can effectively predict turnover behaviour (VanYperen, Citation1998). Actual behaviours are the most accurate measure of continuation, but ITC serves as a useful proxy in the context of volunteers (Kim et al., Citation2009). ITC is a central construct across a breadth of sport referee research (Cuskelly & Hoye, Citation2013; Giel & Breuer, Citation2020; Ridinger, Kim, et al., Citation2017). It is believed that referee motivation (e.g. Giel & Breuer, Citation2020; Hancock et al., Citation2015; Livingston et al., Citation2017; Livingston & Forbes, Citation2016, Citation2017) and perceived organisational support (Choi & Chiu, Citation2017; Cuskelly & Hoye, Citation2013; Giel & Breuer, Citation2020; Kim, Citation2017; Livingston et al., Citation2017; Livingston & Forbes, Citation2016) relate to ITC but a more nuanced understanding of these constructs is still lacking.

Motivation

Sport referee motivation includes intrinsic, extrinsic, sport-related, and social dimensions (Hancock et al., Citation2015). Intrinsic and extrinsic motivations have been thoroughly covered across a number of refereeing contexts (Fowler et al., Citation2019; Livingston & Forbes, Citation2016, Citation2017). While sport-related and social dimensions are intuitive, they were only recently conceptualised by Hancock et al. (Citation2015), who argued for their unique contribution to capture referees’ motivation more fully.

Intrinsic Motivation (IM)

IM captures an individual’s inherent and natural drive to seek out activities and challenges which promote social and cognitive development (Vansteenkiste et al., Citation2004). For referees, IM is a powerful driver and includes proving oneself, personal development, enjoyment, and passion for their sport (Hancock et al., Citation2015). Referees have also identified devotion, love and passion as drivers of their involvement (Auger et al., Citation2010; Forbes & Livingston, Citation2013; Kellett & Warner, Citation2011; Livingston et al., Citation2017; Livingston & Forbes, Citation2017). This was further reinforced by Livingston et al. (Citation2017), who, in their call-to-action paper, cited numerous studies that identified individuals entered into refereeing because of their love for their sport. Further IM research is needed contrasting newer referees and those who have been in the role longer, as well as how IM is experienced when one begins in the role compared to how it drives ongoing participation.

Extrinsic Motivation (EM)

Remuneration, recognition, influence and the potential to exercise power are externally-oriented motivations identified by referees (Hancock et al., Citation2015). EM also includes the encouragement of others (Auger et al., Citation2010; Livingston & Forbes, Citation2017). Although not all referees are paid, remuneration is one of the most oft-studied aspects of referee motivation (Bernal et al., Citation2012; Johansen, Citation2015; Kellett & Warner, Citation2011; Livingston & Forbes, Citation2016; Parsons & Bairner, Citation2015). In the study of Parsons and Bairner (Citation2015), they examined Football referees from across England and found that remuneration influenced their participation as a referee. It has been put forth that remuneration is simply a bonus for referees (Kellett & Warner, Citation2011; Schaeperkoetter, Citation2017), but elsewhere an effective recruitment tool (Cuskelly & Hoye, Citation2004; Kellett & Warner, Citation2011; Livingston et al., Citation2017; Livingston & Forbes, Citation2016; Ridinger, Citation2015; Warner et al., Citation2013). However, insufficient remuneration has been linked to discontinuation (Forbes & Livingston, Citation2013; Kim, Citation2017). Research exploring EM across referee tenure and distinguishing between onset and continuing EM is currently lacking.

Sport-Related Motivation (SRM)

Context-specific factors (i.e. related explicitly to officiating and the sport itself) also motivate referees. This may relate to their previous experiences with the sport or responding to emergent needs in the sport community. Many referees engage because they want to give back to their sport (Livingston & Forbes, Citation2017; Ridinger, Kim, et al., Citation2017; Schaeperkoetter, Citation2017). This forms part of a distinct motivational dimension connected to the sport itself. SRM is comprised of a variety of other drivers, including having a strong connection with the sport (Baldwin & Vallance, Citation2016; Johansen, Citation2015; Ridinger, Citation2015). Often the desire to stay involved can be compromised by injury (Livingston & Forbes, Citation2016) or physical incapacity (Baldwin & Vallance, Citation2016), leading to a decision to enter the officiating ranks in response to observed referee shortages (Livingston & Forbes, Citation2016, Citation2017; Ridinger, Citation2015), and in some cases to fill a quota (Dosseville et al., Citation2013; Livingston & Forbes, Citation2016). Although SRM drives referees, further exploration is required to understand how this may differ at different stages during a referee’s career.

Social Motivation (SM)

SM in this context captures a set of related drivers to integrate oneself within social settings through refereeing. This involves meeting people, making friends and experiencing new things with others (Hancock et al., Citation2015). Opportunities to travel with others (Jiranek et al., Citation2013; Johansen, Citation2015; Wang et al., Citation2011), stay connected in a sporting community (Livingston & Forbes, Citation2017, Citation2016), and experience camaraderie and support from other referees (Baldwin & Vallance, Citation2016) have been explored and contributed to our evolving understanding of this motivation dimension. It has been well documented that a sense of community is an important aspect of the refereeing experience (Kellett & Warner, Citation2011; Livingston & Forbes, Citation2017; Ridinger, Citation2015; Warner et al., Citation2013), and this relates to social motivation. Therefore, sport managers should prioritise creating environments that facilitate connections and friendship (Kellett & Shilbury, Citation2007). The benefits of doing this mean that referees can have informal conversations that can support them through issues they may encounter during their experiences (Cuskelly & Hoye, Citation2004, Citation2013; Livingston & Forbes, Citation2017). It is acknowledged that these latter two dimensions (i.e. SRM and SM) have some conceptual overlap with the EM dimension. However, the Hancock et al. (Citation2015) conceptualisation in which SRM and SM were measured separately is worth exploring further. As with other dimensions of referee motivation, an important next step is to contrast SM across ECRs and SRs as well as establish how SM may differ at role onset versus on an ongoing basis.

Onset and continuing motivation

Despite a compelling rationale to do so in the context of referees, onset motivation (i.e. drivers as one started to referee) and continuing motivation (i.e. drivers of one’s ongoing involvement) have only recently been conceptualised separately (Hancock et al., Citation2015). This is an important emerging area in the body of referee literature, as insights on this will be valuable to sport managers. Specifically, onset and continuing motivation are yet to be linked to other aspects of the referee experience or contrasted between ECRs and SRs. Continuing motivation is potentially more closely related to ITC than onset motivation, although no research yet exists to demonstrate this. Some work has been done to understand the motivation of referees along their journey (Auger et al., Citation2010; Hancock et al., Citation2015; Livingston & Forbes, Citation2017), and at least one study of sport volunteers (albeit not referees specifically) has reported that motivation differs over time (Hoye et al., Citation2008).

Although motivation is an important aspect of the wider referee experience, it does reflect an intrapersonal factor, and it is important to consider what else in the environment influences the ultimate intention to carry on in the role. In fact, organisational factors have been found to influence the continuation of referees (Forbes & Livingston, Citation2013; Hancock et al., Citation2015; Kim, Citation2017; Livingston et al., Citation2017; Livingston & Forbes, Citation2016; Warner et al., Citation2013) and recent focus has been on perceived organisational support (Choi & Chiu, Citation2017; Cuskelly & Hoye, Citation2013; Giel & Breuer, Citation2020; Kim, Citation2017; Livingston et al., Citation2017; Livingston & Forbes, Citation2016). Perceived organisational support has been used broadly in other contexts to good effect in order to understand ITC (Rhoades & Eisenberger, Citation2002). Thus, its role within the referee experience potentially warrants focused attention to advance a more rounded understanding of motivation.

Perceived organisational support

Perceived organisational support refers to the extent to which an individual believes that an organisation values their contribution and cares for them (Kurtessis et al., Citation2017). It is underpinned by OST, which captures fairness, supervisor support and job conditions (Eisenberger & Huntington, Citation1986; Rhoades & Eisenberger, Citation2002). OST explains how favourable treatment by the organisation is perceived and leads to outcomes such as retention (Shanock et al., Citation2019). Within the context of OST, employers reward employees through pay and promotion but also with socio-emotional benefits like approval. A positive perception of the organisation, through social exchange, increases trust, felt obligation and expectation where efforts conducted on behalf of the organisation will be rewarded (Kurtessis et al., Citation2017).

OST builds on the application of Social-Exchange Theory between employer and employee. Employees develop a perception of organisational support as a result of an employer’s response to the socio-emotional needs of an employee and the employer’s willingness to reward increased output and efforts (Baran et al., Citation2012; Eisenberger & Huntington, Citation1986; Rhoades & Eisenberger, Citation2002). OST suggests that the more positive resources provided to an employee, the greater the perceived organisational support. Therefore, the more obliged an employee feels to reciprocate and support the organisation’s pursuit of various outcomes (Caesens & Stinglhamber, Citation2020; Eisenberger & Huntington, Citation1986; Kurtessis et al., Citation2017; Rhoades & Eisenberger, Citation2002). According to OST, if employees experience favourable treatments, this is interpreted as the organisation being caring and supportive, which has a positive influence on outcomes such as performance, engagement and retention (Shanock et al., Citation2019).

Despite the fact that referees are unlike employees in more traditional organisations, the perception of support is still vitally important and is likely to influence their continuing engagement. Many referees feel disappointed by their refereeing organisation (Parsons & Bairner, Citation2015), reflecting a lack of perceived organisational support (Forbes & Livingston, Citation2013). There is also evidence that perceived organisational support declines over time for referees (Livingston & Forbes, Citation2016). Declining perceived organisational support for SRs could be related to another perception – that newcomers to refereeing are being prioritised. Refereeing organisations have been found to value younger referees more than their older counterparts (Livingston & Forbes, Citation2017). Although it has been reported that lower levels of perceived organisational support are linked to discontinuation (Choi & Chiu, Citation2017; Cuskelly & Hoye, Citation2013; Forbes & Livingston, Citation2013; Livingston et al., Citation2017; Warner et al., Citation2013; Webb et al., Citation2018), the consequence is not yet well established and warrants deeper consideration. ECR retention is increasingly a focus for refereeing organisations (Cuskelly & Hoye, Citation2004; MacMahon et al., Citation2014). However, if priority and resources shift too much, SRs may be neglected, and this may lead to referees of varying tenure leaving the role.

Perceived organisational support has been reported as a strong indicator of turnover intention (Choi & Chiu, Citation2017) and to have a positive relationship with ITC (Giel & Breuer, Citation2020; Livingston et al., Citation2017). In contrast, perceived organisational support has also been reported to have no impact on ITC (Cuskelly & Hoye, Citation2013). No study has yet explored the potential moderating effect of perceived organisational support on the motivation-ITC relationship. This despite researchers having previously connecting a referee’s motivation to their perception of organisational support (Livingston & Forbes, Citation2016). Furthermore, exploring this moderating effect is consistent with an ecological systems perspective in which the demands of the task that referees “engage in, and the environmental or systemic factors that weigh on one’s officiating experience” (Livingston & Forbes, Citation2016, p. 344). In the human resource and organisational development literature, perceived organisational support has been used extensively as a moderator, including of ITC as an outcome (Perryer et al., Citation2010). Considering that motivation itself is a relatively weak predictor of continuation (Giel & Breuer, Citation2020; Hoye et al., Citation2008), we argue that exploring perceived organisational support as a moderator and considering motivation at dimension level is needed. Therefore, to understand the rugby union referee experience in more depth, four sets of hypotheses were explored in this study. The hypotheses are numbered based on motivation dimensions, and separately lettered to include the experience and moderation elements of this study.

The first two hypotheses relate to intrinsic motivation:

H1a: The relationship between intrinsic motivation and ITC differs between early-career and seasoned referees.

H1b: Perceived organisational support moderates the relationship between intrinsic motivation and ITC

The next two hypotheses relate to extrinsic motivation:

H2a: The relationship between extrinsic motivation and ITC differs between early-career and seasoned referees.

H2b: Perceived organisational support moderates the relationship between extrinsic motivation and ITC

These two hypotheses relate to sport-related motivation:

H3a: The relationship between sport-related motivation and ITC differs between early-career and seasoned referees.

H3b: Perceived organisational support moderates the relationship between sport-related motivation and ITC

The final two hypotheses relate to social motivation:

H4a: The relationship between social motivation and ITC differs between early-career and seasoned referees.

H4b: Perceived organisational support moderates the relationship between social motivation and ITC

This study also included analyses of each motivational dimension in its full form (i.e. both onset and continuing motivation subscales), as well as each subscale separately. Moderation analyses (i.e. H1b, H2b, H3b and H4b) were most meaningful for the continuing motivation subscales as opposed to the onset scale, at which point a referee would not yet have a perception of how supportive their organisation would be.

Methods

Participants

New Zealand-based referees (n = 318) consented to and completed an online questionnaire in a cross-sectional design, having received an email invitation using a national database. Based on tenure, 123 were classified as early-career referees (ECRs) in their first five years, and 195 were classified as seasoned referees (SRs). The ECR group was younger, with 69% under 44 years old. For the SR subgroup, 66% were over 45 years old (). Overall, 25% of respondents were from the Auckland region, and 96% were male.

Table 1. Age distribution by subgroups.

Instrumentation

A total of 50 items were included in the questionnaire (), most of which were used to measure the complexity of motivation as a key driver of the referee experience. This included 40 items measuring Intrinsic Motivation (IM), Extrinsic Motivation (EM), Sport-Related Motivation (SRM) and Social Motivation (SM). All motivation items were derived from Hancock et al. (Citation2015). Although Hancock et al. (Citation2015) used a four-point scale, to ensure consistency across the remaining items, a 7-point Likert scale was used here (Croasman & Ostrom, Citation2011; Joshi et al., Citation2015). These items originally went through a development process (Auger et al. Citation2010) including focus groups and pilot testing.

Table 2. Questionnaire items.

Eleven motivation scale variations were utilised in the analyses. Four of those scales were the full-form motivational dimensions and reliability was acceptable: IM (α = .86), EM (α = .82), SRM (α = .81), and SM (α = .82). Seven subscales capture the onset (O) and continuing (C) motivation items for each dimension: OIM (α = .77), CIM (α = .80), OEM (α = .79), CEM (α = .70), OSRM (α = .82), OSM (α = .76), CSM (α = .66). A subscale for continuing sport-related motivation (CSRM) was not included in the analyses as the internal reliability of these items was poor (α = .40), despite the onset and continuing items performing reliably in its full form as SRM. Perceived organisational support was measured with seven items (α = .92), and ITC was measured with three items (α = .93). The onset and continuing motivation items were prefaced with “I became a rugby referee in order to” and “I am a rugby referee because” respectively (Hancock et al., Citation2015) but were otherwise distinct. Seven items used to explore perceived organisational support were drawn from Cuskelly and Hoye (Citation2013), who were themselves guided by Eisenberger and Huntington (Citation1986). The three ITC items were also taken from Cuskelly and Hoye (Citation2013). All items were presented on a 7-point Likert scale anchored by strongly disagree (1) and strongly agree (7). Composite variables were calculated for each of the constructs of interest by adding together item scores and dividing by the number of items.

Data analyses

To compare ECRs and SRs across the motivation dimensions, perceived organisational support and ITC, independent samples t-tests were used. To analyse relationships among the constructs of interest, a series of binary logistic regression analyses were deemed appropriate (Pallant, Citation2016). Analyses were initially conducted with full-form motivation dimensions, and then separate analyses were conducted for the onset and continuing subscales. Although measured on a 7-point scale, the ITC variable was not normally distributed and was therefore re-coded as binary. Cases with a mean score of less than six on the ITC composite variable were characterised as “less likely to continue” and six or more as “more likely to continue”.

In the first instance, univariate analyses were conducted to establish direct effects of the motivation dimensions and perceived organisational support on ITC. Variables with a p-value of less than 0.2 for their relationship with ITC in the univariate analyses were included in subsequent multivariable analyses (Bursac et al., Citation2008; Maldonado & Greenland, Citation1993; Vittinghoff et al., Citation2006). This purposeful selection ensured that any potential predictors were included in the regression analyses as independent variables. Subsequent multivariable analyses enabled the identification of dimensions driving ITC, as the dependent construct in the model, while controlling for others. The moderating effect of perceived organisational support on the motivation-ITC relationships were assessed by specifying an interaction term between the perceived organisational support and motivation variables.

Results

For ECRs, intrinsic motivation (M = 5.10, SD = 1.05) had the greatest mean score of the four dimensions, inclusive of both onset and continuing subscales (). This was the same for SRs (M = 5.03, SD = 1.02). For the onset motivation subscales, onset sport-related motivation had the greatest mean score of the onset motivation dimensions for ECRs (M = 4.96, SD = 1.15) and SRs (M = 4.93, SD = 1.20). Whereas for continuing motivation, continuing intrinsic motivation had the greatest mean score for ECRs (M = 5.28, SD = 1.06) and SRs (M = 5.32, SD = 1.12). ECRs (M = 5.08, SD = 1.18; M = 6.14, SD = 1.34) and SRs (M = 5.02, SD = 1.43; M = 6.12, SD = 1.33) reported relatively strong perceived organisational support and ITC. When comparing the groups, only two dimension-level motivational differences emerged. ECRs (M = 3.07, SD = 1.16) reported stronger extrinsic motivation than SRs (M = 2.81, SD = 1.11), t(307) = 2.01, p = .046. Similarly, ECRs (M = 3.87, SD = 1.36) reported higher continuing extrinsic motivation scores than SRs (M = 3.52, SD = 1.39), t(307) = 2.19, p = .030.

Table 3. Motivation mean scores and difference testing by tenure.

Motivation

Initially, univariate binary logistic regressions were undertaken to assess motivation dimensional and perceived organisational support direct effects with ITC (). Perceived organisational support had a positive relationship with ITC for ECRs (OR = 1.61, p = .010) but not for the SR group.

Table 4. Univariate motivation-ITC effects based on binary logistic regression.

The odds ratios reported in the following sections indicate how much more likely referees are to continue for every one unit increase of motivation on the seven-point scale. For ECRs, intrinsic motivation (OR = 1.91, p = .003), extrinsic motivation (OR = 1.57, p = .039), and sport-related motivation (OR = 1.70, p = .017) had a positive relationship with ITC. Furthermore, for the onset motivation subscales, intrinsic motivation (OR = 1.46, p = .033) and sport-related motivation (OR = 1.53, p = .030) had statistically significant relationships with ITC for ECRs. Whereas for the continuing motivation subscales, ECRs’ intrinsic motivation (OR = 2.18, p < .001), extrinsic motivation (OR = 1.57, p = .010), and social motivation (OR = 1.42, p = .035) all had positive relationships with ITC.

In terms of the tenure-based hypotheses (i.e. H1a, H2a, H3a and H4a), fewer motivational dimensions relate to ITC for the SR subset than the ECR subset. Intrinsic motivation (OR = 2.02, p < .001) and social motivation (OR = 1.36, p = .024) had positive relationships with ITC. Onset intrinsic motivation (OR = 1.52, p = .008) and onset social motivation (OR = 1.28, p = .045) also had positive relationships with ITC. Furthermore, continuing intrinsic motivation (OR = 2.11, p < .001) and continuing social motivation (OR = 1.30, p = .033) also had positive relationships with ITC for SRs. Therefore, results support H2a, H3a and H4a but not H1a.

Multivariable binary logistic regression was used to assess the effect of onset and continuing motivation on ITC while controlling for other motivation dimensions (). At this stage, motivation dimensions which did not meet the 0.2 threshold were not included and were shown as gaps in the table. For SRs (OR = 2.90, p < .001), intrinsic motivation had a statistically significant positive relationship with ITC after controlling for social motivation. Continuing intrinsic motivation had a statistically significant relationship with ITC for ECRs (OR = 2.16, p = .018) and SRs (OR = 2.63, p < .001), controlling for extrinsic motivation and social motivation. Prior to exploring moderating effects, binary logistic regression confirmed that age did not influence the continuation of referees. As the sample was majority male, gender-based exploration is not warranted. The level of rugby at which referees engaged was not included as many refereed at multiple levels, from youth through to more competitive matches.

Table 5. Multivariable motivation-ITC effects based on binary logistic regression.

Perceived organisational support moderation

Interaction terms were specified in the logistic regression models to explore the hypothesised moderating effect of perceived organisational support on motivation-ITC relationships (). Perceived organisational support did not moderate any motivation dimension-ITC relationships for ECRs. In contrast, significant perceived organisational support interactions did emerge for SRs. Perceived organisational support was found to interact with sport-related motivation (OR = 0.72, p = .010). Those SR subgroup results provide evidence in support of H3b.

Table 6. POS interaction effects on ITC based on binary logistic regression.

In summary, results provide reasonable support for H3c (perceived organisational support interacting with sport-related motivation) but not for H1b (intrinsic motivation), H2b (extrinsic motivation), and H4b (social motivation).

Discussion

An established four-dimensional conceptualisation of referee motivation formed the basis of this study, and ITC was the outcome of interest. Further evidence was also generated exploring referee perceptions of organisational support – specifically for its potentially moderating effect on the motivation dimension-ITC relationships. H3b was partially supported, but H1b, H2b and H4b were not supported. This leads to the conclusion that POS plays a nuanced role in referees’ intention to continue. Importantly, this study also introduced an explicit focus on both onset and continuing motivation for those four dimensions, as well as contrasting ECRs and SR. These analyses provided new insights.

Perception of organisational support

ECR-SR differences emerged on the direct relationship between perceived organisational support and ITC. Although the relationship was significant for the ECR sample, it was not for SRs. This was an interesting finding as most of the refereeing literature reports that perceived organisational support influences ITC for referees (Choi & Chiu, Citation2017; Giel & Breuer, Citation2020; Kim, Citation2017; Livingston et al., Citation2017; Livingston & Forbes, Citation2016). However, the only study which focused on ECRs as a subgroup also reported that perceived organisational support was not related to ITC (Cuskelly & Hoye, Citation2013), contrary to the findings of this study. The perceived organisational support mean scores and ITC direct effects reported here suggest that ECRs feel supported, which is positively associated with their ongoing involvement. This finding is consistent with OST (Eisenberger & Huntington, Citation1986; Rhoades & Eisenberger, Citation2002; Shanock et al., Citation2019). This highlights that referees were not a homogenous group, and that tenure is an important factor. We argue that there is much more to unpack as it relates to tenure-based differences in the referee experience. Subsequent analyses of perceived organisational support interaction effects alongside motivation dimensions, consideration of tenure and the continuing motivation subscale will reveal further insights about this.

Referee tenure

Contrasting subgroups by tenure has generated insights that contribute to our evolving understanding of organisational aspects of the referee experience. ECRs reported slightly higher mean scores for extrinsic motivation and continuing extrinsic motivation compared to SRs, suggesting that when starting, ECRs were more externally motivated than they might be later in their career. This may relate to thoughts early on about career advancement (Bernal et al., Citation2012). For ECRs, continuing extrinsic motivation was a significant predictor of ITC, while for SRs, they were not. A lack of recognition has been related to discontinuation has been reported elsewhere (Auger et al., Citation2010; Dosseville et al., Citation2013; Hancock et al., Citation2015; Livingston et al., Citation2017), but the new insight here is that the phenomenon is more pronounced for those new to refereeing. On the important relationships between motivation dimensions and ITC, three of four hypotheses were supported. Specifically, there were more significant relationships between motivation dimensions (including onset and continuing subsets) for ECRs than SRs, which highlight important tenure-based motivation distinctions. This finding provides further evidence of tenure-based distinction amongst referees.

Moderating effect of perceived organisational support

Another objective of the current research was to advance our knowledge of the role of perceived organisational support within the referee experience. There is a growing body of literature in the context of refereeing reporting that perceived organisational support directly affects ITC (e.g. Choi & Chiu, Citation2017; Livingston & Forbes, Citation2017), which was confirmed here. In addition, it has been argued that motivation alone is a weak predictor of ITC (Giel & Breuer, Citation2020; Hoye et al., Citation2008). Therefore, moving past perceived organisational support and motivational dimension direct effects on ITC, the focus moves to the potentially moderating role of perceived organisational support. Based on what is reported elsewhere and intuitive appeal, we hypothesised that perceived organisational support (i.e. a perception of an external aspect of the experience) would interact with dimensions of motivation together, shaping a referee’s stated likelihood to stay engaged. Underpinned by OST, which captures fairness, supervisor support and job conditions (Eisenberger & Huntington, Citation1986; Kurtessis et al., Citation2017; Rhoades & Eisenberger, Citation2002), it was argued that perceived organisational support would accentuate a referee’s motivation and subsequent ITC. The results from this study are in line with OST to the extent that perceptions of care, concern, support, and other elements of perceived organisational support were relevant in and of themselves, but also interact with motivation dimensions.

Overall, some support was found for the hypotheses explored in this study. For some of the sport-related motivation relationships with ITC, perceived organisational support was a significant moderator, indicating that a referee’s perception of organisational support plays a role. For SRs, the most noteworthy perceived organisational support interaction effect on ITC was with sport-related motivation. This finding suggests that sport-related motivation, such as a desire to give back, is important (Baldwin & Vallance, Citation2016; Ridinger, Citation2015) but that the driver does not exist in a vacuum. Rather, a referee’s perception of the supportiveness of the organisation to which they are connected facilitates a pathway for sport-related motivation to be realised. So, for example, while a referee may be driven by staying connected to a sport they have been involved with for a lifetime, they feel as though they need a supportive organisation in order to make it happen. No significant perceived organisational support interactions with motivation dimensions were found for the ECR subgroup. This finding was unexpected because it has been reported elsewhere that perceived organisational support declines over the tenure of a referee (Livingston & Forbes, Citation2016).

Theoretical and managerial implications

The aim of this research was to better understand how perceived organisational support relates to a referee’s motivation and ITC. This extension of OST as a theoretical frame into a sport context featuring a voluntary refereeing workforce is an important contribution to the sport management literature. In this study, the utility of OST has been established for voluntary refereeing within sport and paves the way for future researchers to embrace this theoretical framing. More specifically, results suggest that a perception of organisational support is quite nuanced in how it affects ECRs and SRs. Specifically, perceived organisational support has a moderating effect for SRs and a direct effect for ECRs. This nuance has both theoretical and managerial implications. One of Shanock et al.’s (Citation2019) OST principles is to enlist representatives of the organisation to convey organisational support and the perception of it, which is particularly important for a referee workforce deployed across a geographical area. Referee organisations should make every effort to get out and support referees in the field. This support could take the form of engaging with referees in a supportive manner before, during and after matches. While many sport managers attempt to do this and might claim that resources limit more focus on this, the results here suggest this would make a difference, particularly for those who have been refereeing for more than five years.

The results of this study translate to implications for sport managers who recruit and seek to retain referees. Building on a related body of literature (e.g. Choi & Chiu, Citation2017), this study provided evidence that perceived organisational support has a direct effect on ITC for new referees. This highlights how important it is for organisations to not only be supportive but also make sure that they are perceived that way. In the context of refereeing, the provision of education programmes, referee coaching, and recognition are all ways to show organisational support. However, being responsive to questions, available when things go wrong and supportive when referees face difficult situations are other things that can foster the perceived organisational support sentiment. That refereeing organisations should be supportive of referees is probably self-evident to most, but how organisations prioritise, and resource support should be reflected upon regularly. Furthermore, it would be beneficial for sport organisations to highlight the ways they are being supportive such that the referee community will be aware of that supportiveness, regardless of whether they are personally receiving it.

This study has provided new insights about referee tenure that those managing referees should consider. ECRs should be prioritised and require resources to ensure competence and confidence. However, SRs require support as well. The results here show that the organisation’s role is also important for SRs as it influences the sport-related motivation-ITC relationship. In many instances, “refereeing is highly dependent at the grassroots level on attracting and recruiting volunteers to become accredited and qualified and to maintain this involvement in the game over a number of years” (Baldwin & Vallance, Citation2016, p. 49). This mindset needs to shift, whereby support is also offered to SRs rather than focused solely on ECRs. Sometimes, it is assumed that based on experience and knowledge, SRs can look after themselves. The findings suggest that referee managers should uniquely tailor support for the longest-tenured referees in their community, as their motivation alone is unlikely to drive continued participation.

Limitations and future research

Like most of the body of refereeing literature, the primary limitation of this study is its cross-sectional design. Cross-sectional studies can be affected by social desirability and demand effect bias (Herrmann et al., Citation2016). While ITC is widely used as an outcome variable in the wider sport management literature (Cuskelly & Hoye, Citation2013; Kim, Citation2017; Orviz-Martínez et al., Citation2021), it is only a proxy for actual behaviour. Data were collected towards the end of the rugby union season, which was deemed most appropriate, but it is possible that heightened emotions led to inaccurate ITC responses.

A further limitation of this study was that as a result of our focus on relationships amongst constructs including ITC as the outcome, less focus was on generating insights related to gender and level of refereeing. Subsequent research exploring this model should explore whether demographic elements such as age, gender and levels refereed influence continuation.

It is also important to consider the context in which this study was undertaken, with rugby union referees from New Zealand. Due to the various ways sporting organisations support and resource referees, it is important to consider that these findings are not generalisable to other sports or countries. Furthermore, due to the small number of female referees in this study, caution should be taken to related generalisation.

Ethics approval

Ethics approval was granted by the Auckland University of Technology Ethics Committee on 19 August 2019 (19/280).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The authors would like to acknowledge the financial support of New Zealand Rugby on this project.

References

  • Ajzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behaviour. Prentice Hall.
  • Auger, D., Fortier, J., Thibault, A., Magny, D., & Gravelle, F. (2010). Characteristics and motivations of sports officials in the province of Québec. International Journal of Sport Management, Recreation and Tourism, 5, 29–50. https://doi.org/10.5199/ijsmart-1791-874X-5b
  • Baldwin, C. F., & Vallance, R. (2016). Community based rugby referees’ experiences with recruitment and retention. International Sports Studies, 38(1), 49–63. https://doi.org/10.30819/iss.38-1.05
  • Baran, B. E., Shanock, L. R., & Miller, L. R. (2012). Advancing organizational support theory into the twenty-first century world of work. Journal of Business and Psychology, 27(2), 123–147. https://doi.org/10.1007/s10869-011-9236-3
  • Bernal, J. C., Nix, C., & Boatwright, D. (2012). Sport officials’ longevity: Motivation and passion for the sport. International Journal of Sport Management, Recreation and Tourism, 10, 28–39. https://doi.org/10.5199/ijsmart-1791-874X-10b
  • Bursac, Z., Gauss, C. H., Williams, D. K., & Hosmer, D. W. (2008). Purposeful selection of variables in logistic regression. Source Code for Biology and Medicine, 3(1), 1–8. https://doi.org/10.1186/1751-0473-3-17
  • Caesens, G., & Stinglhamber, F. (2020). Toward a more nuanced view on organizational support theory. Frontiers in Psychology, 11, 10–12. https://doi.org/10.3389/fpsyg.2020.00476
  • Choi, H., & Chiu, W. (2017). Influence of the perceived organizational support, job satisfaction, and career commitment on football referees’ turnover intention. Journal of Physical Education and Sport, 17(3), 955–959. https://doi.org/10.7752/jpes.2017.s3146
  • Croasman, J. T., & Ostrom, L. (2011). Using Likert-type scales in the social sciences. Journal of Adult Education, 40(1), 19–22.
  • Cuskelly, G., & Hoye, R. (2004). Problems and issues in the recruitment and retention of sports officials. Australian Sports Commission.
  • Cuskelly, G., & Hoye, R. (2013). Sports officials’ intention to continue. Sport Management Review, 16(4), 451–464. https://doi.org/10.1016/j.smr.2013.01.003
  • Dawson, P., Webb, T., & Downward, P. (2021). Abuse is not a zero-sum game! The case for zero tolerance of match official physical and verbal abuse. European Journal of Sport Science, 417–424. https://doi.org/10.1080/17461391.2021.1881619
  • Dosseville, F., Rioult, F., & Laborde, S.. (2013). Why do sports officials dropout?. Machine learning and data mining for sports analytics, 1969, 1–10.
  • Eisenberger, R., & Huntington, R. (1986). Perceived organizational support. Journal of Applied Psychology, 71(3), 500–507. https://doi.org/10.1037/0021-9010.71.3.500
  • Forbes, S. L., & Livingston, L. A. (2013). Changing the call: Rethinking attrition and retention in the ice hockey officiating ranks. Sport in Society, 16(3), 295–309. https://doi.org/10.1080/17430437.2013.779854
  • Fowler, B., Smith, J., Nordstrom, H., & Ferguson, T. (2019). Ice hockey officiating retention: A qualitative understanding of junior ice hockey officials’ motivations in Canada. Managing Sport and Leisure, 18–31. https://doi.org/10.1080/23750472.2019.1565944
  • Giel, T., & Breuer, C. (2020). The determinants of the intention to continue voluntary football refereeing. Sport Management Review, 23(2), 242–255. https://doi.org/10.1016/j.smr.2019.01.005
  • Hancock, D. J., Dawson, D. J., & Auger, D. (2015). Why ref? Understanding sport officials’ motivations to begin, continue, and quit. Movement & Sport Sciences, 87(1), 31–39. https://doi.org/10.3917/sm.087.0031
  • Herrmann, J.-L., Kacha, M., & Derbaix, C. (2016). “I support your team, support me in turn!” The driving role of consumers’ affiliation with the sponsored entity in explaining behavioral effects of sport sponsorship leveraging activities. Journal of Business Research, 69(2), 604–612. https://doi.org/10.1016/j.jbusres.2015.05.016
  • Hong, E., Jeong, Y., & Downward, P. (2019). Perceived organizational support, internal motivation, and work–family conflict among soccer referees. Managing Sport and Leisure, 24(1–3), 141–154. https://doi.org/10.1080/23750472.2019.1593049
  • Hoye, R., Cuskelly, G., Taylor, T., & Darcy, S. (2008). Volunteer motives and retention in community sport: A study of Australian rugby clubs. Australian Journal on Volunteering, 13(2), 40–48.
  • Jacobs, B. L., Tingle, J. K., Oja, B. D., & Smith, M. A. (2020). Exploring referee abuse through the lens of the collegiate rugby coach. Sport Management Review, 23(1), 39–51. https://doi.org/10.1016/j.smr.2019.03.004
  • Jiranek, P., Kals, E., Humm, J. S., Strubel, I. T., & Wehner, T. (2013). Volunteering as a means to an equal end? The impact of a social justice function on intention to volunteer. Journal of Social Psychology, 153(5), 520–541. https://doi.org/10.1080/00224545.2013.768594
  • Johansen, B. T. (2015). Reasons for officiating soccer: The role of passion-based motivations among Norwegian elite and non-elite referees. Movement & Sport Sciences – Science & Motricité, 87, 23–30. https://doi.org/10.1051/sm/2014012
  • Jordan, P., Upright, P., & Forsythe, S. (2019). Rural Kentucky sport officials’ perspectives on recruitment, training and retention. KAHPERD Journal, 56(2), 59–73.
  • Joshi, A., Kale, S., Chandel, S., & Pal, D. (2015). Likert scale: Explored and explained. British Journal of Applied Science & Technology, 7(4), 396–403. https://doi.org/10.9734/BJAST/2015/14975
  • Kellett, P., & Shilbury, D. (2007). Umpire participation: Is abuse really the issue? Sport Management Review, 10(3), 209–229. https://doi.org/10.1016/S1441-3523(07)70012-8
  • Kellett, P., & Warner, S. (2011). Creating communities that lead to retention: The social worlds and communities of umpires. European Sport Management Quarterly, 11(5), 471–494. https://doi.org/10.1080/16184742.2011.624109
  • Kim, M., Kim, H. S., Simmonda, A., & Warner, S. (2022). Strengthening referees’ psychological well-being through engagement and authenticity. Sport Management Review, 25(2), 254–274. https://doi.org/10.1080/14413523.2021.1930952
  • Kim, M., Trail, G. T., Lim, J., & Kim, Y. K. (2009). The role of psychological contract in intention to continue volunteering. Journal of Sport Management, 23(5), 549–573. https://doi.org/10.1123/jsm.23.5.549
  • Kim, S. (2017). Perceived organizational support as a mediator between distributive justice and sports referees’ job satisfaction and career commitment. Annals of Leisure Research, 20(2), 169–187. https://doi.org/10.1080/11745398.2016.1147363
  • Kurtessis, J. N., Eisenberger, R., Ford, M. T., Buffardi, L. C., Stewart, K. A., & Adis, C. S. (2017). Perceived organizational support: A meta-analytic evaluation of organizational support theory. Journal of Management, 43(6), 1854–1884. https://doi.org/10.1177/0149206315575554
  • Livingston, L. A., & Forbes, S. (2017). Resilience, motivations for participation, and perceived organizational support amoungst aesthetic sports officials. Journal of Sport Behavior, 40(1), 43–67.
  • Livingston, L. A., & Forbes, S. L. (2016). Factors contributing to the retention of Canadian amateur sport officials: Motivations, perceived organizational support, and resilience. International Journal of Sports Science and Coaching, 11(3), 342–355. https://doi.org/10.1177/1747954116644061
  • Livingston, L. A., Forbes, S. L., Pearson, N., Camacho, T., Wattie, N., & Varian, P. (2017). Sport officiating recruitment, development, and retention: A call to action. Current Issues in Sport Science, 2, 557–558. https://doi.org/10.15203/CISS
  • MacMahon, C., Mascarenhas, D., Plessner, H., Pizzera, A., Oudejans, R., & Raab, M. (2014). Development of officials. In Sports officials and officiating: Science and practice (pp. 8–32). Routledge. https://doi.org/10.4324/9780203700525
  • Maldonado, G., & Greenland, S. (1993). Simulation study of confounder-selection strategies. American Journal of Epidemiology, 138(11), 923–936. https://doi.org/10.1093/oxfordjournals.aje.a116813
  • Orviz-Martínez, N., Botey-Fullat, M., & Arce-García, S. (2021). Analysis of burnout and psychosocial factors in grassroot football referees. International Journal of Environmental Research and Public Health, 18(3), 1111. https://doi.org/10.3390/ijerph18031111
  • Pallant, J. (2016). SPSS – survival manual (6th ed.). McGraw-Hill.
  • Parsons, T., & Bairner, A. (2015). You want the buzz of having done well in a game that wasn’t easy: A sociological examination of the job commitment of English football referees. Movement & Sport Sciences, 87(1), 41–52. https://doi.org/10.1051/sm/2014017
  • Perryer, C., Jordan, C., Firns, I., & Travaglione, A. (2010). Predicting turnover intentions: The interactive effects of organizational commitment and perceived organizational support. Management Research Review, 33(9), 911–923. https://doi.org/10.1108/01409171011070323
  • Rhoades, L., & Eisenberger, R. (2002). Perceived organizational support: A review of the literature. Journal of Applied Psychology, 87(4), 698–714. https://doi.org/10.1037/0021-9010.87.4.698
  • Ridinger, L. L. (2015). Contributors and constraints to involvement with youth sports officiating. Journal of Amateur Sport, 1(2), 103–127. https://doi.org/10.17161/jas.v1i2.4946
  • Ridinger, L. L., Kim, K. R., Warner, S., & Tingle, J. K. (2017). Development of the referee retention scale. Journal of Sport Management, 31(5), 514–527. https://doi.org/10.1123/jsm.2017-0065
  • Ridinger, L. L., Tingle, J. K., & Kim, K. R. (2017). Why referees stay in the game. Global Sport Business Journal, 5(3), 22–37.
  • Schaeperkoetter, C. C. (2017). Basketball officiating as a gendered arena: An autoethnography. Sport Management Review, 20(1), 128–141. https://doi.org/10.1016/j.smr.2016.05.001
  • Shanock, L. R., Eisenberger, R., Heggestad, E. D., Malone, G., Clark, L., Dunn, A. M., Kirkland, J., & Woznyj, H. (2019). Treating employees well: The value of organizational support theory in human resource management. Psychologist-Manager Journal, 22(3–4), 168–191. https://doi.org/10.1037/mgr0000088
  • Symonds, M. L., & Russell, W. (2018). Intrinsic and extrinsic motivation of small college sports officials. Journal of Sport Behavior, 41(2), 209.
  • Treglown, L., Zivkov, K., Zarola, A., & Furnham, A. (2018). Intention to quit and the role of dark personality and perceived organizational support: A moderation and mediation model. PLoS ONE, 13(3), e0195155. https://doi.org/10.1371/journal.pone.0195155
  • Vansteenkiste, M., Simons, J., Lens, W., Sheldon, K. M., & Deci, E. L. (2004). Motivating learning, performance, and persistence: The synergistic effects of intrinsic goal contents and autonomy-supportive contexts. Journal of Personality and Social Psychology, 87(2), 246–260. https://doi.org/10.1037/0022-3514.87.2.246
  • VanYperen, N. W. (1998). Predicting stay/leave behavior among volleyball referees. The Sport Psychologist, 12(4), 427–439. https://doi.org/10.1123/tsp.12.4.427
  • Vittinghoff, E., Glidden, D. V., Shiboski, S. C., & McCulloch, C. E. (2006). Regression methods in biostatistics: Linear, logistic, survival, and repeated measures models. Springer.
  • Wang, J.-W., Wei, C.-N., Harada, K., Minamoto, K., Ueda, K., Cui, H.-W., Zhang, C.-G., Cui, Z.-T., & Ueda, A. (2011). Applying the social cognitive perspective to volunteer intention in China: The mediating roles of self-efficacy and motivation. Health Promotion International, 26(2), 177–187. https://doi.org/10.1093/heapro/daq056
  • Warner, S., Tingle, J. K., & Kellett, P. (2013). Officiating attrition: The experiences of former referees via a sport development lens. Journal of Sport Management, 27(4), 316–328. https://doi.org/10.1123/jsm.27.4.316
  • Webb, T., Dicks, M., Thelwell, R., van der Kamp, J., & Rix-Lievre, G. (2019). An analysis of soccer referee experiences in France and The Netherlands: Abuse, conflict, and level of support. Sport Management Review, 23(1), 52–65. https://doi.org/10.1016/j.smr.2019.03.003
  • Webb, T., Rayner, M., & Thelwell, R. (2018). An explorative investigation of referee abuse in English rugby league. Journal of Applied Sport Management, 10(2), 14–25. https://doi.org/10.18666/JASM-2017-V10-I2-8834
  • Webb, T., Rayner, M., & Thelwell, R. (2019). An examination of match official’s perceptions of support and abuse in rugby union and cricket in England. Managing Sport and Leisure, 24(0), 155–172. https://doi.org/10.1080/23750472.2019.1605841
  • Wicker, P., & Frick, B. (2016). Recruitment and retention of referees in nonprofit sport organizations: The trickle-down effect of role models. Voluntas, 27(3), 1304–1322. https://doi.org/10.1007/s11266-016-9705-4
  • Zvosec, C. C., Oja, B. D., & Baer, N. (2021). Team zebra’: An exploration of social identity, motivation, and sport officiating persistence. Journal of Global Sport Management, 1–20. https://doi.org/10.1080/24704067.2021.1899766