Translation, cross-cultural adaptation, validity, reliability and stability of the Dutch Injury - Psychological Readiness to Return to Sport (I-PRRS-NL) scale

ABSTRACT

The Injury Psychological Readiness to Return to Sport (I-PRRS) scale measures the psychological readiness of injured athletes to resume sports participation. The aim of this study was to translate and culturally adapt the I-PRRS scale into Dutch (I-PRRS-NL) and assess its validity, reliability, and stability in patients after anterior cruciate ligament reconstruction (ACLR). The original I-PRRS was translated and culturally adapted from English into Dutch, and tested for clinimetric quality. To assess concurrent validity, 150 patients completed the I-PRRS-NL scale and five questionnaires measuring related constructs 3–16 months after ACLR. All predefined hypotheses regarding correlations between the I-PRRS-NL scale and these questionnaires were confirmed, indicating good concurrent validity. For test-retest reliability, 107 patients completed the I-PRRS-NL scale again two weeks later. The I-PRRS-NL scale showed good internal consistency (Cronbach’s alpha 0.94) and test-retest reliability (ICC 0.89). Standard error of measurement was 4.2 and smallest detectable change was 11.6. No systematic bias between test and retest was found. As the scale appears unstable at item level, only the total score should be used. Overall, the I-PRRS-NL scale showed sufficient validity and reliability to assess the psychological readiness to resume sports among Dutch-speaking patients after ACLR.

KEYWORDS:

• Injury
• patient-reported outcome measure (PROM)
• confidence
• psychology
• ACL reconstruction

Introduction

Involvement in sport activities, like running and soccer, has many health-related benefits, but also entails a risk of sustaining an injury. Anterior cruciate ligament (ACL) rupture is one of the most common sports-related injuries to the knee (Majewski, Susanne, & Klaus, 2006). An ACL rupture often requires surgical treatment (Myklebust & Bahr, 2005). Although return to sports participation is often the most important outcome for patients after ACL reconstruction (ACLR), research has primarily focused on physical factors by evaluating impairment-based outcomes as a measure of surgical success (Ardern, 2015; Walker, Thatcher, & Lavallee, 2007). Many athletes do not return to their preinjury skill level of sports participation despite excellent impairment-based outcomes such as restored knee laxity and muscle strength (Ardern, Taylor, Feller, & Webster, 2014b).

Psychological factors and their impact on recovery, rehabilitation, and return to sports participation have received more attention in recent years (Webster, Nagelli, Hewett, & Feller, 2018). Psychosocial factors like injured athletes’ thoughts, feelings, injury- and performance-related fears, confidence, and actions may influence rehabilitation outcomes (Forsdyke, Smith, Jones, & Gledhill, 2016). It has been recognised that physical and psychological readiness to resume sports after injury do not always coincide (Podlog & Eklund, 2007). This is why attention should be paid to both during the rehabilitation period in order to optimise the outcome.

To evaluate various emotions and psychological factors associated with injury, several questionnaires have been used in previous research (e.g., Gobbi & Francisco, 2006; Kori, Miller, & Todd, 1990; Morrey, Stuart, Smith, & Wiese-Bjornstal, 1999; Thomeé et al., 2008). Two questionnaires have been specifically developed to evaluate psychological factors associated with return to sports: The ACL-Return to Sport after Injury (ACL-RSI) scale and the Injury Psychological Readiness to Return to Sport (I-PRRS) scale. The ACL-RSI was developed specifically for patients after ACL injury (Webster, Feller, & Lambros, 2008). To provide the possibility to compare different patient groups with each other, Glazer developed a generic scale, the I-PRRS scale to assess the psychological readiness of injured athletes to resume sports participation, irrespective of the type of injury (Glazer, 2009). Content validity was established by using the Delphi method (Thomas, Nelson, & Slverman, 2005). Preliminary evidence was provided for the reliability and concurrent validity of this English-language version (Glazer, 2009). Translated versions of the I-PRRS in different languages are required for cross-cultural comparisons of outcome assessment. A valid and reliable version of the I-PRRS scale translated into Persian is available (Naghdi et al., 2016), but no Dutch version of the I-PRRS exists.

In order to use the I-PRRS with Dutch-speaking patients, a valid and reliable Dutch version is needed. Moreover, the smallest detectable change (SDC) and stability of each item of the questionnaire have never been investigated. Hence the aim of this study was to translate and cross-culturally adapt the I-PRRS scale into Dutch (I-PRRS-NL) and to study its psychometric properties in patients after ACLR.

Methods

The study consisted of two stages. First, the I-PRRS scale was translated into Dutch following a standardised procedure (Beaton, Bombardier, Guillemin, & Ferraz, 2000). Second, the translated version was tested for clinimetric properties. Questionnaires were sent by mail with an accompanying letter clarifying the purpose and procedure of the study, explaining that all data would be anonymised, and that return of the questionnaires was taken as consent to participate. The local Medical Ethical Committee judged the procedures employed in this study and waived further need for approval.

Translation

The questionnaire was translated according to the guidelines for cross-cultural adaptation of self-report measures by Beaton et al. (2000). The guidelines consists of five stages: (1) translation by two bilingual translators (native Dutch), (2) synthesis, (3) back translation by two bilingual translators (native English), (4) evaluation by a team of experts consisting of forward and backward translators, two human movement scientists/epidemiologists and a sports physiotherapist, and (5) pre-testing in a group of 15 patients who followed a rehabilitation program after ACLR at our outpatient clinic. Some patients had problems with the translation of Question 5: My confidence in the injured body part to handle to demands of the situation. The expert committee agreed that the sentence structure was unclear and made a minor adjustment. The authors can provide a detailed report of the translation process. For the final version of the I-PRRS-NL scale, see appendix A.

Participants and procedure

Based on a search of digital patient records of Martini Hospital Groningen and University Medical Center Groningen in the Netherlands, patients aged 18 and older who had undergone an ACLR between 1 January 2012 and 1 February 2013 were invited to participate in the study. Patients who were unable to understand written Dutch were excluded. Demographic characteristics were retrieved from the electronic patient records.

Validity

To determine validity, patients who had an ACLR 3–16 months previous to the start of the study were eligible to participate. This time interval was chosen because patients start to perform sport-specific exercises and focus increasingly on return to sport from about three months after ACLR (van Melick et al., 2016), and the expectation was that 16 months after reconstruction the patients had returned to their sport.

Six questionnaires were sent to a total of 241 eligible patients in order to assess concurrent validity: I-PRRS-NL scale, ACL – Return to Sport after Injury scale (ACL-RSI; Slagers, Reininga, & van den Akker-Scheek, 2017), Knee injury and Osteoarthritis Outcome Score (KOOS; de Groot, Favejee, Reijman, Verhaar, & Terwee, 2008), International Knee Documentation Committee score (IKDC; Haverkamp, 2006), Tampa Scale of Kinesiophobia (TSK; Vlaeyen, Kole-Snijders, Boeren, & van Eek, 1995) and the internal subscale of the Multidimensional Health Locus of Control scale (MHLC; Wallston, Stein, & Smith, 1994). To determine current level of work and sports activities, patients completed the Tegner score (Tegner & Lysholm, 1985). It took about 20 minutes to complete these questionnaires. Patients who did not respond after one week received a reminder by mail. Of the 241 patients approached, 150 (62%) returned the first mailing. Demographic characteristics of the participants are shown in Table 1. A non-response analysis, using Chi-square tests, Student’s t-tests and Mann-Whitney U tests (where appropriate) showed no significant differences in patient characteristics between responders and non-responders with regard to gender, age, height, body weight, body mass index (BMI) and time interval between surgery and mailing out the questionnaires.

Table 1. Demographic characteristics of participants and non-responders.

Characteristics	I-PRRS-NL sample (n = 150)	Non-responders (n = 91)	p-value
Age (years)	29.2 (10.8)	26.8 (9.2)	0.06
Time after reconstruction (months)	9.6 (4.0)	9.9 (3.9)	0.94
Gender			0.56
Male (%)	84 (56%)	54 (59%)
Female (%)	66 (44%)	37 (40%)
Height (cm)	178 (9.4)	178 (12)	0.65
Body weight (kg)	79 (17)	81 (16)	0.38
BMI (kg/m2)	24.8 (4.3)	25 (7.3)	0.25
Graft type			0.57
Semitendinosis/Gracilis (%)	143 (95%)	86 (95%)
Bone-tendon-bone (%)	7 (5%)	5 (5%)
Tegner activity level score	5.5 (2.2)	-

Age, time after reconstruction, height, body weight, BMI and Tegner activity level are given as mean (standard deviation). Other values are given as number of patients (%).

Test-retest reliability and stability

To determine test-retest reliability and stability, 122 respondents who underwent an ACLR at least five months before the start of the study received the I-PRRS-NL scale again after a two-week interval. This second mailing had a response from 107 patients (88%). Mean time between receiving the first and second questionnaire was 22.9 days (SD 7.9). Patients who underwent an ACLR 3–4 months before the start of the study were excluded as they were not likely to be in a stable physical condition. Since these patients start with sport-specific exercises during that phase of the rehabilitation, actual changes in status are very likely (van Grinsven, van Cingel, Holla, & van Loon, 2010). To confirm this assumption and to check whether the patients did not change with regard to the construct measured by the I-PRRS scale during the period between test and retest, a global rating of change question (GRC) was included in the second measurement (Kamper, Maher, & Mackay, 2009). The patients were asked if a change in confidence regarding sports resumption had occurred since the last assessment by scoring a 5-point Likert scale ranging from much more confident to much less confident. Participants who reported being much more confident or much less confident were excluded from the test–retest reliability and stability analysis.

Questionnaires

The I-PRRS scale is an easy-to-use, valid, and reliable questionnaire that was developed to assess psychological readiness of injured athletes to return to sports (Glazer, 2009). The questionnaire consists of six items that are scored on a 100-point scale. A score of 0, 50, or 100 indicates that the athlete had respectively little to no confidence, moderate confidence, or maximum confidence for an item. The sum of the scores from the six items divided by 10 result in the I-PRRS total score. A score of 60 indicates high confidence to return to sports, 40 moderate confidence and 20 low confidence.

The ACL-RSI scale is a valid and reliable questionnaire that measures athletes’ emotions, confidence in performance, and risk appraisal in relation to return to sports after ACL injury (Slagers et al., 2017; Webster et al., 2008). The ACL-RSI scale consists of 12 questions scored with an 11-point Numeric Rating Scale (NRS) in the form of boxes to be ticked from 0 to 100. The total score is calculated by taking a percentage of the total scores for the 12 questions. High scores correspond to a positive psychological response (Langford, Webster, & Feller, 2009). The Dutch version of the ACL-RSI scale has good validity (Slagers et al., 2017).

The KOOS is a commonly used instrument to assess symptoms and functional status of people with knee injuries. It contains 42 items divided into five subscales: pain, symptoms, activities of daily living, sports and recreational function, and knee-related quality of life. Questions are answered using a 5-point Likert scale. Total scores are calculated for each subscale, with a maximum score of 100 indicating no knee symptoms. The Dutch-language KOOS has shown good validity (de Groot et al., 2008).

The IKDC score is an 18-item instrument designed for a heterogeneous population of patients with knee complaints to measure pain, symptoms, function and sports activity. A sum score was calculated, then transformed to a 100-point scale, with 100 indicating no restrictions in daily and sports activities and an absence of symptoms. The IKDC score has been translated into Dutch and shows good validity (Haverkamp, 2006).

Fear of re-injury due to movement and physical activity can be assessed with the TSK, which comprises 17 statements scored on a 4-point Likert scale on the subjective experience of injury and physical activity. The sum of the items results in a score between 17 and 68, where 68 indicates a high level of fear (Kori et al., 1990). A valid and reliable Dutch version of the TSK is available (Vlaeyen et al., 1995). For this study the Dutch version, derived from a modified Swedish version for knee injuries was used (Kvist, Ek, Sporrstedt, & Good, 2005; Slagers et al., 2017).

The MHLC scale measures different dimensions of the health locus of control (HLC) construct, that is, people’s beliefs as to whether their health is determined by their behaviour or external forces (Halfens & Philipsen, 1988; Wallston, Wallston, Kaplan, & Maides, 1976). The internal HLC subscale assesses the extent to which a person believes that internal factors are responsible for her health/illness. In this study only the internal HLC subscale is used. We translated the MHLC-C scale into Dutch following the international guidelines (Beaton et al., 2000). Like in previous studies, for the current study the word “condition” was replaced by “knee problems” (Kvist et al., 2013; Slagers et al., 2017).

Statistical analyses

The sample size used followed the recommendations of the consensus-based standards for the selection of health status measurement instruments (COSMIN; Mokkink et al., 2010). COSMIN states that a sample size of at least 100 is considered adequate for studies on measurement properties, and a sample size of 50 for determining test-retest reliability. Data were analysed using IBM SPSS software, version 23.0 for Windows (IBM Corporation, Armonk, NY). Statistical significance was accepted at p ≤ 0.05. Descriptive statistics were used for patient characteristics and to display outcomes of questionnaires.

Validity

As no gold standard exists, validity of the I-PRRS-NL scale was expressed in terms of concurrent validity. Concurrent validity of the I-PRRS-NL scale was determined by evaluating its structural validity and by hypothesis-testing. According to the guidelines proposed by the COSMIN initiative (Mokkink et al., 2010), we formulated ten hypotheses about the magnitude of relationships between the I-PRRS-NL scale and the corresponding (sub)scales of the ACL-RSI scale, KOOS, IKDC score, TSK and MHLC scale (Table 3). If 75% or more of the arbitrarily set number of hypotheses are confirmed, the concurrent validity of the questionnaire is considered good (Terwee et al., 2007). To assess the structural validity an exploratory factor analysis was conducted (one-factor structure).

Table 2. Descriptive statistics of the various measurements and Spearman correlation between the I-PRRS-NL and other questionnaires in the validity study (N = 150).

	Mean	SD	Min.	Max.	r	p-value
I-PRRS	43.2	13.1	0	60
ACL-RSI	56.5	22.2	8.3	100	0.79	< 0.001
KOOS subscale “Pain”	84.1	15.6	22.2	100	0.38	< 0.001
KOOS subscale “Other symptoms”	75.9	16.5	32.1	100	0.36	< 0.001
KOOS subscale “Function in daily living”	91.1	14.8	0	100	0.30	< 0.001
KOOS subscale “Function in Sport and Recreation”	65.9	23.3	0	100	0.53	< 0.001
KOOS subscale “Knee-related Quality of life”	49.7	13.0	18.8	81.3	0.45	< 0.001
IKDC 2000	76.9	16	6.9	100	0.56	< 0.001
TSK	36.6	6.5	17	56	−0.40	< 0.001
MHLC-C	19.4	6.2	6	36	−0.08	0.369

I-PRRS, Injury-Psychological Readiness to Return to Sport scale; ACL-RSI, Anterior Cruciate Ligament – Return to Sports after Injury scale; IKDC, International Knee Documentation Committee Score; MHLC-C, Multidimensional Health Locus of Control, C form; TSK, Tampa Scale of Kinesiophobia; KOOS, Knee injury and Osteoarthritis Outcome Score

Table 3. Predefined hypotheses (all hypotheses were confirmed).

	Expected correlation	True correlation
- High correlation between I-PRRS-NL and ACL-RSI(NL)	> 0.6	0.79
- Moderate correlation between I-PRRS-NL and KOOS(NL) subscale “pain”	0.3–0.6	0.38
- Moderate correlation between I-PRRS-NL and KOOS(NL) subscale “or symptoms”	0.3–0.6	0.36
- Moderate correlation between I-PRRS-NL and KOOS(NL) subscale “Function in daily living”	0.3–0.6	0.30
- Moderate correlation between I-PRRS-NL and KOOS(NL) subscale “Function in Sport and Recreation”	0.3–0.6	0.53
- Moderate correlation between I-PRRS-NL and KOOS(NL) subscale “Knee related Quality of life”	0.3–0.6	0.45
- KOOS(NL) subscales “Function in Sport and Recreation” and “Knee-related Quality of life” have a stronger correlation with ACL RSI than the other three subscales
- Moderate correlation between I-PRRS-NL and IKDC 2000(NL)	0.3–0.6	0.56
- Moderate correlation between I-PRRS-NL and TSK(NL)	0.3–0.6	0.40
- Low correlation between I-PRRS-NL and MHLC-C(NL)	< 0.3	0.08

Spearman’s Rho was calculated between the total score of the I-PRRS-NL scale and the total score of other questionnaires. Correlation coefficients above 0.6, between 0.6 and 0.3, and below 0.3 are considered to be high, moderate, and low, respectively (Hinkle, Wiersma, & Jurs, 1998).

Reliability

Reliability was assessed in terms of internal consistency, test-retest reliability, and measurement error (Mokkink et al., 2010). Internal consistency of the I-PRRS-NL scale was assessed with Cronbach’s alpha using the data from the first administration of the I-PRRS-NL scale. In the quality criteria proposed by Terwee et al. (2007) it is suggested that internal consistency of health status questionnaires should be between 0.7 and 0.95.

For test-retest reliability, we calculated the intraclass correlation coefficient (ICC: 2-way random, type agreement) with corresponding 95% confidence intervals (CI) between the data from the first and second administration of the I-PRRS-NL scale. ICCs above 0.70 are generally considered to be good (Terwee et al., 2007). The standard error of measurement (SEM) was calculated by dividing the standard deviation of the mean difference of the test and retest measurements (SDdiff) by √2 (de Vet, Terwee, Knol, & Bouter, 2006). The smallest detectable changes (SDC) for the individual score and for the group were calculated according to Beaton et al. (2000) (SDCind = 1,96x√2xSEM; SDCgroup = SDCind/√n).

Additionally, to test for absolute agreement a Bland and Altman analysis was performed (Bland & Altman, 1986). The mean difference between the first and second administration of the I-PRRS-NL scale with a 95% CI was calculated. Zero lying within 95% CI of the mean difference was considered a criterion for absolute agreement. Hence when zero lies outside the 95% CI, a measurement bias is indicated (Bland & Altman, 1986). The 95% limits of agreement were also calculated. Limits of agreement are formulated as the mean difference ±1.96 x SDdiff (Bland & Altman, 1986).

Stability

Stability was assessed in terms of proportion of agreement on test and retest, calculated for each item of the questionnaire separately (Nevill, Lane, & Duncan, 2015; Nevill, Lane, Kilgour, Bowes, & Whyte, 2001; Wilson & Batterham, 1999). The proportion of participants with differences within a reference value of ± 10 (not considered to be a clinically relevant change) were also assessed. Nevill et al. (2001) argue that most participants (90%) should record differences within this value. Additionally, frequency histograms were constructed and 2.5% and 97.5% percentiles were calculated.

Floor and ceiling effects

The validity and reliability of a questionnaire may be jeopardised if floor or ceiling effects are present. It is then likely that extreme items will be missing in the lower or upper ends of the questionnaire, so respondents with the lowest or highest possible score cannot be distinguished from each other and changes in these patients cannot be measured (Terwee et al., 2007). Because the I-PRRS scale consists of six items that are scored on a 100-point scale, floor effects were evaluated when scoring on six items between 0–5 (total score 0–3) and ceiling effects when scoring on six items between 95–100 (total score 57–60). Floor and ceiling effects are defined as 15% of the participants achieving the minimum or maximum score, respectively (McHorney & Tarlov, 1995).

Results

Validity

An overview of the scores on the various questionnaires is shown in Table 2. The one-factor structure of the I-PRRS-NL scale showed an explained variance of 76% and an eigenvalue of 4.6. The Spearman correlation coefficients between the I-PRRS-NL scale and the ACL-RSI scale, KOOS subscales, IKDC score, TSK and MHLC-C scale are shown in Table 2. The I-PRRS-NL scale showed a high correlation with the ACL-RSI scale (r = 0.79) and a moderate correlation with the five subscales of the KOOS (r = 0.30 - 0.53). The I-PRRS-NL scale had a stronger correlation with the KOOS subscales “sports and recreational function” (r = 0.53) and “knee-related quality of life” (r = 0.45) than with the other KOOS subscales. Moderate correlation was found with the TSK (r = −0.40) and a low correlation with the MHLC-C scale (r = −0.08). All of the predefined hypotheses to determine concurrent validity were confirmed (Table 3).

Reliability and stability

The Cronbach alpha of I-PRRS-NL scale was 0.94, indicating good internal consistency. The results of the test-retest reliability analyses are presented in Table 4. Of the 107 participants who returned the second administration of the I-PRRS-NL, three (2%) reported to have much more confidence and were excluded from the test-retest reliability analysis, leaving 104 participants. The mean score for the I-PRRS-NL scale was 43.9 (SD 13.0) at the first measurement and 43.9 (SD 11.7) at the second measurement. The result of the Bland and Altman analysis is shown in Figure 1; the mean difference was 0.08 with the 95% CI of the mean difference containing zero (95% CI −1.04–0.88; SD 5.9), indicating no systematic bias. The ICC was 0.89 (95% CI 0.84–0.92), indicating excellent test-retest reliability. SEM was 4.2, SDCind was 11.6 and SDCgroup was 1.1.

Table 4. Test-retest reliability measures of the I-PRRS-NL scale (N = 104).

I-PRRS-NL First administration (SD) I-PRRS-NL Second administration (SD) Mean difference (95% CI; SD) LOA ICC (95% CI) SEM SDC – individual SDC – group

43.9 (13.0) 43.9 (11.7) 0.08 (−1.04–0.88; 5.9) -11.48–11.64 0.89 (0.84–0.92) 4.2 11.6 1.1

Abbreviations: SD = standard deviation; CI = confidence interval; LOA = limits of agreement; ICC = intraclass correlation coefficient; SEM = standard error of measurement; SDC = smallest detectable change

Figure 1. Bland and Altman graph of the test and retest.

Mean difference 0.08; 95% CI −1.04–0.88; containing zero

The proportion of agreement and the frequency distributions of the six items within the scale is illustrated in Table 5. The 2.5^th and 97.5^th percentiles, the proportion of participants with differences within ± 10 points (as measure of stability), and the proportion of agreement for all six items are given in Table 6. All items fail to demonstrate stability as less than 90% of participants’ differences from test to retest were ≤± 10. Percentages ranged between 67.4% (item 4) and 85.1% (item 6).

Table 5. Frequency distribution of test-retest differences (within-individuals) for items 1–6 of the I-PRRS.

Differences between test 2 and test 1	Frequency (%)
	Item 1	Item 2	Item 3	Item 4	Item 5	Item 6
−81/-90	1 (1.0)	0	0	0	0	0
−71/-80	0	0	0	0	0
−61/-70	0	1 (1.0)	0	0	0	0
−51/-60	1 (1.0)	0	0	0	1 (1.0)	1 (1.0)
−41/-50	0	3 (2.9)	1 (1.0)	1 (1.0)	0	3 (2.9)
−31/-40	0	2 (1.9)	1 (1.0)	2 (1.9)	1 (1.0)	1 (1.0)
−21/-30	2 (1.9)	1 (1.0)	2 (1.9)	3 (2.9)	3 (2.9)	2 (1.9)
−11/-20	9 (8.7)	13 (12.5)	7 (6.7)	10 (9.7)	9 (8.7)	4 (3.8)
−1/-10	28 (27.0)	22 (21.2)	11 (10.6)	13 (12.6)	24 (23.4)	25 (24.9)
0	39 (37.5)	36 (34.6)	44 (42.3)	36 (34.6)	33 (32)	42 (40.8)
1/10	15 (14.5)	18 (17.3)	23 (22.1)	21 (20.2)	17 (16.5)	20 (19.4)
11/20	5 (4.8)	4 (3.8)	5 (4.8)	7 (6.8)	8 (7.7)	3 (2.9)
21/30	3 (2.9)	1 (1.0)	3 (2.9)	3 (2.9)	3 (2.9)	0
31/40	0	1 (1.0)	1 (1.0)	3 (2.9)	2 (1.9)	0
41/50	1 (1.0)	2 (1.9)	3 (2.9)	3 (2.9)	1 (1.0)	1 (1.0)
51/60	0	0	2 (1.9)	1 (1.0)	1 (1.0)	0
61/70	0	0	1 (.0)	0	0	0
71/80	0	0	0	0	0	1 (1.0)
81/90	0	0	0	0	0	0
91/100	0	0	0	1 (1.0)	0	0

Difference = test 2 – test 1 (within-individuals). Frequency distribution is given as number of participants (%)

Table 6. 2.5th and 97.5th percentiles, proportion of participants with test-retest differences lower than or equal to ± 10 (PA ± 10), and proportion of agreement (PA) for the six items of the I-PRRS.

Item	Percentiles
	2.5%	97.5%	PA ± 10 (%)	PA (%)
1	−41	30	82 (79.0)	39 (37.5)
2	−50	44	76 (73.1)	36 (34.6)
3	−32	60	78 (75.0)	44 (42.3
4	−37	54	70 (67.4)	36 (34.6)
5	−34	44	74 (71.9)	33 (32)
6	−50	32	87 (85.1)	42 (40.8)

Floor and ceiling effects

No significant floor and ceiling effects were found. One patient (0.7%) scored below 3 (six questions 0–5) and 17 patients (11.5%) scored 57 or higher (six questions 95–100).

Discussion

The aim of this study was to translate the I-PRRS scale into Dutch (I-PRRS-NL scale) and to study its validity and reliability in order to investigate psychological factors relevant to resuming sports after ACL reconstruction among Dutch-speaking patients. The results indicate good validity and reliability for the I-PRRS-NL scale.

Validity

The I-PRRS consists of only six items. The scale is considered unidimensional because factors with less than five items are generally considered weak and unstable (Costello & Osborne, 2005). The exploratory factor analysis supported the one-factor structure, so the scale cannot be separated. These outcomes were comparable with the original English-language I-PRRS (Glazer, 2009).

Concurrent validity was assessed by correlating the results of the I-PRRS-NL scale with another scale measuring psychological factors associated with return to sports (ACL-RSI scale), patients’ self-reported knee function (KOOS and IKDC score), fear of re-injury (TSK) and internal health locus of control (MHCL-C scale). To our knowledge, these questionnaires have not been used before to test hypotheses on concurrent validity of the I-PRRS scale. Our predefined hypotheses were therefore primarily based on previous research comparing these questionnaires with the ACL-RSI scale. We found a high correlation between the I-PRRS scale and the ACL-RSI scale. This met our expectations, as both questionnaires measure a comparable construct. However, there are some differences. The I-PRRS scale is developed for injured athletes in general, with all questions focusing on confidence on sports resumption; the ACL-RSI scale is specifically developed for patients after ACLR, and besides the questions about emotions and risk appraisal in relation to sports resumption only five out of 12 items are about general sports confidence or knee-related confidence in performance.

The expected and found moderate correlations between the I-PRRS scale and the KOOS subscales and IKDC score are comparable with previous validity studies of the ACL-RSI scale (Kvist et al., 2013; Slagers et al., 2017). Also, both studies showed higher correlations with the KOOS subscales “sports and recreational function” and “knee-related quality of life” than with the other KOOS subscales.

The TSK was initially developed to measure pain-related fear of movement and re-injury, and is often used with patients who have chronic musculoskeletal pain (Kori et al., 1990). Fear of re-injury has been linked to decreased physical performance and contributes to reluctance to engage in activities that may lead to re-injury (Kori et al., 1990; Vlaeyen et al., 1995). Greater fear of reinjury has been shown to lead to a decrease in athletes’ return to sports (Kvist et al., 2005; Tripp, Stanish, Ebel-Lam, Brewer, & Birchard, 2007). However, the I-PRRS scale has no items about fear of re-injury, therefore a moderate correlation between the I-PRRS scale and the TSK was hypothesised and confirmed by our data.

Additionally, in accordance with our hypothesis, which was based on previous studies showing a low correlation between the ACL-RSI and MHLC-C scales (Kvist et al., 2013; Slagers et al., 2017), we found a low correlation between the I-PRRS-NL and MHLC-C scales. This is in line with research showing that the MHLC-C scale fails to predict rehabilitation outcome (Murphy, Foreman, Simpson, Molloy, & Molloy, 1999) Furthermore, the MHLC-C scale is not associated with returning to pre-injury recreational sports activity (Ardern et al., 2014a). In conclusion, all tested pre-defined hypotheses were confirmed. As this was above the 75% criterion set in the COSMIN guidelines, it can be concluded that the I-PRRS-NL scale has good concurrent validity (Terwee et al., 2007).

Reliability and stability

Internal consistency of the I-PRRS-NL scale was good, which is consistent with the original English and the Persian I-PRRS scales (Glazer, 2009; Naghdi et al., 2016). Test-retest reliability of the I-PRRS-NL scale was good, with an ICC of 0.89 and also comparable to previous research into the Persian version of the I-PRRS scale (Naghdi et al., 2016). The SEM was low (4.2), as were the SDC at the individual level (11.6) and especially at the group level (1.1). As the SEM and SDC of the I-PRRS scale have not been previously investigated, no comparison with literature could be made. However, the low SDC at the group level strongly indicates that the I-PRRS-NL scale is useful for group comparisons. Still, SEM and SDC are distribution-based measures that provide no information on which change is considered important by ACLR patients. To determine whether the SEM and SDC values are acceptable depends on which changes are minimally important on the I-PRRS-NL scale. SDC values should be smaller than the minimal important change (MIC) to distinguish clinically important changes from measurement errors (Terwee et al., 2007). The MIC of the I-PRRS scale has not yet been determined, hence further research is needed to evaluate responsiveness (sensitivity to change) and MIC of the I-PRRS-NL scale.

The unstable behaviour of all six items of the I-PRRS-NL represents a point of concern. It is important that each item shows equal stability. Unstable items could lead to a misleading interpretation. One explanation for the unstable items is that the measured construct actually changed in the 2-week time period but that this was not detected with the GRC question (only 3 participants reported much more confidence and were thus excluded). The stability of the items in the I-PRRS has not been investigated before. Actually, it should be included in the development phase of a questionnaire. The presence of unstable items implies we can only use the I-PRRS total score, as it showed good test-retest reliability.

No significant floor or ceiling effects were observed in this study, since less than 15% of the respondents achieved the lowest or highest possible total score. This is in agreement with the Persian version of the I-PRRS scale (Naghdi et al., 2016).

Strengths and limitations

A strength of this study is that participants filled out the questionnaires during the period of rehabilitation when return to sports was a key issue. Furthermore, a strength of this study is that a comparable number of women and men were included. In previous I-PRRS scale validity studies participants were mostly men. Other sports injury outcome studies and return-to-sports literature also show gender-related bias, limiting generalisability of findings (Forsdyke et al., 2016). Males and females may have different physical and psychological responses to injury, which can lead to different injury experiences and outcomes (Costello, Bieuzen, & Bleakley, 2014). It has been suggested that men are often praised for playing through pain and injury and thus may hide or minimise their symptoms in an effort to return to sport sooner, whereas women are typically more concerned about long-term implications of the athletic injury and their future health (Covassin & Elbin, 2011; Granito, 2002).

A limitation of this study is that we only included participants who had undergone ACLR, while the I-PRRS scale is developed for injured athletes in general. To develop a scale that measures the psychological readiness of all injured athletes to return to sports participation, items that were too sport- or environment-specific or were not appropriate for all returning athletes, were eliminated (Glazer, 2009). Other studies have shown the validity and reliability of the I-PRRS scale in a heterogeneous group of patients (Glazer, 2009; Naghdi et al., 2016). Another limitation is the lack of information about validity and reliability of the Dutch-language version of the MHLC-C scale. The process of translating the valid and reliable original MHLC-C scale following the international guidelines, without a validity study to evaluate the translation of the I-PRRS is less reliable.

Responsiveness of the I-PRRS scale has not been examined in this or in previous studies. Further research is needed to obtain information about the ability of this questionnaire to detect clinically important changes over time.

Conclusion

The I-PRRS was successfully translated into Dutch, the I-PRRS-NL scale. The I-PRRS-NL scale showed sufficient validity and reliability to assess the psychological readiness to resume sports among Dutch-speaking patients after ACLR. Because of the presence of unstable behaviour in all six items of the scale, only the total score should be used. The I-PRRS can be used in cross sectional research and in clinical practice as screening instrument to identify patients who are at risk of not returning to the sport.

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplementary Material

Supplementary data for this article can be accessed here.