Measuring exercise adherence in patients with low back pain: development, validity, and reliability of the EXercise Adherence Scale (EXAS)

ABSTRACT

Objectives

To develop an instrument to measure adherence to frequency, intensity, and quality of performance of home-based exercise (HBE) programs recommended by a physical therapist and to evaluate its construct validity and reliability in patients with low back pain.

Methods

The Exercise Adherence Scale (EXAS) was developed following a literature search, an expert panel review, and a pilot test. The construct validity of the EXAS was determined based on data from 27 participants through an investigation of the convergent validity between adherence, lack of time to exercise, and lack of motivation to exercise. Associations between adherence, pain, and disability were determined to test divergent validity. The reliability of the EXAS quality of performance score was assessed using video recordings from 50 participants performing four exercises.

Results

Correlations between the EXAS and lack of time to exercise, lack of motivation to exercise, pain, and disability were rho = 0.47, rho = 0.48, rho = 0.005, and rho = 0.24, respectively. The intrarater reliability of the quality of performance score was Kappa quadratic weights (K_qw) = 0.87 (95%-CI 0.83–0.92). The interrater reliability was K_qw = 0.36 (95%-CI 0.27–0.45).

Conclusions

The EXAS demonstrates acceptable construct validity for the measurement of adherence to HBE programs. Additionally, the EXAS shows excellent intrarater reliability and poor interrater reliability for the quality of performance score and is the first instrument to measure adherence to frequency, intensity, and quality of performance of HBE programs. The EXAS allows researchers and clinicians to better investigate the effects of adherence to HBE programs on the outcomes of interventions and treatments.

KEYWORDS:

• Exercise adherence
• low back pain
• validity
• reliability
• measurement instrument

Background

Low back pain (LBP) is a major health problem affecting an estimated 576,989,000 (95% confidence interval: 518,940,400 to 637,177,900) people globally in 2017 (James et al., 2018). LBP has been the leading cause of disability in patients with musculoskeletal disorders since 1990, and its global prevalence has continued to increase (James et al., 2018). From 2012 to 2014, the total aggregate medical costs for spine-related problems were an estimated 315.4 USD billion in the United States of America alone (United States Bone and Joint Initiative, 2020). The impact of LBP on patient functioning and the economic burden on society call for effective treatments (Buchbinder et al., 2018).

Previous research has shown that exercise therapy is effective in reducing pain intensity and disability in patients with LBP and is cost-effective when combined with stratified care based on risk prognosis (Gordon and Bloxham, 2016; Hill et al., 2011). These exercise therapy interventions often require patients to adhere to a home-based exercise (HBE) program. Adherence to an HBE program is defined as the extent to which a person’s exercise behavior corresponds with agreed recommendations by a health-care professional (World Health Organization, 2003). These recommendations pertain to frequency (i.e. number of exercise sessions per day or week); intensity (i.e. number of repetitions per exercise session); and quality of performance of the HBE program. Furthermore, in this study, an HBE program is defined as a specific exercise or set of specific exercises recommended by a health-care professional to be completed at home to improve impairments in body functions (e.g. joint mobility, muscle strength, or joint stability). Although HBE programs have been shown to be effective, adherence in patients with LBP varies from approximately 70% to 90% and declines significantly over time (Hammer, Degerfeldt, and Denison, 2007; Kolt and McEvoy, 2003). Additionally, adherence is difficult to assess due to the high rate of socially desirable answers provided by patients using diaries to record adherence, as well as the lack of a clinimetrically tested, standardized measure of exercise adherence (Bollen et al., 2014; Frost et al., 2017; McLean et al., 2017; Stone et al., 2003). As a result, the treatment effects of HBE programs on LBP can be underestimated due to poor adherence rates in both research and clinical practice. To better investigate the effects of patient adherence to HBE programs on treatment outcomes, researchers require a reliable and valid measure of adherence (Bollen et al., 2014; Frost et al., 2017). Additionally, a reliable and valid measure of adherence will allow clinicians to optimize patient adherence to HBE programs and improve treatment outcomes by tailoring treatments to individual patients. For example, strategies to increase self-efficacy, guidance, or exercise attention can be employed to improve low adherence to HBE programs (Bachmann, Oesch, and Bachmann, 2018; Picha and Howell, 2018).

Current measures of adherence to HBE programs employ a variety of strategies to measure adherence behavior (Bollen et al., 2014; Frost et al., 2017; Uzawa and Davis, 2018). Bollen et al. (2014) found 29 questionnaires, 29 diaries, two visual analog scales, and a tally counter. Most of these instruments had been used in only one study and lacked clinimetric testing, emphasizing the absence of a reliable, valid, and standardized means to measure adherence behavior (Bollen et al., 2014). Moreover, the existing instruments focus mainly on adherence to frequency and intensity recommendations of HBE programs (Uzawa and Davis, 2018). However, based on findings in patients with osteoarthritis treated by a physical therapist, quality of performance is an important factor in the treatment effects of HBE programs (Pisters et al., 2010). Patients may perform exercises in the exact frequency and intensity recommended by their physical therapist, but if the quality of performance is lacking, the intended effect of the exercise (e.g. muscle strengthening) is far less likely to be achieved. Poor quality of performance of exercises can be especially problematic when trying to assess the effectiveness of HBE programs for the treatment of patients with LBP in both clinical practice and research environments.

Unfortunately, there is currently no instrument that can measure adherence to frequency, intensity, and quality of performance recommendations of HBE programs (Bollen et al., 2014; Frost et al., 2017; Uzawa and Davis, 2018). Therefore, the aims of the current study are to develop an instrument to measure adherence to frequency, intensity, and quality of performance of HBE programs recommended by a health-care professional and to evaluate its construct validity and reliability.

Methods

Development

This study was performed in primary care physical therapy practices in the Netherlands. In developing the Exercise Adherence Scale (EXAS), the goal was to create an instrument to be used during face-to-face treatment sessions by a physical therapist or other health-care professionals to record HBE recommendations and patient-reported adherence to HBE recommendations. Furthermore, an observational component for assessing patients’ quality of performance of HBEs was to be included. The resulting instrument measures patient adherence to HBE recommendations from a physical therapist on intensity, frequency, and quality of performance.

The instrument was developed using a three-step process consisting of a literature search to create items, a face and content validity check by an expert panel, and a pilot test involving a small sample of patients. In the first step, the literature was searched for studies reporting on adherence to HBE programs, and the tools used to quantify adherence were extracted where possible. The studies found used primarily patient diaries or short questionnaires aimed at quantifying adherence to intensity and frequency recommendations of HBE programs, such as the Sport Injury Rehabilitation Adherence Scale (Brewer et al., 2000). None of the studies found reported on the quality of performance. Based on these findings, the authors created a first draft of the EXAS with a quality of performance component.

In the second step, an expert panel comprising researchers from the fields of health-related measurement instrument creation, LBP, and adherence was created. The expert panel provided feedback on the relevance and wording of the EXAS and suggested additions where needed in a two-round iterative process, thereby further refining the instrument. In the last step, five physical therapists pilot-tested the EXAS in patients with LBP to ensure that the questions were comprehensible and unambiguous. Based on feedback from the physical therapists and their patients, the final version of the EXAS was produced.

The final version of the EXAS is an interview-based instrument with an observational component, completed by the physical therapist together with the patient during each of the patient’s visits (Supplemental File). During the patient’s first treatment session, the physical therapist records the recommendations for the HBE program (i.e. type of exercises, frequency, and intensity) and shares them with the patient. During the patient’s follow-up visits, the physical therapist uses the EXAS to record the frequency and intensity of HBE performance as reported by the patient in a standardized format. Additionally, the physical therapist asks the patient to perform the exercises and rates the quality of performance on a 5-point scale (i.e. poor, moderate, reasonable, good, and excellent). The EXAS contains a qualitative description for the “poor,” “reasonable,” and “excellent” categories to facilitate the rating process (Table 1). Based on the experiences of the physical therapists in the pilot test, completing the EXAS requires approximately five minutes.

Table 1. Quality of performance score matrix

Excellent	Good	Reasonable	Moderate	Poor
Score 1	Score 0.8	Score 0.6	Score 0.4	Score 0.2
All parts of the home-based exercise are performed perfectly according to the recommendations by the therapist. There is no room for improvement. It is certain the desired effect of the exercise has been achieved.		Most parts of the exercise are performed well according to the recommendations by the therapist. Important parts of the exercise can be improved. The desired effect of the exercise is likely to have been achieved.		The majority or all of the parts of the exercise are not performed according to the recommendations by the therapist. It is very unlikely that the desired effect of the exercise has been achieved.

The EXAS score for the HBE program is calculated in three steps. In step one, the ratio between the frequency and intensity of HBE performance reported by the patient and the corresponding recommendations from the physical therapist is calculated for each exercise and multiplied by 100 to determine the adherence rate (1). If the patient-reported performance of frequency and intensity exceeds therapist recommendations, an adherence rate of 100% is scored instead.

$\begin{array}{r}\mathrm{A}\mathrm{d}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{e}\end{array}=\text{break}\frac{Numberofdays\ast numberoftimesperday\ast sets\ast repetitions\text{break}reportedbythepatient}{Numberofdays\ast numberoftimesperday\ast sets\ast repetitions\text{break}recommendedbythetherapist}$

In step two, the quality of performance score is used to calculate the adherence score for the individual exercise. To obtain the adherence score, the adherence rate for the individual exercise is multiplied by the quality of performance score for the individual exercise (2).

$\mathrm{A}\mathrm{d}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=\mathrm{A}\mathrm{d}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{e}\ast qualityofperformancescore$

The quality of performance score depends on the physical therapist’s rating of the patient’s quality of performance of each exercise. Currently, there is no theoretical basis for the impact of the quality of performance on the effectiveness of adherence to HBE recommendations. Therefore, the authors used their clinical experience and experience with instrument development to determine the magnitude of the impact of the quality of performance. In this study, each point on the quality of performance scale reflects 20% effectiveness (Table 1).

In the third and final step, the EXAS score is obtained by calculating the mean of the adherence scores for all individual exercises in the HBE program (3).

$\mathrm{E}\mathrm{X}\mathrm{A}\mathrm{S}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=\text{break}\frac{\mathrm{A}\mathrm{d}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}\mathrm{e}\mathrm{x}\mathrm{e}\mathrm{r}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{e}1+\text{break}\dots +\mathrm{A}\mathrm{d}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}\mathrm{e}\mathrm{x}\mathrm{e}\mathrm{r}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{e}n}{n}$

In the clinimetric study, the construct validity and reliability of the EXAS were investigated. Intrarater reliability was assessed only for the quality of the performance rating scale of the EXAS. For both the construct validity and reliability assessments of the EXAS, the physical therapists using the instrument were provided information on the theoretical background of adherence to HBE programs, in addition to receiving training in scoring the EXAS and incorporating the EXAS in clinical practice. Training involved completing the EXAS using data from a test patient and discussing the process with one of the researchers (RA or RG).

Construct validity

Construct validity refers to the extent to which scores obtained with a given measurement instrument relate to scores obtained with other instruments in a manner that is consistent with theoretically derived hypotheses, assuming the measurement instrument validly measures the construct of interest (Mokkink et al., 2010).

Currently, there is no gold standard for the measurement of adherence to HBE recommendations. Therefore, construct validity was determined by testing convergent and divergent validity using four theoretical hypotheses. Convergent validity is the degree to which a measure correlates with other measures to which it is similar (Frost et al., 2007). Discriminant (divergent) validity is the degree to which a measure does not correlate with (diverges from) measures that are dissimilar (Frost et al., 2007).

The factor “Barriers” has been found to be the strongest indicator of nonadherence to HBE programs in Dutch patients with LBP (Sluijs, Kok, and van der Zee, 1993). Lack of time to exercise and lack of motivation to exercise were among the barriers reported most frequently by patients who did not adhere to HBE recommendations and were chosen for hypothesis testing of convergent validity.

Essery, Geraghty, Kirby, and Yardley (2017) reviewed the literature on predictors of adherence to home-based physical therapies and found results for highly varied samples. They found that associations between adherence and a variety of possible predictors of adherence ranged mostly from no association to approximately r = 0.50. Therefore, the associations between perceived barriers and adherence were expected to be moderate (r = 0.30 to r = 0.50).

Pain and disability were reported as factors by both adherent and nonadherent patients. Therefore, both pain and disability were expected to be unrelated to adherence to HBE recommendations and were chosen to test hypotheses of divergent validity (Sluijs, Kok, and van der Zee, 1993). Consequently, the correlations between adherence to HBE recommendations, pain, and disability were expected to be low (r = 0.00 to r = 0.30).

The resulting hypotheses to be tested were as follows: 1) The association between lack of time to exercise and the EXAS is between r = 0.30 and r = 0.50; 2) The association between lack of motivation to exercise and the EXAS is between r = 0.30 and r = 0.50; 3) The association between pain and the EXAS is between r = 0.00 and r = 0.30; and 4) The association between disability and the EXAS is between r = 0.00 and r = 0.30

Participants and setting

For the validity study, 16 physical therapy primary care practices with 42 physical therapists participated and agreed to recruit patients with LBP according to the following inclusion criteria: the first visit to a physical therapist for the current episode of LBP as the primary complaint, current episode of LBP lasting more than four weeks at the first visit to a physical therapist, between the ages of 20 and 65, and having sufficient command of the Dutch language to read and understand questionnaires and spoken or written instructions. Patients were excluded if they had previously been diagnosed with lumbar radiculopathy, spinal osteoarthritis, or other conditions as the cause of their LBP or if they were unable to perform exercises due to physical or mental issues.

Measurements

Adherence to HBE

Recommendations were measured with the newly developed EXAS. The EXAS score was calculated using the previously stated assumption of 20% effectiveness for each point on the quality of performance scale.

Barriers

The barriers “lack of time to exercise” and “lack of motivation to exercise” were measured using single-item Likert scales based on the barriers subscale used by Sluijs, Kok, and van der Zee (1993). Lack of time to exercise was reported on a 5-point scale ranging from 5 (“always”) to 1 (“never”). Lack of motivation to exercise was measured on a 4-point scale ranging from 1 (“very motivated”) to 4 (“not motivated”).

Pain

Pain was measured with the Numeric Rating Scale for pain (NRS Pain) (Chiarotto et al., 2018; Chiarotto, Terwee, and Ostelo, 2016; Downie et al., 1978). Patients were asked to rate the intensity of their current pain on an 11-point numeric scale ranging from 0 (“no pain”) to 10 (“worst pain imaginable”).

Disability

Disability was measured with the Dutch-language version of the Quebec Back Pain Disability Scale (QBPDS) (Schoppink et al., 1996). The QBPDS quantifies disability caused by LBP in daily activities. The questionnaire consists of 20 items, and the total score ranges from 0 (no disability) to 100 (completely restricted). Moderate evidence for positive reliability and validity of the Dutch-language version of the QBPDS has been reported in a review by Speksnijder et al. (2016).

Demographics

The following personal and demographic characteristics of the participants were measured: age (in years), gender, height (in centimeters), weight (in kilograms), level of education (i.e. elementary school, high school, vocational school, college, or university), and duration of symptoms (up to 3 months, 3 to 6 months, 6 to 12 months, or more than 12 months).

Procedures to test validity

All patients with LBP who visited the participating physical therapy practices and agreed to participate were screened for eligibility using the inclusion and exclusion criteria. Written informed consent was obtained from the participants prior to the start of the study. The patients received the usual care, and the physical therapists used the EXAS to record HBE recommendations. Additionally, measurements of pain, disability, barriers, and patient characteristics were completed. One week after the HBE program was recommended to the patient, the physical therapist completed the EXAS together with the patient during a follow-up visit.

Reliability

Reliability is defined as “the extent to which scores for patients who have not changed are the same for repeated measurement under several conditions: for example, … by different persons on the same occasion (interrater) or by the same persons (i.e. raters or responders) on different occasions (intrarater)” (Mokkink et al., 2010). The EXAS is designed to be used by a physical therapist who both recommends the HBE program to the patient and rates the quality of performance during the patient’s follow-up visit. However, in clinical practice, it is not uncommon for patients to have more than one physical therapist during their treatment period. For this reason, both intrarater reliability and interrater reliability of the EXAS quality of performance were assessed.

Participants and setting

For the reliability study, two researchers included a convenience sample of healthy adults aged 18–65 years with sufficient command of the Dutch language to read and understand written or spoken instructions. Potential participants were excluded if they were unable to perform exercises due to physical or mental issues or if they experienced pain or discomfort when performing exercises. Recruitment took place at Fontys University of Applied Sciences in the Netherlands.

Measurements

Quality of performance of the exercises by the participants was rated using the scoring matrix of the EXAS (Table 1). The following personal and demographic characteristics of the participants were measured: age (in years); gender; height (in centimeters); weight (in kilograms); and level of education (i.e. elementary school, high school, vocational school, college, or university).

Procedures to test reliability

All participants provided written informed consent prior to the start of the study. Subsequently, personal and demographic data were collected.

Four different exercises commonly recommended by physical therapists in HBE programs for LBP were selected for reliability testing by a panel of physical therapists specialized in treating patients with LBP. The selected exercises were the squat, the deadlift using a broomstick, the lunge, and the bridge. For each exercise, participants were asked to watch an instructional video showing an actor performing the exercise; additional written instructions were also available. Participant performance of the exercise was recorded using two high-definition video cameras (JVC Everio GZ-HM300, JVC, Yokohama, Japan) capturing video at 30 fps and set up at a distance of 3 meters from the front and from the left side. The process was repeated until the participant had completed all four exercises and recordings were successfully collected. Subsequently, two physical therapists were asked to view the video recordings and rate the quality of performance of the exercises by the participants. The physical therapists both had 10 or more years of experience treating patients with LBP, but they worked in different settings. The first physical therapist worked in a health-care center, and the second physical therapist worked in a primary care physical therapy practice. After one week, the first physical therapist repeated the process to complete data collection.

Data analysis

A priori, a sample size of 50 participants for both the validity and the reliability testing was used in accordance with the recommendations made for a good rating by the COSMIN initiative (Terwee et al., 2012). All analyses were performed using IBM SPSS Statistics for Windows, Version 23.0 (Armonk, NY: IBM Corp). All data were anonymized before analysis.

Personal and demographic characteristics

Descriptive statistics were applied to describe the samples for the validity and reliability studies. For continuous data, means and standard deviations were calculated. For categorical data, percentages were calculated instead.

Validity

Validity was assessed using Spearman’s rho for the correlations, as all comparator data were collected using ordinal scales. As no gold standard exists for the measurement of adherence to HBE recommendations, it was decided a priori that at least three of the four pre-determined hypotheses would need to be accepted to confirm the validity of the EXAS.

Reliability

Intrarater and interrater reliability were assessed using Cohen’s kappa with quadratic weights (Kottner et al., 2011). Additionally, the 95% confidence interval was calculated. Results were interpreted using the guidelines proposed by Cicchetti and Sparrow (1981). These guidelines state that kappa scores: below 0.40 are poor; between 0.40 and 0.59 are fair; between 0.60 and 0.74 are good; and between 0.75 and 1.00 are excellent.

Results

Patient characteristics

For the validity study, 30 patients with LBP were included. Before data collection was completed, three patients withdrew without providing a reason. Therefore, validity was determined based on data from 27 patients. Missing values on all variables varied between 0% and 7%. In cases of missing data, pairwise exclusion was performed. At the time of testing, the mean age of the patients was 39.2 (± 11.1) years. Thirteen patients had been suffering from LBP for a period of less than 3 months, one subject for 3 to 6 months, and 13 subjects for more than 6 months. Further demographic data of the participants, adherence rates, and the EXAS score can be found in Table 2.

Table 2. Patient characteristics of the validity study of the exercise adherence scale

	n	%	Mean (SD)	Range
Total sample	27
Age, in years	27		39.5 (11.3)	21–59
Male	16	59.3
Female	11	40.7
Height (cm)	27	100	174 (10.1)	160–197
Weight (kg)	27	100	79.8 (16.2)	58–112
Education
Elementary school	1	3.7
High school	6	22.2
Vocational school	18	66.6
College or university	2	7.4
Duration of symptoms
0 to 3 months	13	48.1
3 to 6 months	1	3.7
More than 6 months	13	48.1
Adherence rate*	27	100	67.4 (27.2)	16–100
EXAS score	27	100	57.1 (25.9)	12.8–100

n: number of participants in sample; SD: Standard Deviation; cm: centimeters; kg: kilograms; *: Adherence rate is calculated as the percentage of patient-reported adherence to therapist home-based exercises recommendations; EXAS: Exercise Adherence Scale

In total, 50 participants performing four different exercises were recorded for the reliability study, resulting in 200 observations. The average age of the participants in the reliability study was 25.6 (±7.37) but ranged from 18 to 55 years (Table 3).

Table 3. Patient characteristics of the reliability study of the exercise adherence scale quality of performance score

	n	%	Mean (SD)	Range
Total sample	50
Age, in years	50	100	25.6 (7.37)	18–55
Male	31	62
Female	19	38
Height (cm)	50	100	177.1 (11.3)	151–200
Weight (kg)	50	100	73.3 (11.6)	50.4–106.1
Education
Vocational school	1	2
College or university	49	98
EXAS Quality of performance	200*		4^†	1–5

n: number of participants in sample; SD: Standard Deviation;

cm: centimeters; kg: kilograms; EXAS: Exercise Adherence Scale; *: quality of performance scores from the first assessment only; ^†: Mode reported instead of mean

Validity

For convergent validity, the association between lack of time to exercise and the EXAS was rho = 0.47 (p = .013), and the association between lack of motivation to exercise and the EXAS was rho = 0.48 (p = .011) (Table 4). For divergent validity, the association between pain and the EXAS was rho = 0.24 (p = .22), and the association between disability and the EXAS was rho = 0.005 (p = .98).

Table 4. Associations between the exercise adherence scale and lack of time to exercise, lack of motivation to exercise, pain, and the Quebec back pain disability scale

	Lack of time to exercise	Lack of motivation to exercise	Pain	QBPDS
n	27	27	27	26
EXAS	0.47*	0.48**	0.24	0.005

QBPDS: Quebec Back Pain Disability Scale; n: number of participants in sample; EXAS: Exercise Adherence Scale; *: p < 0,05; **: p < 0,001; Spearman’s rho was used for all associations

Reliability

For intrarater reliability, Cohen’s kappa using quadratic weights was K_qw = 0.87 (95%-CI 0.83–0.92), p < .001, with a total of 200 observations of four exercises performed by 50 healthy subjects. In 200 observations, disagreement between repeated ratings of the same video by the same therapist occurred in 41 ratings. Out of these 41 ratings, only one differed by 2 points, and in all other cases, the difference was 1 point.

Interrater reliability was much lower with K_qw = 0.36 (95%-CI 0.27–0.45), p < .001. The raters disagreed on the score in 142 cases, 77 ratings differed by 1 point, 53 ratings differed by 2 points, and the remaining 12 cases differed by 3 points.

Discussion

The aims of the current study were to develop an instrument to measure adherence to frequency, intensity, and quality of performance of HBE programs recommended by a health-care professional and to evaluate its construct validity and intrarater reliability, resulting in the development of the EXAS. The instrument contains HBE recommendations from the health-care professional, patient-reported adherence to intensity and frequency of the HBE recommendations, and an observational component. This approach is in line with recommendations from Peek, Carey, Mackenzie, and Sanson-Fisher (2019) who suggested that adding an observational component to the assessment of adherence might more accurately reflect a patient’s efforts to follow HBE recommendations from a physical therapist.

For the validity of the EXAS, the strength of the correlations between the EXAS, time to exercise, and motivation to exercise were moderate and confirmed the initial hypotheses. It was hypothesized that the associations would be moderate at best due to the large variety of factors related to adherence to HBE programs and the consistently moderate associations found in the literature (Peek, Carey, Mackenzie, and Sanson-Fisher, 2019; Sluijs, Kok, and van der Zee, 1993).

Divergent validity hypotheses were also all moderate as expected a priori. Both pain and disability were not significantly associated with the EXAS. It can be reasoned that pain and disability prevent patients from exercising or reduce adherence to HBE programs, which can potentially increase pain and disability. However, the opposite may also be true. Patients experiencing more pain and disability might be more motivated to exercise to reduce their symptoms. This ambiguity is reflected in the lack of association between pain and disability and the EXAS. A possible explanation for this can be found in the different strategies patients use to cope with pain and disability (Cabak et al., 2015). Indeed, two of the most-reported strategies to cope with pain by patients with chronic LBP are praying and hoping (i.e. passive strategy) and increased behavioral activity (i.e. active strategy).

The intrarater reliability of the quality of performance component of the EXAS is excellent (r = 0.87, 95% CI 0.83–0.92). With this score, the reliability estimate exceeds the standard threshold of 0.70 for use as a between-groups comparison measure (Frost et al., 2007). This result is very similar to the intrarater reliability results found in a study using a 10-point rating scale (Hermet et al., 2018). Of the six physical therapists rating patient quality of performance using this 10-point scale, four scored between 0.82 and 0.88, with the remaining two scoring 0.72 and 0.74, respectively. The primary difference between this study and the current study is the scale on which quality of the performance was rated. In the study by Hermet et al. (2018), a 10-point rating scale was used, whereas, in the current study, a 5-point ordinal scale with additional explanation was used to provide a more standardized means of interpreting the different ratings.

The results for interrater reliability of the EXAS quality of performance are poor. This finding is almost identical to the results of Hermet et al. (2018), who found an interrater reliability of 0.34 (95%-CI 0.07–0.48) for primary care physical therapists rating strength exercises. They proposed that different backgrounds and expectations between physical therapists might be the cause of low interrater reliability, as higher reliability scores were found in trained physical therapists. The large number of disagreements between ratings by untrained therapists in the current study appears to support this hypothesis when compared with the much lower number of disagreements between repeated ratings by the same therapist. As a result, clinicians and researchers using the EXAS to assess adherence to HBE programs should consider training or instructing health-care professionals in the scoring of quality of performance to increase interrater reliability.

During the data collection phase of the current study, a new measure of adherence was published (Newman-Beinart et al., 2017). The Exercise Adherence Rating Scale (EARS) is a 6-item questionnaire aimed at measuring adherence behavior and exploring reasons for nonadherence. The full instrument consists of three sections: Prescribed Exercise Questionnaire (Section A), Exercise Adherence Rating Scale (EARS) (Section B), and What helps or hinders doing your exercises? (Section C) (Newman-Beinart et al., 2017). Notably, the questions on frequency and intensity from Section A are similar to but less detailed than the frequency and intensity parts of the EXAS. Quality of performance is entirely absent from the EARS, whereas the EXAS collects no information on reasons for adherence behavior. When used complementarily, the EXAS and EARS provide detailed and extensive information on adherence to frequency, intensity, and quality of performance recommendations from a health-care professional, as well as on reasons for the adherence behavior reported by the patient.

Strengths and limitations

The first strength of this study is that it is the first to develop an instrument to measure adherence to frequency, intensity, and quality of performance of HBE recommendations in patients with LBP (Uzawa and Davis, 2018). During the development phase of the instrument, patients, physical therapists, and experts were involved, in accordance with the advice from Terwee et al. (2007). In addition to development, the clinimetric properties of the instrument were also assessed. Using a measure with known validity and reliability provides a better understanding and interpretation of the findings when assessing adherence. As a result, clinicians are better able to tailor their treatments to individual patients. Researchers can use the EXAS to assess the effectiveness of HBE interventions and statistically control for adherence of the study participants, possibly reducing the underestimation of treatment effects. A second strength of this study is the high number of 200 observations used for the reliability testing of the EXAS. Furthermore, video recordings were used to rate the quality of performance, which eliminated the impact of possible variations in the performance of exercises by patients or differences in the rater’s point of view between measurements. This increases confidence in the findings for intrarater and interrater reliability.

The current study also has several limitations. The first limitation is that the participants recruited for the reliability study were healthy subjects. Healthy subjects perform better during functional movement tasks when compared to patients with chronic LBP (Ko, Noh, Kang, and Oh, 2016). However, the reliability of the scoring system for the quality of exercise performance depends on the agreement between different ratings made by the same rater. The underlying causes for better or worse performance by the patient are not relevant. In daily practice, a physical therapist will select exercises for an HBE program and tailor the difficulty of these exercises to correspond with the patient’s level of ability, thereby eliminating any differences in performance with a healthy subject.

The second limitation is the low number of only four different exercises used for the reliability study. As a physical therapist can select from a vast list of possible exercises when designing an HBE program, using all of these exercises would have been impractical. Therefore, an expert panel selected four exercises commonly recommended to patients with LBP to be used in the study.

The third limitation is the potential patient bias when reporting adherence to frequency and intensity recommendations. However, the impact of overreporting of adherence by patients is most likely mitigated by adding the quality of performance score to the assessment of adherence. Quality of performance is likely to be low in nonadherent patients, resulting in a lower EXAS score and a more realistic approximation of actual adherence. However, more research is required to confirm this hypothesis.

Another limitation is the lack of reliability testing of the frequency and intensity aspects of the EXAS. Although the validity of the EXAS and the reliability of the quality of performance assessment were investigated in the current study, additional research is needed to determine the clinimetric properties of the EXAS in patients with LBP.

The last limitation is the relatively small sample size for the validity study of the EXAS. A sample size of 50 or greater is recommended by the COSMIN checklist for a “good” rating, but practical reasons prevented the achievement of this goal (Terwee et al., 2012). Although 42 physical therapists agreed to recruit patients with LBP for the study, many of them did not manage to do so during the inclusion period of the study. Nevertheless, given the homogeneity of the patients in the validity study, it appears unlikely that including more patients would have yielded different results.

Adherence to HBE programs remains a complex and multi-dimensional construct. Although the EXAS appears to be a valid and reliable instrument, it is still inferior to direct observation. The EXAS shares this disadvantage with every measure of adherence, as all current measurement instruments for adherence to HBE recommendations rely on the patient’s memory, perception, and honesty. Despite this limitation, the EXAS is the only instrument incorporating the quality of exercise performance in the assessment of adherence.

Additionally, the EXAS score is an interesting theoretical construct that may allow for new ways to study which determinants of an HBE program are most important for the patient. For instance, an important question to answer in future research will be whether the quality of performance of exercises contributes to treatment effects or whether attention should be focused on adherence to frequency and intensity recommendations alone. Although a focus on the quality of performance of exercises could potentially deter patients from exercising at home, the added attention to detail could also improve a patient’s feeling of being supported by their therapist, increase self-efficacy, and increase the perceived importance of exercising at home as part of their treatment leading to increased adherence. In daily practice, primary care physical therapists and other clinicians often rely on their training and experience to tailor treatment to respond to the individual needs of patients with LBP to achieve the best outcomes. Indeed, tailoring interventions to the patient has been found to increase patient outcomes and enhance treatment effects (Foster et al., 2014). However, for HBE programs, it remains unclear whether the specific exercises selected by the physical therapist, the quality of performance of the exercises by the patient, or the increase in physical activity from doing exercises are responsible for the effects found. The EXAS score allows researchers to investigate whether clinicians should focus on correct performance of exercises, on adherence to frequency and intensity, or on both.

Conclusion

The EXAS demonstrates acceptable construct validity for the measurement of adherence to HBE programs. Additionally, the EXAS shows excellent intrarater reliability and poor interrater reliability for the quality of performance score and is the first instrument to measure adherence to frequency, intensity, and quality of performance of HBE programs. The EXAS allows researchers and clinicians to better investigate the effects of adherence to HBE programs on the outcomes of interventions and treatments.

Conflicts of interest

None declared.

Acknowledgments

The authors would like to thank Thom Habets and Floor Vluggen for their assistance with data collection and Erik-Jan Haan for his assistance as a rater for reliability testing.

Supplemental material

Supplemental data for this article can be accessed on the publisher’s website.