Measuring wearing time of knee-ankle-foot orthoses in children with cerebral palsy: comparison of parent-report and objective measurement

Abstract

Purpose state: Orthotic wearing time may be an important confounder in efficacy studies of treatment in children with spastic cerebral palsy (SCP). Most studies measure parent-reported wearing time (WT_parent) with questionnaires, but it is questionable whether this yields valid results. This study aims to compare WT_parent with objectively measured wearing time (WT_obj) in children with SCP receiving orthotic treatment.

Method: Eight children with SCP participated in this observational study. For one year, they received knee-ankle-foot orthosis (KAFO) treatment. WT_parent was measured using questionnaires. WT_obj was measured using temperature sensor-data-loggers that were attached to the KAFOs. The 2.5th and 97.5th percentiles and median of differences between methods (per participant) were used to calculate limits of agreement and systematic differences.

Results: There was no systematic difference between WT_parent and WT_obj (0.1 h per week), but high inter-individual variation of the difference was found, as reflected by large limits of agreement (lower limit/2.5th percentile: −1.7 h/week; upper limit/97.5th percentile: 11.1 h/week).

Conclusions: Parent-reported wearing time (WTparent) of a KAFO differs largely from objectively measured wearing time (WT_obj) using temperature sensors. Therefore, parent-reported wearing time (WT_parent) of KAFOs should be interpreted with utmost care.

Implications for Rehabilitation

• Low wearing time of orthoses may be a cause of inefficacy of orthotic treatment and incorrect reported wearing time may bias results of efficacy studies.

• Results of this study show that parent-reported wearing time is not in agreement with objectively measured wearing time.

• Parent-reported wearing time of KAFOs should be interpreted with utmost care.

• Objective methods are recommended for measuring orthotic wearing time.

Keywords:

• Orthotic devices
• self-report
• parent-report
• questionnaires
• wearing time
• cerebral palsy

Introduction

According to the International Society for Prosthetics and Orthotics consensus document,[1,2] orthotic treatment is applied in efforts to prevent deformities and muscle shortening in children with spastic cerebral palsy (SCP). Applying a high tensile strain onto muscles by stretching the muscle tendon complex using orthoses is presumed to increase joint range of motion or prevent its loss.[2] Wearing time is an important factor in determining effectiveness, as efficacy of treatment with orthoses is suggested to be dependent on duration that muscles are strained.[3] Two recent studies,[4–6] investigating the efficacy of ankle-foot orthoses (AFO) and knee-ankle-foot orthoses (KAFO) straining the calf muscles, used questionnaires filled in by the participant’s parents to assess wearing time. However, it is questionable whether such parent-reported wearing time (WT_parent) is a valid measure.

Research focusing on objective methods to measure wearing time has been reported for patients with scoliosis, wearing spine orthoses. Several objective methods are available, such as data acquisition using temperature sensors [7–10] or using force transducers [9,11] attached to the orthoses. These methods yielded reliable and valid wearing time estimates.[7–11] Temperature sensors have been mentioned as most feasible with good agreement with actual wearing time, with no reported mechanical problems and the sensors being easy to use.[9]

Objective measures have been used for comparison of self-reported wearing time and actual wearing time (further referred to as objectively measured wearing time (WT_obj)). Previous studies assessing the reliability and validity of self-reported wearing time measurements in patient with spine orthoses showed that, in general, self-reports by patients or parents led to overestimation.[8,12–16] In particular cases, patient-reported wearing time was twice the WT_obj.[14,16] Also children with clubfoot deformities, who were wearing a foot abduction orthosis, patient-reported wearing time (by parents) was overestimated.[17] In efficacy studies, such overestimation may lead to erroneous conclusions if orthoses were not worn long enough to be effective.

To the best of our knowledge, in children with SCP who wear KAFOs or other orthoses to prevent loss of ankle-foot range of motion, WT_parent has not been compared to WT_obj. As wearing instructions, aim of treatment and design of orthoses are different from patients wearing spine orthoses, validity of WT_parent may vary from the above described studies. Therefore, we aim to compare WT_parent in children with SCP, collected using questionnaires, and WT_obj, measured using temperature sensor-data-loggers attached to the KAFO. It is hypothesised that WT_parent differs from objective measurements.

Methods

This observational study is part of a randomized controlled trial that tested the efficacy of KAFO treatment. Approval for this study was obtained from the Medical Ethics Committee of VU University Medical Center.[4,5] All participants (if older than 12 years) and their parents signed an informed consent form.

Participants and study design

All children participating in this study were also participants in the experimental group of a multi-centre study investigating the efficacy of KAFOs to prevent equinus in children with SCP (Splint study [4,5]). In the Splint study, 30 children were recruited and 17 of them were randomized to the KAFO group. They were recruited between February 2010 and April 2012. The KAFOs of children who were recruited in Amsterdam or Venlo (eight children, allocation was determined by the child’s residence) were equipped with a temperature sensor-data-logger to measure wearing time (Figure 1). These eight children were included in the present study. Although the sensor-data-logger was visible for the participants, they were not informed about the actual aim of the device. Besides that wearing time was measured objectively, parents of participants were asked to report KAFO wearing time for their child. The participants were aged between 4 and 16 years and were able to walk independently (gross motor function classification system (GMFCS) level of I–II) or with walking aids (GMFCS level III).[18] All participants were instructed to wear KAFOs for at least 6 h every other night.

Figure 1. The temperature sensor-data-logger attached to the KAFO. The KAFO as shown in the figure is configured with a fixed knee angle and a variable ankle angle using a spring that provides variable ankle dorsiflexion. The temperature sensor-data-logger was attached to the KAFO with a nut (upper inset) and bolt (lower inset). The flat side (lower inset) of the sensor-data-logger touched the participants skin while wearing the KAFO.

Measurements

WT_parent in hours/week, was collected using questionnaires. For all days of the 4th week of each month during the treatment period, parents of participants were asked to complete online questionnaires. These questionnaires were sent to the parents by email (Netquestionnaire Nederland B.V., Utrecht, The Netherlands). A reminder was sent after two weeks if parents did not respond. The following multiple choice question was asked: “How many hours did your child wear his/her KAFO?”. Answers were given as seven categories, ranging from 0–1 h to 6 h or more.

WT_obj in hours/week, was measured with a temperature sensor-data-logger (UTBI-001, Onset Computer Corporation, Bourne, MA) during the treatment period. The sensor-data-logger was embedded in the shelf of the KAFO, touching upper-leg skin of the participant. Temperature was recorded every 15 min. KAFO wearing time was derived from time–temperature data. An increase of at least 3 °C in temperature within 45 min, indicated that the KAFO was put on. As KAFO temperature is not expected to increase over normal body temperature (37 °C), periods with higher temperatures were ignored, because it was assumed that these temperature increases were artefacts caused by external heating (e.g., KAFO exposed to direct sunlight or heater radiation). The KAFO was assumed to be removed if temperature decreased to values similar to previous periods that KAFO was not worn.

Analyses

For each participant, WT_parent and WT_obj per week were calculated for all weeks that data of both methods were available. A week was only analysed if WT_parent and WT_obj measurements were available for at least four days. If only 4–6 days of a week were available, wearing time of those days was used to estimate total wearing time of seven days. For WT_parent, the centre value of each time category (i.e., 0.5 for 0–1 h, 1.5 for 1–2 h up to 6.5 for 6 h or more) was used to calculate total wearing time of the week.

Agreement between the two methods was assessed according to the non-parametric Bland and Altman method.[19] The non-parametric method was used because of violation of normal distribution of data due to some extreme values for differences in wearing time between measurement methods. For each participant, median of the differences between both measurement methods per week was determined. In addition, the group median of differences and 2.5th and 97.5th percentile score were determined, reflecting the systematic difference between methods and limits of agreement, respectively.

Results

Temperature sensor data were available for eight participants of the Splint study (mean age: 8.7 ± 1.9 years; gender: six males, two females; GMFCS: level I (n = 2), level II (n = 4), III (n = 2)). The period of KAFO treatment of four participants was shorter than one year because other treatment (e.g., botulinum toxin treatment) was indicated clinically (n = 3) or the KAFO caused too much discomfort (n = 1) and therefore the protocol was not finished. As a consequence, KAFO treatment varied from two months to one year. Individual response to questionnaires ranged from 22 to 100% with a median of 87%. For about 25% of the sent questionnaires, reminders had to be sent. Acceptable time–temperature data that was available per participant ranged from 10 to 100% (median: 66%). Missing time–temperature data was caused by (1) too high air temperature because of weather conditions, leading to a too small increase in temperature if the KAFO was worn (3% of total missing time–temperature data) or (2) technical problems with sensor-data-loggers (due to sensor failure that was not related to KAFO wearing), leading to empty data-files (97% of total missing time–temperature data). The period that data of both WT_parent and WT_obj were available ranged from 1 to11 weeks per participant, with a median of three weeks.

Individual median, interquartile range, minimum and maximum values of WT_parent and WT_obj are presented in Figure 2. Temperature data show that KAFO wearing time was lower than prescribed and in one case, KAFO wearing time was 0 h. In addition, data show that, in some participants, there were large differences between WT_parent and WT_obj. The median of these differences per week are shown per participant in Figure 3. This figure shows that there was no systematic difference between both methods (median: 0.13 h per week), but that differences between methods show large variability between participants (ranging from −2.0 to 12.9 h per week), resulting in large limits of agreement (lower limit/2.5th percentile: −1.7 h per week; upper limit/97.5th percentile: 11.1 h per week). This indicates that agreement between methods is low.

Figure 2. Box-and-whisker plots of wearing time per week per participant. Comparison of parent-reported wearing time per week (WT_parent) and objectively measured wearing time per week (WT_obj) per participant. Numbers of weeks included in analyses are for each participant noted on the x-axis. For participant 107, WT_parent and WT_obj were reported and measured as 0 h per week. The middle line of each box shows the median wearing time. The outer lines of each box show the 25th and 75th percentile of the wearing times and the end of the whiskers shows the maximum and minimum values.

Figure 3. Bland–Altman plot: median differences per participant. Bland–Altman plot of parent-reported wearing time per week (WT_parent) and objectively measured wearing time per week (WT_obj). Upper and lower dotted lines indicate limits of agreement (2.5th and 97.5th percentile). Dotted line in middle indicates median of the difference between WT_parent and WT_obj for all participants.

Comparing measurements within participants, a high variability in differences between the two methods was shown as well. Individual data of weekly measurements of two participants is shown in Figure 4. For participant 102, WT_parent ranged from 0 to 12 h per week, while WT_obj ranged from 0 to 8.5 h per week (Figure 4(A)). Differences between WT_parent and WT_obj ranged from −5.5 to 5.5 h per week with a median of 0 h per week (Figure 4(C)). For participant 207, WT_parent ranged from 0 to 16.5 h per week, while WT_obj ranged from 0 to 8.5 h per week (Figure 4(B)). Differences between WT_parent and WT_obj ranged from −1 to 10 h per week with a median of 2.5 h per week. The wide range of within participant difference between WT_parent and WT_obj was also expressed in high individual limits of agreement (see Figure 4(C,D), participant 102: lower limit/2.5th percentile: −5.4 h per week, upper limit/97.5th percentile: 5.2 h per week; participant 207: lower limit/2.5th percentile: –0.9 h per week, upper limit/2.5th percentile: 9.4 h per week).

Figure 4. Individual data: examples. A + B: comparison of parent-reported wearing time per week (WT_parent) and objectively measured wearing time per week (WT_obj) for two participants. If no column is visible, wearing time is 0 h. C + D: Bland–Altman plot of parent-reported wearing time per week (WT_parent) and objectively measured wearing time per week (WT_obj) for the same participants as in A + B. Upper and lower dotted lines indicate limits of agreement (2.5th and 97.5th percentile). Dotted line in middle indicates median of the difference between WT_parent and WT_obj for all measurements.

Discussion

The present study shows that WT_parent of KAFOs differs from WT_obj. The differences in reported wearing time between methods vary considerably per participant: some parents reported higher wearing times than what was measured by temperature sensor-data-loggers while others reported lower wearing times. In literature regarding spine orthoses, mostly overestimation of self reported wearing time/WT_parent is reported (differences range from 0 to 150%).[8,12–16] It should be noted that these conclusions are often based on means of all participants. In accordance with the reported range of differences between wearing time measurement methods, the present study found no systematic difference or at most a small overestimation of WT_parent. The large range of overestimation reported in literature may indicate a similar trend as found in this study that self reported wearing time/WT_parent and WT_obj varies considerably between participants. Therefore it may be concluded that WT_parent and WT_obj are not comparable.

In the present study, high values of limits of agreement between WT_parent and WT_obj were found. If the WT_obj method is considered as a golden standard, the limits of agreement reflect the error of WT_parent. In the present study, the limits of agreement indicate that only if parents reported that the KAFO was worn for more than 11 h per week, it can be concluded with certainty that the KAFO was worn for at least a short period. However, participants often did not wear their KAFO for such long period in general (i.e., Figure 2 shows that participant 102 and 207 did wear their KAFO not more than 8.5 h per week). This indicates that parent reported wearing time cannot distinguish whether participants wore their KAFO or not. It should be noted that one outlying value (participant 109, Figure 1) affected error of parent-reported outcome considerably. However, sub-analyses excluding this outlier still showed an upper limit of agreement of 2.4 h per week. Taking into account that the median of WT_obj of the remaining seven participants was about 5 h per week, agreement between methods is still very low.

The response rate of questionnaires was relatively high (approximately 85%). Additional correlation analysis showed that there was no correlation between response rate and agreement between WT_parent and WT_obj (Spearman’s rho =0.252, p= 0.5). Another factor possibly affecting agreement was delayed response to questionnaires: it was often necessary to send reminders to parents after two weeks. This may indicate that parents did not complete questionnaires on the day of KAFO wearing and may explain the variability in response. However, uncertainty about delayed responses and therefore variable recall time will always be part of any parent report method. In this study, WT_parent could not be influenced by the results of WT_obj when WT_parent response was delayed because parents and children were not informed about the actual aim and results of the temperature sensor-datalogger. Overestimation in WT_parent was larger compared to under estimation: some parents reported KAFO wearing for periods that the temperature-data-logger did not register any higher temperatures. This may be the result of a more general phenomenon that people adjust their answers according to socially desired answers. It is unknown whether other methods of self-reporting may yield more accurate results.

Due to validity problems of parent-reported wearing time, we recommend the use of objective techniques to measure KAFO wearing time. We agree with Hunter et al.,[9] that it is feasible to use temperature sensor-data-loggers to measure KAFO wearing time. However, the high percentage of missing data with the current device indicates that technical failures can occur and need to be solved. In this study, we were able to collect data for each participant after replacing the nonfunctioning sensors. In most datasets, it was straightforward to distinguish KAFO wearing from non-KAFO wearing using criteria described above. However, at high ambient temperatures (about 25 °C), it is more difficult to determine KAFO wearing time because differences in temperature with or without a leg in the KAFO was smaller. Comparison of data from temperature sensors inside the KAFO and additional outside temperature sensors [20] may solve this problem.

Limitations of the study

A limitation of this study is the low number of participants. Non-parametric Bland–Altman plots are less reliable in small sample sizes.[19] However, because of the large intra-individual variation, we believe that research using larger samples will yield similar conclusions.

A second limitation is that WT_parent was assessed by only one method and cannot be generalized to other self/parent-reporting wearing time methods. We chose to send online questionnaires during each day. This was only done for every 4th week of the month to decrease burden on participants.

Applications to clinical practice

Measuring parent-reported orthotic wearing time with online daily questionnaires for one week per month does not show to agree with objective measurements using a temperature sensor-data-logger. Outcome of parent-reported questionnaires has to be interpreted with extreme care.

Conclusions

Wearing time of a KAFO assessed by questionnaires is not in agreement with of wearing time assessed by temperature sensor-data-loggers and should therefore be interpreted with utmost care. Using objective measurement methods is preferred. Regarding temperature sensors, we recommend to use a more sophisticated method than used in this study to measure orthotic wearing time with temperature sensor-data-loggers, like the method described by Bus et al.,[20] in order to overcome difficulties in determining orthotic wearing time at high ambient temperatures.

Acknowledgements

We thank all participants and their parents for their participation in this study. We also thank OIM/Noppe orthopedie, Noordwijkerhout, the Netherlands; Kamer orthopedie, Amsterdam, the Netherlands; and Orthopedie Heckert & van Lierop, Venlo, the Netherlands for their cooperation.

Disclosure statement

The authors report no declarations of interest.

Additional information

Funding

The Splint study is financially supported by a grant from the Phelps Stichting voor Spastici, Bussum, The Netherlands [grant number: 2008032]. In addition to funding, the Splint study is financially supported by a donation of Ultraflex Europe, Ternat, Belgium on behalf of Ultraflex Systems, Inc., Pottstown, Pennsylvania, USA. The “temperature sensor-data-loggers” were partly sponsored by PEDAK Meettechniek BV Heythuysen, The Netherlands. Although we received financial support from Ultraflex Europe, Ultraflex Systems Inc. and PEDAK Meettechniek, they did not have authority over this study.