Within and between-day reliability of energetic cost measures during treadmill walking

Abstract

The efficacy of assistive devices used during walking is often measured as a reduction in metabolic cost. Metabolic cost is typically assessed within a day or on multiple days, yet the benefit of performing within-day vs. between-day metabolic assessments is unknown. The purpose of this study was to determine the within-day minimal detectable change of standard measures of physiologic performance using a conventional portable metabolic system (K4b² Cosmed, Rome, Italy), and compare these to between-day values. Twenty healthy adults completed two identical data collection sessions on separate days. In each session they performed three bouts of treadmill walking interspersed with three bouts of rest while oxygen consumption ($\dot{{VO}_2}$), carbon dioxide production, and heart rate were measured. Intraclass correlation coefficients (ICC) and minimal detectable change values were calculated for non-resting within-day, as well as all between-day comparisons. All within-day measures were clinically reliable (ICC > 0.96), while between-day measures were generally less reliable (ICCs > 0.82). Within-day minimal detectable change values (walking heart rate = 4.9 bpm; gross $\dot{{VO}_2}$ = 0.80 mL/kg/min; net $\dot{{VO}_2}$ = 0.80 mL/kg/min; cost of transport = 0.022 J/Nm) were about half as large as between-day values. The results of this study suggest that, where possible, physiologic changes should be assessed within a single day of testing to maximize ability to detect small changes in performance.

Keywords:

• reliability
• gait
• energetics
• metabolic cost
• assistive walking devices

Public Interest Statement

A key component of the design and prescription of medical devices that assist in walking is a thorough understanding of how patients will benefit from their use. Reducing the energy patients must expend to walk is often a primary goal of such devices. In order to accurately determine if this goal is being met, it is important for researchers to separate everyday changes in energy expenditure from changes caused by using the device. This work aims to provide a standard against which changes in energy expenditure within one day of testing can be measured. With this benchmark, researchers will have a better understanding of which assistive devices result in meaningful reductions in energy expenditure. This will allow for more rigorous assessments of assistive walking devices, ultimately providing physicians and patients with a more thorough understanding of their benefits.

1. Introduction

A reduction in metabolic cost is one of the primary goals of assistive devices used during walking such as prostheses (Herr & Grabowski, 2012; Houdijk, Pollmann, Groenewold, Wiggerts, & Polomski, 2009; Major, Twiste, Kenney, & Howard, 2014; Segal et al., 2012; Zelik et al., 2011), orthoses (Balaban et al., 2007; Menotti, Laudani, Damiani, Mignogna, & Macaluso, 2014; Menotti, Laudani, Damiani, Orlando, & Macaluso, 2014; Zissimopoulos, Fatone, & Gard, 2007), or exoskeletons (Malcolm, Derave, Galle, & De Clercq, 2013; Sawicki & Ferris, 2008, 2009). The efficacy of such devices is often tested within a single day (Balaban et al., 2007; Houdijk et al., 2009; Malcolm et al., 2013; Menotti, Laudani, Damiani, Mignogna, et al., 2014; Menotti, Laudani, Damiani, Orlando, et al., 2014; Segal et al., 2012; Zelik et al., 2011; Zissimopoulos et al., 2007) or across multiple days where acclimation to the device is required (Major et al., 2014; Sawicki & Ferris, 2008, 2009). Typical changes observed in energetic measures range from 3 to 8% for prostheses (Herr & Grabowski, 2012; Major et al., 2014; Segal et al., 2012; Zelik et al., 2011), 2 to 12% for orthoses (Balaban et al., 2007; Menotti, Laudani, Damiani, Orlando, et al., 2014; Zissimopoulos et al., 2007), or 7 to 15% for exoskeletons (Malcolm et al., 2013; Sawicki & Ferris, 2008, 2009).

Metabolic device evaluation studies have typically utilized portable metabolic units such as the Cosmed K4b² system (Cosmed, Rome, IT). The between-day reliability and/or minimal detectable change (MDC) of several physiologic measures associated with the K4b² system in various populations have been previously reported (Darter, Rodriguez, & Wilken, 2013; Stookey et al., 2013; Thomas, Buckon, Schwartz, Sussman, & Aiona, 2009; Welch, Strath, & Swartz, 2015). Although comparisons are often made within a single day of testing, the reliability and MDC of energetic measures collected within a single day during treadmill walking are not known. Given the benefit of collecting data on a single day where factors such as hydration and fitness are more controlled, we would expect the consistency of these measures to improve. Therefore, the purpose of this study was to calculate within-day MDCs for several physiological measurements, including heart rate, oxygen consumption, and cost of transport, using a K4b² system, and compare these to between-day values. Establishing within-day MDCs provides a benchmark by which investigators can determine true changes in human performance and allow for more precise assessments of assistive devices used during walking.

2. Methods

2.1. Participants

Twenty healthy adults (9 females, 11 males) were recruited using flyers and an electronic community registry (UMClinicalStudies.org). Their mean (SD) age, height, weight and BMI were 30 (15) years, 1.74 (0.08) m, 72.4 (11.3) kg, and 23.8 (2.8) kg/m², respectively (see supplemental material for additional details). All subjects provided written informed consent prior to participation in this institutionally approved study.

2.2. Experimental protocol

Participants performed two identical test sessions in the morning at least one day apart (14 (13) days). Participants were instructed to abstain from caffeine, avoid strenuous exercise, and fast for at least 4 h prior to testing.

A K4b² portable metabolic system was used to measure the rate of oxygen consumption ($\dot{{VO}_2}$, mL/kg/min), carbon dioxide production ($\dot{{VCO}_2}$, mL/kg/min), and heart rate. This system consists of a mask, heart-rate monitor, and a collection unit that wirelessly transmits data to a laptop where it can be observed in real time (Figure 1(A)). First, participants were asked to remain seated and still for at least 5 min to obtain metabolic cost measures for seated rest. Then, they walked on a treadmill at a speed normalized to leg length and approximated a comfortable walking speed (average speed = 1.22 (0.03) m/s) (Gates, Wilken, Scott, Sinitski, & Dingwell, 2012). Participants walked for a minimum of 5 min, or until they reached steady-state, which was identified in real-time by visually observing a plateau in $\dot{{VO}_2}$ (average time = 9.1 (1.98) min; Figure 1(B)). Once this plateau was reached, participants walked for an additional 3 min to obtain metabolic cost measures for steady-state walking. This process was then repeated such that all participants completed three bouts of walking which were interrupted by 5 min bouts of seated rest.

Figure 1. Experimental Setup: (A) Oxygen consumption and CO₂ production were measured using a COSMED K4b2 system. (B) Oxygen consumption data for a representative subject over one day of testing

Notes: (A) The portable system consists of a mask that covers the nose and mouth, a heart rate monitor worn on the torso, a collection unit that wirelessly transmits the data to a laptop where it can be observed in real time. (B) Subjects walked until the graph of their rate of oxygen consumption, ($\dot{{VO}_2}$), reached a plateau (i.e. steady state). They then walked for an additional 3–5 min. Walking bouts are encompassed in vertical red lines. This process was repeated such that there were three periods of seated rest followed by three periods of walking. Steady-state rest was measured over 1 min periods (solid black line) while steady-state walking was measured over 3 min periods (solid red).

Steady-state walking intervals were later objectively determined using MATLAB (MathWorks, Natick, MA, USA, v.R2014b). Steady-state was defined as a 3 min interval wherein the minute average $\dot{{VO}_2}$ and $\dot{{VCO}_2}$ varied by <10%, and the average respiratory quotient $\frac{\dot{{VCO}_2}}{\dot{{VO}_2}}$ was between 0.7 and 1.0 and varied by <5% (McClave et al., 2003; Reeves, Davies, Bauer, & Battistutta, 2004). We began by assessing the last 3 min of walking. The interval was then iteratively shifted earlier by one breath until all steady-state criteria were met.

Performance measures included gross $\dot{{VO}_2}$ (mL/kg/min), net $\dot{{VO}_2}$ (gross $\dot{{VO}_2}$ − seated rest $\dot{{VO}_2}$: mL/kg/min), cost of transport (COT, J/Nm), and heart rate (HR, bpm). Seated rest $\dot{{VO}_2}$ was calculated as the average $\dot{{VO}_2}$ over the last 90 s of the first resting bout on each day and was included only in between-day analyses. Metabolic rate was estimated from $\dot{{VO}_2}$ and $\dot{{VCO}_2}$ according to standard formulae (Brockway, 1987). COT, defined as the energy cost of moving 1 kg of body mass 1 m (J/N m), was calculated by dividing net metabolic rate by walking speed (Mian, Thom, Ardigo, Narici, & Minetti, 2006).

2.3. Statistical analysis

Intraclass correlation coefficients (ICCs) were calculated to assess reliability within- and between-days (Portney & Watkins, 2009). Specifically, we used an ICC_3,k model for absolute agreement. Within-day ICCs were calculated for the first day of testing. Between-day ICCs were calculated from averages across bouts for each day. Only variables with a significant F-test (p < 0.05) had a sufficiently heterogeneous group for the ICC to be valid (Portney & Watkins, 2009). ICC > 0.90 indicated reasonable agreement for clinical measures, ICC > 0.75 indicated good agreement, while ICC <  0.75 indicated poor to moderate agreement (Portney & Watkins, 2009). Pooled SD and ICCs were used to estimate the standard error of the measure (SEM) according to the equation:(1) $$\text{SEM}={\text{SD}}_{\text{pooled}}\times \sqrt{1-\text{ICC}}$$(1)

where SD_pooled is the pooled standard deviation across testing bouts or days.

Subsequently, minimal detectable change (MDC₉₅) values were estimated for each variable according to the equation (Weir, 2005):(2) $${\text{MDC}}_{95}=\text{SEM}\times 1.96\sqrt{2}$$(2)

where 1.96 corresponds to a 95% confidence level. MDCs are defined as the smallest change in a measure that exceeds random variation (Lassere, van der Heijde, & Johnson, 2001), and are often used to quantify the limits of expected variation in a measure. Bland-Altman plots comparing daily average values were used to identify proportional bias (Bland & Altman, 1999). Pearson’s correlations were performed to determine if testing separation, subject age, or subject weight were correlated with percent difference in performance measures across days for each subject. All statistical analyses were performed in SPSS version 22 (IBM Corp., Armonk, NY, USA).

3. Results

All 20 subjects were included in the analysis. Data from one subject (S08) was lost for one session on day two of testing due to equipment malfunction. F-tests were significant for all variables (p < 0.01), indicating sufficient heterogeneity. No proportional bias was observed in any measure (see Supplemental material).

All within-day and one between-day measure (walking HR) indicated clinically meaningful reliability (ICC > 0.96; Table 1). Between-day measures had good reliability (ICC > 0.82), except for seated rest $\dot{{VO}_2}$, which had poor reliability (ICC = 0.65).

Table 1. Physiologic measures between- and within-day for treadmill walking

Physiologic measure	Mean (SD)		Within-day reliability			Between-day reliability
	Day 1	Day 2	ICC3,k (95% CI)	SEM	MDC95 (% Average)	ICC3,k (95% CI)	SEM	MDC95 (% Average)
Resting heart rate (bpm)	69.0 (8.5)	69.9 (9.7)	–	–	–	0.85 (0.61, 9.4)	3.6	9.9 (14.2)
Seated rest $\dot{{VO}_2}$ (mL/kg/min)	3.58 (0.55)	3.83 (0.66)	–	–	–	0.65 (0.11, 0.86)	0.37	1.01 (27.3)
Walking heart rate (bpm)	94.2 (8.4)	96.6 (9.8)	0.96 (0.91, 0.98)	1.8	4.9 (5.2)	0.92 (0.81, 0.97)	2.5	7.1 (7.4)
Walking gross $\dot{{VO}_2}$ (mL/kg/min)	12.15 (1.64)	12.35 (1.4)	0.97 (0.93, 0.99)	0.27	0.80 (6.3)	0.82 (0.55 0.93)	0.65	1.80 (14.7)
Walking net $\dot{{VO}_2}$ (mL/kg/min)	8.66 (1.56)	8.48 (1.32)	0.97 (0.94, 0.99)	0.27	0.80 (8.8)	0.82 (0.54, 0.93)	0.62	1.71 (19.9)
Cost of transport (J/Nm)	0.345 (0.052)	0.349 (0.045)	0.98 (0.95, 0.99)	0.008	0.022 (6.3)	0.86 (0.64, 0.94)	0.018	0.051 (14.7)

The within-day MDCs were about half the magnitude of between-day MDCs (Table 1). The MDC for walking heart rate exhibited the smallest comparative difference, with a within-day MDC of 4.9 bpm, which was about 30% less than its between-day MDC of 7.1 bpm (Table 1).

The percent change across days for energetic measures were not significantly correlated with the days between testing (p ≥ 0.24), subject age (p ≥ 0.17), or subject weight (p ≥ 0.54) for any performance measure (Table 2).

Table 2. Pearson’s correlations for energetic measures and subject details

	Walking heart rate (bpm)	Walking gross $\dot{{VO}_2}$ (mL/kg/min)	Walking net $\dot{{VO}_2}$ (mL/kg/min)	Cost of transport (J/Nm)
Days between testing	−0.027	−0.094	−0.275	−0.113
Age	0.320	0.112	−0.036	0.095
Weight	0.147	−0.023	−0.042	−0.081

4. Discussion

The purpose of this study was to determine within-day MDCs for various energetic measures of walking performance and compare these to between-day values. All within-day measures were deemed clinically reliable (ICC > 0.96). Between-day comparisons of physiologic measurements generally exhibited good agreement (ICC > 0.82), but were less reliable than within-session measures. Therefore, consistent with our expectations, the K4b² system was able to reliably detect smaller changes in physiologic measures for comparisons made within a day than comparisons made across different testing days.

The results for between-day reliability were consistent with those reported previously on healthy adults using the K4b² system (Darter et al., 2013) and another portable metabolic system (Blessinger et al., 2009). While seated rest and walking HR MDCs in the present study were 3–4 bpm lower than those previously reported (Darter et al., 2013), between-day oxygen consumption MDCs were roughly 0.86 and 0.61 mL/kg/min higher than previously reported values (Darter et al., 2013) for gross and net $\dot{{VO}_2}$, respectively. This discrepancy may be the result of a larger separation between collection days in the current study. In the present study, testing days were separated by an average of two weeks, compared to five days of average separation in other studies (Blessinger et al., 2009; Darter et al., 2013). The longer between-day interval in the present study may have allowed for greater changes in fitness, body weight, stress levels, or other factors known to influence energetic measurements (Donahoo, Levine, & Melanson, 2004).

Consistent with prior studies (Darter et al., 2013; Thomas et al., 2009), resting $\dot{{VO}_2}$ was less reliable than walking $\dot{{VO}_2}$. Littlewood et al. (2002) suggested that the K4b² device is highly variable when acquiring resting energetic measurements, and may only be suitable for measuring the energetics of activity (Littlewood et al., 2002). When comparing the Cosmed system to the Deltatrac II, a metabolic cart specifically designed to measure resting energetics, Littlewood et al. (2002) revealed a mean bias for REE and VO₂ of 268 ± 702 kcal/day and 51.6 ± 125.6 mL/min, respectively. The exact reason for this discrepancy is not well understood, as its design is similar to devices that have been reported to produce consistently accurate resting measures (Littlewood et al., 2002). However, supine rest, longer and stricter fasting periods, and resting prior to data collection have been shown to improve the between and within-day reliability of resting measures obtained with the Cosmed K4b² system (Welch et al., 2015). These stricter resting protocols should be used if resting measures are of particular importance to a study.

There are several limitations to this study. As mentioned above, the protocol may have limited our ability to determine accurate energetic measures for seated rest. To obtain more reliable measures we could have had participants rest for longer periods of time. This may have improved the reliability for both seated rest and net VO₂. Second, although participants were asked to abstain from exercise and to fast and avoid caffeine, we have no way of determining whether they actually did these things. Failure to adhere to these guidelines could cause increased variability from day-to-day. This would be true for any study unless participants were observed over extended periods of time in the laboratory. Third, there was a wide variation in subject characteristics and range of time between testing sessions. Testing days were separated by an average of two weeks and the age range of subjects tested was large (19–65 years). These factors were not correlated with changes in any measure across days, however. Although this may increase the variability of the data, testing a wide variety of subjects with varying lengths of testing separation increases the generalizability of the results.

In conclusion, the K4b² system is able to detect small changes in physiologic measurements within a day. Within-day MDCs were about half the size of between-day values. These results emphasize the benefit of using within-day methodological designs when feasible as the ability to determine differences in metabolic cost dramatically improves.

Funding

This work was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health [grant number 2UL1TR000433]. Deanna H. Gates is supported by the Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health [grant number K12HD073945].

Declaration

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Additional information

Notes on contributors

Audra Davidson

Audra Davidson is a researcher and Dr Emily S. Gardinier is a postdoctoral fellow in The Rehabilitation Biomechanics Laboratory, run by Dr Deanna H. Gates. In the Rehabilitation Biomechanics Laboratory, we focus on the study of repetitive human movements such as walking and reaching. Throughout these studies, we aim to determine which aspects of movement a person actively controls and how this function can be modeled most effectively. Along with governing control strategies, these models can be used to design both passive and active devices, which can mimic biological function and restore or improve function in individuals with disability. Another focus of our research is determining appropriate outcomes to measure performance with new technology. Without such guidelines, it is difficult to assess the efficacy of assistive devices. The following work aids in this endeavor, providing additional tools for the evaluation of assistive devices.