Effects of different feedback variants on pacing adherence in a field based running test

ABSTRACT

Running field tests are utilised to assess the athlete’s fitness level with high specificity. However, in graded exercise tests the lack of speed control can result in an uneven pace, reducing reproducibility and accuracy. The purpose of this study was to develop feedback variants (FV) and investigate their effect on the athlete’s ability to keep the pace constant. Forty-eight participants completed four trials of a Conconi test (randomised) over four paces (7.5 − 10 km·h⁻¹). A smartphone app provided athletes with four FVs: Classic, Sound (verbal), Vibration, and Sound & Vibration. We found a significant effect of FV on adherence (defined as time within 0.3 km·h⁻¹ of target speed): F(3,141) = 41.45, p < .001, η² = .268. Adherence in Classic (M± SD) 39.51 ± 13.51% was significantly lower (p < .001) than in Vibration (53.24 ± 14.98%), Sound (60.34 ± 11.21%) and Sound & Vibration (59.17 ± 15.17%). The novel FV provided better adherence compared to the conventional method, with Sound and Sound & Vibration showing the highest adherence. Generally, the novel FVs have the advantage of being easily implemented and decreasing the required time for setup, data collection and processing, therefore proving useful in conducting field performance tests.

KEYWORDS:

• Testing
• data
• measurement
• technology
• feedback

1. Introduction

Running in its different forms (jogging, sprinting, short or long distance running) is an ever-popular sport performed by numerous people (Schnohr et al., 2013). One of the reasons for its popularity might stem from the fact that it is a simple sport requiring little equipment. Moreover, several studies have shown that running is associated with physical and mental benefits (Lee et al., 2014; Schnohr et al., 2013). Consequently, running is picked up by a growing number of people with a significant proportion of them aiming to train methodologically in order to increase the physical and mental benefits (Etxegarai et al., 2018; Vickers & Vertosick, 2016).

To be efficient, training needs to be performed within the individual optimal intensity zones depending on the individual goals. One way to identify individual intensity zones is to implement various field performance tests in the training protocol. Field tests vary from covering as much distance as possible over a fixed period such as the Cooper test (Cooper, 1970) to graded exercise tests such as the Conconi test (Conconi et al., 1982) or the “Université de Montréal track test” (Uger & Boucher, 1980). In a graded exercise test athletes are required to run at a fixed pace for either a given duration or distance (Conconi et al., 1982; Uger & Boucher, 1980). The target pace is increased in a spatial (Conconi et al., 1982) or temporal manner (Uger & Boucher, 1980) until the athlete is no longer able to continue. Based on measured values such as heart rate, blood lactate or oxygen consumption, several different thresholds can be computed (Conconi et al., 1982; Uger & Boucher, 1980). These are later used by coaches for setting training volume and intensity of their athletes.

Any performance test needs to be objective, reliable, valid and reproducible in order to be useful. Reproducibility of a performance test can be ensured by requiring athletes to run at a predetermined speed and for that reason tests are often performed on a treadmill in a laboratory where the conditions can be easily controlled. However, laboratory tests are often criticised for not reflecting the competitive environment (Nigg et al., 1995; Sinclair et al., 2013). Therefore, numerous field tests such as the Cooper or Conconi test have been implemented so that the athlete can perform their usual physical activity. During these tests, the examiner has to control whether the athletes perform at the desired intensities and stop the test if athletes fail to comply.

During running tasks, the process or strategy behind the distribution of speed or energy is defined as pacing (Pryor et al., 2020). Pacing is not only essential for performance testing but also in running competitions. For flat running courses, the so-called “even pacing” is a strategy often applied by athletes in order to achieve the best results (Abbiss & Laursen, 2008; Hanley, 2015). While, traditionally, pace-related information was only provided to an athlete by a coach, it can now be given by a technical device through acoustic, optic or haptic feedback. These devices are commonly referred to as feedback systems (Baca, 2008). A simple type of feedback system can take the form of a sports watch. More sophisticated systems can also employ mobile technology such as smartphones or tablet computers. For example, Tate and Milner (2017) used a tablet computer to visualise noise levels during treadmill running in order to help participants reduce impact forces. With the help of the so-called PerPot model a smartphone-based feedback system has been shown to help novice athletes to perform a 10 km trial by means of verbal messages (Dobiasch et al., 2018). Furthermore, smartphones have proven effective in assisting runners by keeping their cadence constant by employing verbal messages, vibrations, or auditory cues (e.g. beating-rhythm) (Aranki et al., 2018; Balvis et al., 2016). However, these systems are tailored for activities with constant pace rather than changing paces.

The feedback provided by such a system can be classified by its modality into tactile, auditory or visual (Sigrist et al., 2013). Furthermore, the timing of feedback can be categorised as either terminal (after the action) or concurrent (during the action) (Lauber & Keller, 2014). Usually, sports watches provide concurrent feedback about the heart rate and speed of the athlete. Moreover, modern systems also offer online platforms where terminal feedback in the form of diagrams and computed metrics are available for further analysis (e.g. Strava, Strava Inc., USA or TrainingPeaks, Peaksware LLC, USA).

While for professional runners the usage of technical aids that provide external feedback is or has been forbidden for some competitive events (e.g. IAAF Regulations C2.1 Rule 144/6.4.4 (IAAF, n.d.)), recreational runners are usually allowed to use them. Setting up a sports watch to achieve an “even pacing” during a race or training session can be a fairly simple task. Nevertheless, setting up other profiles can be cumbersome and, moreover, most systems do not offer support for adaptive profiles. Furthermore, setting up an incremental test can be quite a difficult and error-prone task. Most commercially available systems do not allow setting up individual stop criteria in case of an athlete no longer able to maintain the desired pace. Consequently, the adoption of sports watches or other systems for pacing is not yet pervasive enough.

The effects of mobile feedback systems have been studied with recreational and professional athletes. For example, a previous study by Strohrmann et al. (2014) found that arm carriage can be improved while using vibration feedback similarly as if instructed by a human. Furthermore, Fortmann et al. found that auditory feedback might help experienced runners maintain a constant cadence during training (Fortmann et al., 2012), while Abernethy et al. examined and reported the influence of gait modes (preferred and non-preferred gait speeds) on cognition using an auditory probe reaction task (Abernethy et al., 2002).

Feedback systems are also used for injury prevention and recovery processes which is why several research applications aiming to modify the running pace of athletes have been developed (Agresta & Brown, 2015; Napier et al., 2015). For example, Van Gelder et al. showed that visual feedback might be able to help athletes reduce or control tibial acceleration during treadmill running (Van Gelder et al., 2018). Additionally, Sheerin et al. found that the use of tactile feedback was as effective as the visual counterpart when retraining runners to reduce tibial acceleration (Sheerin et al., 2020). Similarly, Wood and Kipp (2014) showed a reduction in peak acceleration using “thresholds sounds” (i.e. a sound was played to participants when acceleration exceeded a threshold) in their study. Moreover, a study by Afzal et al. showed that tactile feedback using vibrations could help stroke patients to achieve gait symmetry (Afzal et al., 2019).

Previous studies have mostly investigated the effects of feedback at a constant pace. However, this does not reflect the varying intensities used in many testing protocols. Therefore, the main goal of this study is to examine the resulting adherence (defined as the time spent within 0.3 km·h⁻¹ of target speed) of different feedback mechanisms (haptic, visual, and auditory) during different running pace levels. In order to investigate the effects of the feedback variants on adherence, we created a custom application for smartphones which makes the results easily applicable for everyday users as well as practitioners and athletes. We hypothesised that for recreational athletes, different feedback mechanisms will have different effects on pace adherence. Moreover, we expect that novel feedback variants (vibration, sound or sound and vibration) will lead to better results compared to classical acoustic signals.

2. Methods

2.1. Participants

Initially, 55 participants were recruited for the study; however, seven were excluded due to an insufficient number of completed stages in the trial runs. Therefore, a total of 48 recreational runners (Age: 26 ± 3.8 years, Mass: 72 ± 11.88 kg, Height: 177 ± 9.81 cm, 33 males, 15 females) participated in the study and completed all feedback runs. Participants reported to actively participate in 3 to 20 hours of various sports activities (running, swimming, weightlifting and cycling) per week. The study was approved by the ethics committee of the University of Vienna. Written informed consent was obtained from all participants prior to the experiment. Additionally, participants were asked to fill out a cardiovascular risk assessment questionnaire based on the ACSM guidelines (Thompson et al., 2010).

2.2. Protocol

The participants were equipped with a heart rate monitor (Garmin Soft Strap HRM, Switzerland) and a Foot Pod device (Garmin SDM4, Switzerland) to measure heart rate and speed/distance, respectively. Additionally, questions about the sporting background of the participants were asked such as whether they had performed running tests before and whether they train at a fixed speed (possible answers: “never”, “sometimes”, “often”). In order to reduce learning effects subjects were randomly assigned to a sequence of the feedback variants (Classic, Vibration, Sound, Sound & Vibration) based on their arrival on site.

Before testing, the participants performed a preliminary run over 800 m. This calibration run, which also served as a warm-up, was a self-paced run and was used to obtain the calibration factor for the Foot Pod device. During this run, the speed was (M± SD) 10.59 ± 1.27 km·h⁻¹, 95% confidence interval: 10.22 − 10.96 km·h⁻¹ with a mean coefficient of variation of 19.38 ± 7.12%, 95% CI: 17.31 − 21.44%.

For the purpose of the study, an app was created using the PEGASOS framework (Dobiasch et al., 2016, 2019) that records data from the heart rate monitor (1 Hz) and Foot Pod synchronously (2 Hz). Speed was filtered using a five-second rolling average filter. This setting was based on the nature of the task (running with constant speed) and prior experiments. The app was installed on a Samsung A3 (2017 version) smartphone with a standard Android 8.0 operating system. By means of verbal explanation of the feedback variants and the testing procedure, participants were familiarised with the technical aspects of the app and the feedback signals. Additionally, prior to testing in order to get accustomed, we presented the participants with a sequence of the feedback signals. We standardised the position of the phone on the upper arm and secured it with a commercially available phone holster. During the familiarisation period participants were questioned about whether they could sense the vibrations of the phone. In case of a negative response, the phone was repositioned on the anterior part of the lower arm.

For the Classic variant (see below) participants were asked to start approximately 10 m before the first cone. For all other feedback variants, the trial started when the participant reached an initial target speed of 7 km·h⁻¹. The participants then had to complete 200 m stages, while the speed increased by 0.5 km·h⁻¹ with each stage. A trial was completed (irrelevant of where the athlete was) when one of the following criteria was met: either the athlete reached 90% of their self-reported maximal heart rate or the participant stopped running. Between two trials, a break of at least 20 min was implemented to allow participants to recover from the previous run.

Post-trial participants were asked to subjectively rank the feedback variants according to their personal preferences.

2.3. Feedback variants

In this study, four different feedback variants were examined a conventional variant (Classic) and three novel variants (Vibration, Sound and Sound & Vibration).

The Classic variant resembled the conventional method of the Conconi test. Cones were placed around the running track at a distance of 20 m between each other and correct positioning was ensured, as the track was equipped with markers for that specific purpose. The feedback consisted of an acoustic (beep) signal which was issued at predefined intervals. Participants were instructed to be at the next cone whenever a beep was sounded. At the beginning of the trial participants were instructed to start a few metres behind their first cone. An automatic countdown was applied to assist participants in their spatio-temporal adjustment before the first cone. Furthermore, participants were instructed to adjust their speed in order to reach the respective cone with their hip whenever a beep sound was signalled.

While the Classic variant had only one feedback signal, each of the novel feedback variants consisted of two different feedback signals: one for running slower and one for running faster than the target speed, while no feedback was given when participants were running at the target speed. A minimal interval of at least 5 s was ensured between any two consecutive feedback signals in order to avoid information overload.

The Vibration variant used two different vibration patterns for the feedback signals. The “too slow” signal consisted of 200 ms vibrations followed by a 150 ms break and the “too fast” signal consisted of 1000 ms vibrations followed by a 250 ms break. Each of these patterns was repeated three times whenever a participant did not run at a predefined pace.

The Sound variant used the Text-to-Speech software of the phone to provide “verbal” feedback. Since participants were expected to be fluent in German, commands were given in this language. The voice commands were “Zu langsam” (“too slow”) or “Zu schnell” (“too fast”) for signalling deviations from the target speed. Since Text-to-Speech software relies on pre-installed voices to convert text to speech, the phones were configured to use the pre-installed voice provided by Google (Google LLC, USA).

The Sound & Vibration variant combined the feedback signals of the two variants Sound and Vibration. Owing to that the phone vibrated with the same patterns as described above while simultaneously giving verbal feedback whenever a participant was outside the predefined pace.

Additionally, we also examined the participants’ preference of the feedback variant in order to identify possible individual preference effects on adherence.

2.4. Adherence

We defined adherence of a trial as the relative amount of time spent within ± 0.3 km·h⁻¹ of the assigned target speed. Previous pilot trials revealed that 0.3 km·h⁻¹ deviation from the target speed was sensitive enough to detect inaccuracies at a given running pace. Both feedback and calculation of adherence were based on the filtered (five-second rolling average) speed data sent from the Foot Pod device. For data cleaning purposes, the first and last – potentially incomplete – stage was removed from the analysis.

2.5. Statistics

Calculations for this study were conducted with R version 4.0.3 (R Core Team, 2020). Effects were assessed using repeated-measures ANOVA. Sphericity was tested using Mauchly’s Test. The Greenhouse Geiser correction was used when sphericity could not be assumed. Effect sizes were calculated using the partial eta squared (η²). We used post-hoc analysis using Bonferroni correction in case of significant main effect.

3. Results

We found a significant main effect of feedback variant on adherence (F(3,141) = 41.45, p < .001, η² = .268) (Figure 1). A subsequent Bonferroni post hoc analysis revealed significant differences between the feedback variants (Figure 1). The Classic variant (39.51 ± 13.51%) produced lowest adherence and differed significantly from Vibration (53.24 ± 14.98%): p < .001), Sound (60.34 ± 11.21%): p < .001) and Sound & Vibration (59.17 ± 15.17%): p < .001 (Figure 1). Moreover, Vibration differed significantly from Sound (p = .010) and, additionally, a tendency for significant difference was observed between Vibration and Sound & Vibration (p = .051) (Figure 1). However, no difference was found between Sound and Sound & Vibration (p > 0.999) (Figure 1).

Figure 1. Boxplot of adherence achieved with the feedback variants during complete trials (n = 48). *** significant difference (p < .001). ** significant difference (p < .01)

In order to have a consistent number of participants at each target speed, trials were only included up to the 10kmh⁻¹ stage. We found a significant effect of target speed on the adherence in the four different feedback variants (F(5,235) = 9.04, p < .001, η² = .098, F(5,235) = 4.65, p < .001, η² = .054, F(5,235) = 9.80, p < .001, η² = .119, F(5,235) = 5.61, p < .001, η² = .065 for the Classic, Vibration, Sound and Sound & Vibration variants, respectively; Figure 2). To further examine changes in adherence over the course of the trials we compared the coefficient of variation (CV) of adherence in relation to target speed. This analysis provides more insight on the reasons for low adherence: high variability can induce low adherence. A significant difference in CV of adherence between target speeds was found (χ²(5) = 14.286, p = .014) using Friedman’s test (Figure 3).

Figure 2. Changes in adherence with increasing speed for the different feedback variants. Each sub figure shows the mean adherence (y-axis) at a given target speed (x-axis) for a feedback variant (n = 48). Data was only included up to the 10kmh⁻¹ stage in order to have an equal amount of values per target speed. *** significant difference (p < .001), ** significant difference (p < .01), * significant difference (p < .05)

Figure 3. Coefficient of variation (CV) of adherence for different target speeds

No effect of trial on adherence could be identified using a repeated-measures ANOVA (F(3,141) = 0.25, p = .865, η² = .003). The mean adherence for the respective trials was (M± SD): 52.90 ± 16.33% in the first, 52.80 ± 16.41% in the second, 52.08 ± 18.09% in the third and 54.47 ± 13.27% in the last trial.

We found no effect of athletes’ participation in previous running tests on adherence (F(1,46) = 0.52, p = .476, η² = .007) and, furthermore, the results showed no effect of whether participants reported training at fixed speeds (F(2,45) = 2.11, p = .134, η² = .051).

A repeated measures ANOVA showed a significant effect of preference on adherence (F(2.49,117.16) = 3.31, p = .030, η² = .038) (Figure 4a). However, the post-hoc analysis did not show significant differences in adherence between the variants when ordered according to the preference of the athletes (Figure 4a). Nevertheless, a tendency to a difference between the first (48.60 ± 16.70%) and second preference (57.37 ± 16.86%) could be observed (p = .057) (Figure 4a). Feedback variants ordered according to the preferences of each participant individually are presented in Figure 4b.

Figure 4. Adherence of the different feedback variants ordered by preference and the corresponding rankings by the athletes. (a) shows differences between the feedback variants preferred by the athletes. Pref. 1–4 describes the order of feedback variant preferred by the athletes. (b) bars indicating the frequency of a feedback ranked in the respective preference (1–4) by the athletes

4. Discussion

The main result of this study was that a significant effect of feedback variants on mean adherence has been found. The three novel variants showed significantly higher adherence than the classic feedback variant.

A possible explanation for the low adherence of the classical variant could be the frequency of feedback. While the novel variants provide continuous feedback, the conventional variant provides feedback only whenever an athlete is at a specific cone. For example, at 10 km·h⁻¹ the maximal amount of feedback over 200 m could be 10 while with the novel variants it can reach ~14 or even ~19 signals at the lower target speeds (7.5 km·h⁻¹). This type of feedback is decoupled from the athlete’s pace as feedback is given at fixed time intervals and, as a consequence, athletes will receive feedback only in relation to a cone and not in relation to their actual pace.

Another explanation could be the way the feedback is used by the participants. While the novel variants do not allow athletes the choice of pacing since they provide a constant pace control, athletes using the conventional variant can control their own pace throughout the short duration interval. Each interval is 20 m long, and the participant is left with a choice on how to pace it. Owing to that some participants admitted (personal communication) having applied a strategy optimise cone timing in order to be at the next cone at the right time rather than running at a constant speed. As a consequence, the participants could have run faster or slower at some points during an interval or might even have had to wait at the cone. This could have resulted in a lower adherence to the classic variant.

The second important result of this study was the significant differences in adherence between different target speed levels. A common result for all variants was the increased adherence over the first three stages (observable in Figure 2). It appears that after the 8.5 km·h⁻¹ target speed a plateau was reached and no further increase in adherence could be observed. One explanation for the decreased adherence during the first three stages might be the preferred running speed of the athletes. Most athletes reported (personal communication) that they were not usually running at speeds as low as the target speeds of the first three stages. This can be supported by the fact that the calibration speed of the participants, which could be seen as the preferred speed, exceeded 10 km·h⁻¹. Consequently, a higher proportion of their attention could be placed on their running technique (Abernethy et al., 2002). We additionally hypothesised that due to the higher attention required when running at non-preferred speeds, participants were less accurate. To test this hypothesis, we additionally compared the coefficient of variation (CV) of adherence of the different feedback variants and target speeds. We expected that at target speeds lower than the preferred speeds the participants would adjust their pace more than they would at higher speeds, thus resulting in a higher variation in adherence. The results show significant differences in CVs between the target speeds (Figure 3). This is also apparent in the combined CV, where at the first two target speeds the coefficient of variation shows the highest values and then reaches a plateau (8.5 km·h⁻¹), indicating that there is probably a threshold or preferred speed where the speed variation is minimised. Therefore, we can conclude that the participants tend to run with less variation at their preferred pace rather than the non-preferred pace.

Next, to notice is that the pure vibration feedback had lower adherence on average than the feedback variants with sound (Sound, Sound & Vibration). The reason for this difference could lie in difficulties with sensing the vibrations or sensing differences between the vibration patterns. Some participants reported difficulties sensing the vibrations or distinguishing between the patterns during running. This can also be supported by the findings of the participants’ preference where the vibration variant was the least preferred among the four (Figure 4). However, it seems that the ability to sense vibrations depends on the task and individual since other studies reported no difficulties (Strohrmann et al., 2014). One factor that could contribute to the lower sensitivity might have been the clothing of the athletes as some participants wore a jacket during the entire testing session. Additionally, some participants further reported problems sensing differences between the two patterns, which could be related to the low pulse intensity produced by the phones. However, as the aim of this study was to find a simple way of providing feedback, and having additional devices such as an external vibration motor as previously described (Van Gelder et al., 2018) would have been beyond the scope of this study. Since other studies showed good results when using tactile feedback provided by an external apparatus (Afzal et al., 2019; Sheerin et al., 2020), future research should investigate the difference between phone vibrations and vibrations provided by dedicated external motor devices on pace adherence.

Another important finding of the present study is that the results did not yield significant differences between the trials. Firstly, this may suggest that the recovery time between the trials was sufficient for participants and thus the adherence was not decreasing due to fatigue. Secondly, this indicates that the trial order of the participants did not affect the outcome of the study. Additionally, it can be assumed that none of the feedback variants provoked a significant learning effect for the participants and, therefore, we can conclude that the results are not skewed by participants who might have prior experience with running tests. Furthermore, it can be assumed that the pacing behaviour is not altered by a single session.

A further finding from this study shows an effect of subjective feedback preference on adherence was found. This can be observed in Figure 4b where the first preference of participants was the Classic variant, which provided the lowest overall adherence (Figure 1). Additionally, the Vibration variant was the least preferred among the variants (Figure 4b) although it showed higher adherence than the classic one but less than the other novel types. These results could indicate that the problems associated with sensing the vibration (see discussion above) could play an important role in the test procedure and hence influence the results. Since the Classic variant showed the least adherence and the Vibration variant was the least preferred, we could suggest that the other novel types (Sound, Sound & Vibration) should be chosen for incremental performance speed tests.

5. Limitations

One limitation of this study is the lack of a true speed measurement (e.g. timing gates) since it was only assessed using a Foot Pod device. It can be argued that the Foot Pod can only accurately measure the speed for speeds close to the calibration speed, while for other values (lower than the target speed) the device would over- or underestimate the measured speed. However, since the average calibration speed of the participants was 10.59 ± 1.27 km·h⁻¹ (95% confidence interval: 10.22 − 10.96 km·h⁻¹), which was higher than the maximal target speed (10 km·h⁻¹), any possible systematic error (under- or overestimation) in speed measurement would not affect the main outcome of this study, which the higher feedback frequency of the novel variant required the participants to keep the scheduled pace and hence increase the adherence.

The validity and reliability of the speed values calculated by the used Foot Pod device may constitute a limitation of this study. Whereas Mahendran et al. (2016), Willy et al. (2016), and Han et al. (2020) report a high reliability or accuracy of the steps count, there is a lack of respective studies on speed values. However, since the same approach was implemented for all participants and feedback variants, a systematic error should probably not change the main outcome of this study, that is, the novel feedback variants showed a better adherence than the classical method.

Another limitation could be the nature of the task used for the evaluation of the feedback variants. Due to the stepwise increase in target speeds the results might not apply to other tasks such as variable pacing on uneven tracks. However, the protocol was chosen in order to reflect typical field testing scenarios. Nevertheless, future work should investigate whether the findings of this study apply to variable pacing as well.

Finally, the authors would like to point out that the testing protocol was based on the Conconi test just for simplicity. This study neither proves nor refutes the validity of the original protocol. The presented feedback variants could also be applied to any other protocol where running at a fixed pace is required.

6. Conclusion

The results of this study suggest that mobile feedback systems might be a valuable tool for athletes and coaching staff to conduct (fitness) tests due to their increased adherence. Moreover, as a result of their ease of use they might decrease the time required for setting up the testing site and can also reduce the effort required for data collection and further processing.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Hochschuljubiläumsstiftung der Stadt Wien under Grant H-327152/2018 [H-327152/2018];