Real-time feedback by wearables in running: Current approaches, challenges and suggestions for improvements

ABSTRACT

Injuries and lack of motivation are common reasons for discontinuation of running. Real-time feedback from wearables can reduce discontinuation by reducing injury risk and improving performance and motivation. There are however several limitations and challenges with current real-time feedback approaches. We discuss these limitations and challenges and provide a framework to optimise real-time feedback for reducing injury risk and improving performance and motivation. We first discuss the reasons why individuals run and propose that feedback targeted to these reasons can improve motivation and compliance. Secondly, we review the association of running technique and running workload with injuries and performance and we elaborate how real-time feedback on running technique and workload can be applied to reduce injury risk and improve performance and motivation. We also review different feedback modalities and motor learning feedback strategies and their application to real-time feedback. Briefly, the most effective feedback modality and frequency differ between variables and individuals, but a combination of modalities and mixture of real-time and delayed feedback is most effective. Moreover, feedback promoting perceived competence, autonomy and an external focus can improve motivation, learning and performance. Although the focus is on wearables, the challenges and practical applications are also relevant for laboratory-based gait retraining.

KEYWORDS:

• Framework
• gait-retraining
• personalised
• motor learning
• sports watch
• running

1. Introduction

Running is one of the most popular sporting activities, but also an activity with high discontinuation rates (Baltich, Emery, Whittaker, & Nigg, 2017). Running-related injuries and lack of motivation are common reasons for discontinuation (Clough, Dutch, Maughan, & Shepherd, 1987; Fokkema et al., 2019; Janssen, Scheerder, Thibaut, Brombacher, & Vos, 2017; Koplan, Rothenberg, & Jones, 1995). When individuals stop exercising, their risk of developing various psychophysical health conditions increases (C. S. Chan & Grossman, 1988; I. M. Lee et al., 2012). Running injuries have also been associated with failure to start and maintain a physically active lifestyle (Sallis, Hovell, & Hofstetter, 1992). Prevention of running-related injuries and maintenance or improvement of motivation are therefore of major importance to reduce discontinuation and maximise the psychophysiological health benefits of running.

The development towards more unorganised sports participation (Krouse, Ransdell, Lucas, & Pritchard, 2011) has been accompanied by an exponential increase in the availability and use of running wearables such as smartphone applications and sports watches (Janssen et al., 2017). These wearables can measure physiological and biomechanical variables and provide (real-time) feedback in an attempt to enhance performance, prevent injuries and improve motivation. Although the number of wearables that provide real-time feedback is rapidly growing, there are several limitations and challenges to current real-time feedback approaches. Two recent reviews have already discussed some challenges in using wearables for running injury prevention (Johnston & Heiderscheit, 2019; Willy, 2017). Johnston and Heiderscheit (2019) proposed a framework for a mobile monitoring system in running but did not specify how this framework could be used to reduce injury risk or improve performance and motivation. Further, Willy (2017) discussed the importance of quantifying biomechanical loading for injury prevention in runners and the associated technology, best practices, applications and challenges. However, the application of motor learning principles in real-time feedback was discussed only briefly, and the integration of individual motives in optimising feedback was not discussed, even though both aspects are also important for maximising the effectiveness of real-time feedback. Moreover, both reviews primarily focused on the use of wearables for injury prevention and did not consider the applicability of wearables to improve performance and motivation and thereby reduce discontinuation. Finally, several important challenges that clinicians, researchers and developers of wearable technology face when implementing real-time feedback were not discussed, which limits the applicability.

A framework that integrates different scientific fields, considers running from both an injury prevention and performance perspective, and provides practical implications can help clinicians, researchers and developers of wearable technology improve the application of real-time feedback and thereby increase its effectiveness on injury prevention and improvement of performance and motivation. However, such a framework is currently unavailable. Rather, most research that aims to reduce dropout is relatively narrow in focus and does therefore not consider the interaction and integration of all aspects in a holistic approach. In this review, we therefore integrate insights and empirical evidence from different scientific disciplines and propose a framework that can be used to optimise real-time feedback in running wearables. The overall aim of this framework is to reduce discontinuation by decreasing injury risk and improving motivation and performance (Figure 1). To this purpose, we first discuss why individuals run and how feedback can be better targeted to their motives to help maintain or improve motivation. We then discuss why and how real-time feedback of running technique and workload can be applied to reduce injury risk and enhance performance, thereby indirectly also improving motivation. We also review different feedback modalities and motor learning feedback strategies and discuss how these can be applied to more effectively apply real-time feedback. Finally, several important challenges in applying real-time feedback have not been addressed in previous reviews and we therefore also discuss challenges and provide suggestions on how to overcome them. Importantly, practical applications are provided throughout the review to facilitate applying the discussed topics.

Figure 1. Real-time feedback framework to reduce discontinuation in running.

Discontinuation (i) from running can be reduced by helping individuals to maintain or improve motivation (g) and by reducing injury risk (h). Real-time feedback from wearables has great potential to contribute to these outcomes. Specifically, wearables can provide personalised real-time feedback based on the individual preferences, experiences and motives to optimally enhance compliance and motivation (a). Further, real-time feedback on technique may help to modify technique, thereby reducing injury risk and improving performance (b). The improved performance may in turn also increase motivation by promoting the competence aspect of the self-determination theory. Running workload also has a strong relation with injuries and performance. Real-time feedback on the metabolic and/or mechanical intensity may help individuals exercise at an appropriate intensity, in line with the goal for the session to optimally enhance performance and decrease injury risk (c). Real-time feedback on the workload may therefore indirectly also contribute to an enhanced motivation. The dashed arrow between technique and intensity indicates that the technique will depend on factors such as speed and fatigue, while speed and fatigue will also depend on the technique used. This mutual relation should be considered when providing real-time feedback. Further, the motives of the individual will also partly determine how feedback about the running technique and exercise intensity is most effectively communicated as illustrated by the dashed line from motives to technique and workload. The dashed line between injuries and performance and motivation further illustrates that injuries will have a negative effect on these outcomes. Finally, to maximise the effectiveness of real-time feedback, it has to be communicated in a way that is understandable for individuals with no to minimal knowledge about biomechanics or exercise physiology and it has to be provided by appropriate modalities (f) and in line with motor learning strategies (d).

2. Motives to run and differences in preferred feedback content

Every runner has their own motives to run and these differ depending on gender, age, experience and running distance (Bell & Stephenson, 2014; Fosberg, 2015; Hanson, Madaras, Dicke, & Buckworth, 2015; Krouse et al., 2011; Kuru, 2016; Masters, Ogles, & Jolton, 1993; Ogles & Masters, 2003; Ogles, Masters, & Richardson, 1995; Rohm, Milner, & McDonald, 2006; Shipway & Holloway, 2013; Stragier, Vanden Abeele, & De Marez, 2018; Tjelta, Kvåle, & Shalfawi, 2018). The feedback content that each individual prefers differs depending on the motive(s) to run (Breedveld, Scheerder, & Borgers, 2015; Deelen, Ettema, & Kamphuis, 2018; Janssen et al., 2017; Stragier et al., 2018; Vos, Janssen, Goudsmit, Lauwerijssen, & Brombacher, 2016). Most wearables currently however assume that runners are interested in improving their performance (running faster and/or longer) and therefore provide generic performance-related feedback such as running speed or distance (Mueller, Tan, Byrne, & Jones, 2017). Personalising this feedback to the individuals’ motives may better motivate the individual and thereby reduce motivation-related discontinuation (Figure 1, box A). The motives to run (Hanson et al., 2015) and preferred feedback content may also differ between sessions (e.g., low-intensity vs high-intensity training) and change over a longer time span (e.g. (Clermont, Duffett-Leger, Hettinga, & Ferber, 2019; Kuru, 2016)). Enabling runners to customise their preferences is therefore important for personalised feedback and provides autonomy to the runner, which has further motivational benefits (see section 5.3). Table 1 provides an (non-exhaustive) overview of the preferred feedback content per motive and examples of their implementation in wearables.

Table 1. Running motives with their preferred feedback content and examples.

Running motives*	Preferred feedback content	Examples of implementation in wearable
Physical health	Physical health and/or weight related information	Estimated total number of calories burned (Temir, O’Kane, Marshall, & Blandford, 2016) or energy usage per minute Estimated physical fitness level (e.g., estimated VO2max as predictor of longevity and risk factor for developing adverse health conditions (Strasser & Burtscher, 2018))
Social motive	Social affiliation and/or recognition	Interacting via a smartphone and headphones with another runner that runs in a remote location and/or on a different speed (Mueller, O’Brien, & Thorogood, 2007; Mueller et al., 2012; Mueller et al., 2010; O’Brien & Mueller, 2007) Flying drone that serves as a jogging companion (Mueller & Muirhead, 2014, Mueller & Muirhead, 2015), which also can provide social support (Romanowski et al., 2017) Allowing others to show digital support on the wearable during running (Curmi, Ferrario, & Whittle, 2014; Knaving, Woźniak, Fjeld, & Björk, 2015; Woźniak, Knaving, Björk, & Fjeld, 2015) Displaying heart rate data or running pace to group members on the back of a t-shirt to facilitate group running (Mauriello, Gubbels, & Froehlich, 2014)
Achievement motive	Information on personal achievements and/or competition with others	Estimated performance capacity (product of fitness and fatigue) Running workload (intensity, duration, frequency) measures such as speed, heart rate and distance Estimated progress towards reaching a specific goal Comparison with estimated performance capacity of others (e.g., friends) Average running speed and distance in relation to others (for example, on online leaderboards such as Strava (Stragier et al., 2018)) Real-time competition via wearable with another runner in a remote location Gamification such as ‘Zombies, Run!’ that motivates participants to improve in-game performance (Moran & Coons, 2015)
Psychological motive	Psychological coping, self-esteem and/or life meaning related information	Cues that help focus on the running experience rather than on daily worries (e.g., “enjoy the nature around you”) Cues that help to feel more confident, proud of oneself or mentally in control of the body (e.g., “you have already run 3 km today, great job!”)

^*Motives to run are classified based on the categories adopted in the motivations of marathoners scale (Masters et al., 1993). Although other approaches have also been used to determine the motives to run, these motives can generally be grouped into one of the categories identified by the motivations of marathoners scale.

3. Real-time feedback on running technique

Numerous studies have related specific components of running technique to running injuries and running economy (Figure 1 box B & C) (Ceyssens, Vanelderen, Barton, Malliaras, & Dingenen, 2019; Moore, 2016), with the latter representing a proxy for performance. Running technique is therefore an important determinant of running injuries and running performance. Modifying running technique by real-time feedback may consequently reduce injury risk and enhance performance, thereby improving motivation and decreasing discontinuation. Indeed, a randomised controlled trial showed that eight laboratory-based gait (technique) retraining sessions with visual-based real-time feedback resulted in a lower injury rate during the 12-month follow-up (Chan et al., 2018). Although it is unknown whether real-time feedback provided by wearables is also effective at reducing injuries, recent studies provide indirect evidence for this notion (Baumgartner, Gusmer, Hollman, & Finnoff, 2019; Willy et al., 2016). Acute decreases in running economy have however been observed with running technique modifications (de Ruiter, Verdijk, Werker, Zuidema, & de Haan, 2014; Hunter & Smith, 2007; Snyder & Farley, 2011; Townshend, Franettovich Smith, & Creaby, 2017), suggesting that modifying running technique in an attempt to reduce injury risk may not be effective for enhancing running economy. In contrast to the acute decreases, short-term (1–14 weeks) gait retraining interventions can modify running technique without significant changes in running economy (Clansey, Hanlon, Wallace, Nevill, & Lake, 2014; Craighead, Lehecka, & King, 2014; Ekizos, Santuz, & Arampatzis, 2018; G. Fletcher, Bartlett, Romanov, & Fotouhi, 2008; Hafer, Brown, deMille, Hillstrom, & Garber, 2015; Messier & Cirillo, 1989). Acute detrimental effects can therefore be overcome or even lead to improvements in running economy over longer training periods. Both indirect evidence (De Ruiter, Van Daal, & Van Dieen, 2019; Moore, Jones, & Dixon, 2012) and direct evidence (Quinn, Dempsey, LaRoche, Mackenzie, & Cook, 2019) supports this idea.

3.1. Challenges in modifying running technique with real-time feedback

3.1.1. Which individuals benefit from real-time feedback on running technique?

Laboratory-based studies usually apply gait retraining to individuals that are currently injured or are believed to be at greater injury risk. Studies on currently-injured individuals show that real-time feedback can be effective to prevent injury- or pain-related discontinuation (Agresta & Brown, 2015; Dos Santos et al., 2019; Noehren, Scholz, & Davis, 2011). Similarly, gait retraining for individuals that were above a threshold shown to increase injury risk was effective at modifying injury risk factors (Bowser, Fellin, Milner, Pohl, & Davis, 2018; Napier, MacLean, Maurer, Taunton, & Hunt, 2018; Willy et al., 2016). These findings collectively suggest that real-time feedback on running technique can be relevant for individuals that are currently injured or at greater injury risk. In contrast, a recent study instructed all runners in the intervention group to reduce vertical impact and showed an overall reduced injury rate (Chan et al., 2018), suggesting real-time feedback on running technique can be relevant for all individuals.

Overall, we suggest that real-time feedback on running technique is primarily relevant for individuals with a current or frequently returning injury, or exhibit a technique that increases their injury risk. Novice runners have a greater injury risk (Buist et al., 2010; Kemler, Blokland, Backx, & Huisstede, 2018) and show larger differences between their preferred and optimal economical running technique (de Ruiter et al., 2014) compared with experienced runners. Novice runners may therefore benefit most from real-time feedback on running technique.

3.1.2. Which running technique components should be measured and modified?

Due to the growing number of biomechanical components of running technique that can accurately be measured by wearables, it becomes increasingly important to know which components are relevant to use in real-time feedback. In line with Phillips, Farrow, Ball, and Helmer (2013), we suggest that components are suitable for real-time feedback if they I) have a strong relation with injuries or running economy, II) can be measured accurately during various conditions, and III) are modifiable.

The strength of evidence for the relation of common biomechanical components with injuries and running economy is summarised in Figure 2. Real-time feedback can be provided on these components in an attempt to reduce injuries and improve running economy. For prospective studies on injuries, the inconsistent relations may be because laboratory-based studies have several limitations such as small sample sizes, a limited ability to measure the multifactorial nature of running injuries and they usually only determine the technique once before the follow-up, while technique can change during the follow-up period (e.g. (Shen, Mao, Zhang, Sun, & Song, 2019)). Data gathered in-field does not have these specific limitations and can therefore also be used to establish new relationships between running technique, injuries and performance (e.g. (Kiernan et al., 2018)).

Figure 2. Evidence heatmap showing the strength of evidence for the relation of several common biomechanical components with running injuries and running economy.

Figure 2. (Continued)

Figure 2. (Continued)

* The most commonly investigated biomechanical components from a recent systematic review on the relation between running technique and running injuries among prospective studies (Ceyssens et al., 2019) are included in this figure. Four additional prospective studies that were published after the search of the systematic review was finished were also included (Becker et al., 2018; Morris et al., 2019; Shen et al., 2019; Winter et al., 2019). The methodological quality of these studies was determined using a modified Downs and Black scale (Downs & Black, 1998) from Ceyssens et al. (2019) and can be found in supplementary file I. Briefly, the strength of evidence was classified as:

1) Strong evidence (dark green): Consistent findings among three or more studies, with a minimum of two high quality studies;

2) Moderate evidence (lighter green): Consistent findings among two or more studies, with at least one high quality study;

3) Limited evidence (light green): Findings from at least one high quality study or two low or moderate quality studies;

4) Very limited evidence (very light green): Findings from one low or moderate quality study;

5) Inconsistent evidence (blue): Inconsistent findings among multiple studies (e.g., one or multiple studies reported a significant association, while one or multiple studies reported no significant association). When findings were inconsistent, the usual (non-significant) direction of the association was specified;

6) Conflicting evidence (orange): contradictory results between studies (e.g. one or multiple studies reported a significant association in one direction, while one or multiple studies reported a significant association in the other direction);

7) No evidence (gray): No study has investigated the association with this variable.

A modified version of the quality assessment scale for cross-sectional studies was used to classify the strength of evidence for running technique with running economy. It is important to note that this is based on cross-sectional studies as there are only a very limited number of prospective studies examining changes in running technique and running economy (Lake & Cavanagh, 1996; Moore et al., 2012; Moore, Jones, & Dixon, 2016; Nelson & Gregor, 1976).

Accurate data are considered important by users of wearables (Clermont et al., 2019; Lazar, Koehler, Tanenbaum, & Nguyen, 2015; Rupp, Michaelis, McConnell, & Smither, 2016; Tholander & Nylander, 2015), in particular as training becomes more serious (Kuru, 2016). Wearables should therefore use validated and reliable variables in real-time feedback. Numerous studies have investigated the validity and reliability of biomechanical components derived from sensors such as accelerometers and pressure insoles and these components are also increasingly validated in settings that better reflect in-field conditions. Although many (mostly spatiotemporal) variables can be measured accurately, this is not true for all variables, for example, due to sampling frequency (Mitschke, Zaumseil, & Milani, 2017), operating range (Mitschke, Kiesewetter, & Milani, 2018) or sensor locations (Raper et al., 2018). Clinicians, design engineers, and researchers should, therefore, investigate if variables have been validated, preferably in conditions that reflect in-field use.

The final criterion is that a variable should be modifiable by the end-user. In running, almost all variables are modifiable, but some variables are likely easier and more directly to modify. It is, for example, easier to transfer to a forefoot strike pattern when the step rate can be increased at the same time rather than trying to adopt a forefoot strike while keeping the step rate at the baseline level (Huang et al., 2019).

3.1.3. When to modify running technique?

Deciding when to modify running technique can be done by establishing a reference range for each component and comparing values of the individual runner as established during several runs (e.g. (Ahamed, Benson, Clermont, Pohl, & Ferber, 2019; Benson, Ahamed, Kobsar, & Ferber, 2019)) to this reference range, with feedback being provided when a variable is outside the reference range for a specified time.

Elite athletes are often used to establish a reference range based on the assumption that they use an optimal technique due to many years of training. However, even if elite athletes use an optimal technique, their reference values and reference values from laboratory-based studies are largely specific to the context in which they are measured. Context-specific reference ranges can be established by collecting data in-field in a variety of conditions and these can be personalised by using runners with similar characteristics. However, especially novice individuals may not exhibit an optimal running technique from an economical and injury-risk reduction perspective and using novice runners with similar characteristics as reference is therefore also not appropriate. A solution could be to define cut-off values for components that are associated with a greater injury risk and/or poorer performance (Bowser et al., 2018; Napier et al., 2018; Willy et al., 2016).

The approach of using a reference range does implicitly assume that variability reflects an error and that there is an ideal technique that is similar for all individuals which should be pursued. However, this “one-size-fits-all” approach may not be optimal as each individual is believed to have a personal optimal technique due to anatomical differences (e.g. (Tenforde, Borgstrom, Outerleys, & Davis, 2019)) and previous running experience. Indeed, several studies have shown technique to differ between (Brisson & Alain, 1996; Glazier & Lamb, 2017; Gloersen, Myklebust, Hallen, & Federolf, 2018; Morriss, Bartlett, & Fowler, 1997) and within individuals (Glazier & Lamb, 2017; Horst, Eekhoff, Newell, & Schollhorn, 2017; Riza, 2017). It can therefore be questioned to what extent an “ideal” technique should be aspired, for example, by using deviations from the average movement by 1 standard deviation as a criterion for technique modification (Bowser et al., 2018). Nevertheless, we contend that using a reference range based on cut-off values from individuals with similar gender and anthropometrical characteristics can improve the technique more in line with a general “ideal” model that may reduce biomechanical loading and hence injury risk, and also improve running economy, while still allowing for individual variation.

3.1.4. How to modify running technique?

Running with a technique that is considered less injury-prone may instantly reduce the risk of several injuries. However, the biomechanical load will be distributed differently and hence load other tissues that may not be adapted to this load, thereby increasing injury risk. Changing from a heel strike to a forefoot strike, for example, increases plantar flexors and Achilles tendon forces, which may lead to plantar flexor strains and Achilles tendinopathy if these tissues are not accustomed to this load (Chan et al., 2018; Fokkema et al., 2019).

In novice runners, larger technique modifications can be achieved without substantially affecting running economy (de Ruiter et al., 2014). Tissues are however not fully adapted to the load and relatively small technique modifications are therefore recommended to prevent injuries. Even though tissues of more experienced individuals are likely better adapted, small modifications are also recommended to prevent large decreases in running economy and hence performance and motivation. A recent study developed algorithms that use a personalised “steepness curve” based on the physical profile of the runner and data from previous runs to individualise feedback (Aranki, Peh, Kurillo, & Bajcsy, 2018). Results from such studies may provide further insights into how quickly running technique can be modified.

4. Real-time feedback on running workload

The workload of a running programme is determined by the intensity, frequency and duration/distance. Rapid increases in running workload have been associated with injuries (Damsted, Glad, Nielsen, Sorensen, & Malisoux, 2018). Further, many recreational runners assume that running faster or longer is better and therefore tend to train at the same intensity every day, leading to a relatively monotonous training programme. This is in contrast to elite athletes that perform large amounts of low-intensity training alternated with fewer higher-intensity training and thus have more variation (Seiler, 2010). This training performed by elite athletes is likely more effective for improving performance than continuously training at a moderate to high intensity (Kenneally, Casado, & Santos-Concejero, 2018). Performing approximately the same medium- to high-intensity workout, every day has also been linked to a higher risk of illness and injuries compared to more day-to-day variation in load (Anderson, Triplett-McBride, Foster, Doberstein, & Brice, 2003; Foster, 1998; Piggott, Newton, & McGuigan, 2009). These findings collectively indicate that rapid increases in running distance or intensity and a monotonous training programme are suboptimal for performance and also increase injury risk. Wearables should therefore provide real-time feedback on the intensity and duration/distance of the run based on a pre-determined training goal to help individuals exercise at an appropriate intensity for an appropriate duration (Figure 1, box C).

4.1. How to quantify the workload?

Since running duration and frequency are relatively easy to quantify, we will not discuss these in detail. The intensity can be measured in various ways (Table 2) and it is therefore important to know which measures are relevant for real-time feedback. We suggest that variables are suitable for real-time feedback if: I) they have a strong relation to the actual metabolic and/or mechanical intensity, II) it is possible to accurately measure them, and III) they are modifiable by the athlete.

Table 2. Advantages and drawbacks of different intensity measures in running.

Intensity measure	Advantages and disadvantages	Validity (ability of the wearable to accurately measure the variable of interest)
Metabolic intensity
Rating of perceived exertion (RPE)	Rating of perceived exertion refers to how hard the exercise feels and is based on the idea that athletes can accurately monitor the psychophysiological stress during exercise and adjust the intensity accordingly. The perceived exertion does however not always correspond well to more objective markers of metabolic intensity (Borresen & Lambert, 2009), suggesting it may not always provide an accurate indication of the metabolic intensity. Further, novice runners in particular are not always able to accurately determine their perceived exertion (Tholander & Nylander, 2015), which may result in training at a lower or higher intensity than intended, thereby potentially leading to suboptimal performance and injuries.	n.a.
Heart rate	Since the introduction of chest straps, heart rate has been used to objectively quantify the internal load (Achten & Jeukendrup, 2003; Terbizan, Dolezal, & Albano, 2002). Heart rate shows an almost linear relationship with oxygen consumption at submaximal intensities and can therefore be used as a surrogate marker or to estimate the metabolic intensity during submaximal steady-state running, although individual differences and environmental factors prevent a very precise estimate (Achten & Jeukendrup, 2003; Borresen & Lambert, 2009). Further, heart rate takes approximately 2 minutes to reach a steady state and is therefore not a very accurate indicator of the metabolic intensity during high-intensity interval sessions.	Whether heart rate can accurately be measured with wearables depends on the used method. Chest straps are generally considered to provide an accurate indication of heart rate over a wide range of intensities, whereas optical (wrist-worn) heart rate monitors do generally only provide an accurate indication at slow to moderate running speeds (Lee & Gorelick, 2011; Stahl, An, Dinkel, Noble, & Lee, 2016; Støve, Haucke, Nymann, Sigurdsson, & Larsen, 2019; Thomson et al., 2019).Wrist-worn accelerometer-based estimates of heart rate have also been found to be accurate at low to moderate running speeds (Shcherbina et al., 2017). Other emerging technologies such as smart textiles are promising (J. W. Lee & Yun, 2017), but require further validation with larger samples.
Muscle oxygen delivery and utilisation	Near-infrared spectroscopy (NIRS) systems can be used to assess skeletal muscle oxygen delivery and utilisation and thereby potentially estimate energy costs during exercise. However, it remains largely unknown whether oxygen delivery and utilisation measured in a small area of a muscle can provide a valid indication of whole body energy cost, among others due to differences in blood flow between muscles and within muscle regions (Perrey & Ferrari, 2018). The findings of a recent study do however suggest that wearable NIRS measured at the vastus lateralis provided a more accurate indication of exercise intensity than heart rate during a run in hilly terrain (Born, Stoggl, Swaren, & Bjorklund, 2017). However, further research on the validity of this technique in other populations (e.g., overweight individuals) and at different muscle locations is required.	Multiple studies have estimated the lactate ‘threshold’ using (wearable) NIRS (Borges & Driller, 2016; Farzam, Starkweather, & Franceschini, 2018; Perrey & Ferrari, 2018). Although the lactate threshold was estimated accurately during running in one study (Borges & Driller, 2016), the accuracy of these estimates differs substantially between systems (Farzam et al., 2018) and can therefore lead to training at a too high or low intensity, thereby potentially leading to suboptimal performance and injuries.
Running speed	Running speed as measured by global positioning system is another frequently used indirect measure of the metabolic intensity, with the assumption that the metabolic intensity increases linearly with an increase in running speed and vice versa (Bransford & Howley, 1977). However, running speed does not always accurately reflect the metabolic intensity due to differences in training status, running surface, slope and weather conditions. Therefore, relying only on running speed as a marker of metabolic exercise intensity is not always appropriate.	Running speed derived from global positioning systems has generally found to be accurate (Hovsepian, Meardon, & Kernozek, 2014; Townshend, Worringham, & Stewart, 2008; Varley, Fairweather, & Aughey, 2012), but high accelerations that may occur during sprint-interval training may not always be measured accurately with wearables that use a lower sampling frequency (Scott, Scott, & Kelly, 2016).
Accelerometry	Tri-axial accelerometers implemented in wearables are increasingly used to estimate energy expenditure. Algorithms estimate this energy expenditure based on variables including the users’ height, sex, weight, exercise modality and sometimes also the heart rate, (Roos, Taube, Beeler, & Wyss, 2017; Shcherbina et al., 2017). The estimated energy expenditure can however differ substantially from actual energy expenditure due to differences in the algorithms (e.g., whether heart rate is incorporated (Montoye, Vusich, Mitrzyk, & Wiersma, 2018; O’Driscoll et al., 2018)) and inter-individual differences in energy expenditure even when other variables such as height and heart rate are similar.	Some studies show good agreement between energy expenditure estimated by accelerometer-based, wrist-worn wearables and gold-standard energy expenditure, while others show these wearables to exhibit a substantial error (Nuss et al., 2019; O’Driscoll et al., 2018; Shcherbina et al., 2017). The error generally increases with increases in running speed (Shcherbina et al., 2017) and since most studies used relatively low to moderate running speeds, the error may be larger for competitive runners or during high-intensity sessions. Further, the accuracy also differs between wearables (O’Driscoll et al., 2018). Overall, energy expenditure as estimated by wrist-worn accelerometers may therefore lead to incorrect intensity prescription depending on the device and speed used and should therefore be used with caution.
Running power	Based on the strong relationship between oxygen consumption and power in cycling, several wearables have incorporated algorithms to compute running power as surrogate of oxygen consumption and hence submaximal energy costs. An advantage of this metric is that it responds immediately to changing intensities in contrast to for example heart rate. While there is a relation between oxygen consumption during steady-state submaximal running and running power derived from a chest strap (Aubry et al., 2018) or foot pod (Austin, Hokanson, McGinnis, & Patrick, 2018), these relations are generally weak to moderate and power should therefore be used with caution as a surrogate measure of metabolic demands.	There is currently no generally accepted way to measure running power and this metric has therefore only been validated against measures of metabolic cost.
Mechanical intensity
Running speed	Increases in running speed lead to higher peak values in most biomechanical load-related variables (J. G. Hunter, Garcia, Shim, & Miller, 2019; Matijevich, Branscombe, Scott, & Zelik, 2019) and may therefore be used as a proxy of mechanical intensity. However, running technique, surface and incline may also affect the mechanical load on tissues and running speed alone does therefore likely not provide an accurate indication of tissue loading.	See running speed under metabolic intensity earlier in this table.
Foot pressure	Foot pressure can be measured using several wearable insoles that capture the force acting normal to the surface of each sensor (Burns, Deneweth Zendler, & Zernicke, 2018; Renner, Williams, & Queen, 2019; Seiberl, Jensen, Merker, Leitel, & Schwirtz, 2018; Stoggl & Martiner, 2017). Increases in insole pressure are often assumed to reflect increases in internal tissue loading, which is important to quantify for injury prevention. Matijevich et al. (2019) recently showed that ground reaction forces do generally however not correlate well with bone (tibia) loading and only have a small contribution to bone load magnitude. Some ground reaction force metrics were even negatively correlated to bone load and may therefore even provide misleading information in some situations. Although insole pressure does not correspond exactly to ground reaction forces due to the damping effect of the shoe (Barnett, Cunningham, & West, 2001), foot pressure derived from wearable insoles should therefore also be used with caution as a proxy of (internal) tissue loading.	Pressure insoles often aim to estimate ground reaction forces. The validity of pressure insoles to estimate vertical ground reaction forces differs between systems as these have been found to both overestimate (Burns et al., 2018) and underestimate (Renner et al., 2019; Seiberl et al., 2018; Stoggl & Martiner, 2017) vertical ground reaction forces compared to force platforms. Further, differences often become larger with higher speeds due to limited sampling frequencies of wearable insoles. The validity is therefore highly variable between different wearables.
Tibial/foot acceleration	An accelerometer attached to the tibia/foot is often used as a surrogate marker of ground reaction forces and tissue loading. Although accelerometers are easy to use and provide reliable metrics (Raper et al., 2018), it has been argued they do not provide a good indication of tissue (e.g., bone) loading (Matijevich et al., 2019) and similar to pressure insoles, they should therefore be used with caution as a surrogate of mechanical (internal) intensity.	Vertical ground reaction forces estimated from one accelerometer placed at for example the tibia do usually not provide a good indication of actual ground reaction forces, especially at higher running speeds (Raper et al., 2018; Verheul, Gregson, Lisboa, Vanrenterghem, & Robinson, 2018) Although different sensor placements and/or combinations of multiple sensors may provide a more accurate indication, the estimated mechanical intensity should be interpreted with caution.

The metabolic intensity of exercise is usually defined as the relative amount of energy expended per minute (kJ∙kg⁻¹∙min⁻¹) (Achten & Jeukendrup, 2003) or caloric cost per kilometre (kcal∙kg⁻¹∙km⁻¹) (J. R. Fletcher, Esau, & Macintosh, 2009). We use mechanical intensity to refer to tissue stress and strain as these are important variables for (mal)adaptation. Wearables can usually not measure energy expenditure or tissue stresses and strains directly and therefore rely on indirect measures. The advantages, drawbacks and concurrent validity of several intensity measures are discussed in Table 2. Overall, all intensity measures have their own advantages and limitations and there is therefore no single method that is most suitable for real-time feedback. A combination of different measures may provide the best indication of the actual intensity, particular for novice runners that still need to learn how the perceived intensity corresponds to an objective intensity measure such as heart rate (Tholander & Nylander, 2015). However, real-time feedback on running intensity can also be relevant for well-trained runners as it has been shown that (well-trained) athletes often train too hard during low-intensity sessions and not hard enough during high-intensity sessions (Foster, Heimann, Esten, Brice, & Porcari, 2001).

Finally, real-time feedback on heart rate data has helped runners to maintain pace (Kuru, 2016), suggesting runners can (easily) modify this variable. However, not all runners know at what intensity they should run and would therefore like to receive information on what to do (Kuru, 2016; Lazar et al., 2015). Real-time feedback should therefore specify whether the runner should try to decrease or increase the intensity based on the goal of the session rather than just providing numbers.

5. How to provide feedback?

Motor learning strategies and the frequency and modality of real-time feedback affects its effectiveness (Figure 1, box D & F). The next sections therefore briefly discusses these aspects.

5.1. Feedback frequency

The feedback frequency can influence learning and performance and can be categorised into several methods. We briefly discuss the most relevant methods for real-time feedback and their application. A first consideration regarding feedback frequency is whether feedback should be provided during (concurrent or real-time) or after running. Although most wearables measure various metrics during running, the data is often only made available after running, which limits the usefulness for modifying running technique and reducing injuries, adopting an appropriate exercise intensity or motivating the individual (Fokkema et al., 2019; Mueller et al., 2017; Tholander & Nylander, 2015). Real-time feedback is therefore often preferred over feedback after running, but feedback after running is complementary to real-time feedback (Clermont et al., 2019).

One real-time feedback method is continuous feedback, which involves feedback provision without interruption. Disadvantages of continuous feedback are that it can be perceived as annoying and that individuals can become dependent on the feedback, which hinders learning. Methods that provide feedback less often are therefore usually preferred. One of these methods is bandwidth feedback, which involves providing feedback only when performance (e.g., heart rate) falls outside of a predetermined range. Feedback frequency can also decrease over time, which is known as faded feedback. A final method is self-determined feedback in which the individual can self-choose when to receive feedback. This latter method has motivational benefits (see section 5.3).

The optimal feedback frequency depends on factors such as the individuals’ experience, difficulty of the skill that needs to be learned and specific feedback that is provided (Lauber & Keller, 2014; Wulf & Shea, 2002). Due to this complexity, only few general recommendations can be made. First, real-time feedback is generally preferred over delayed feedback, but both can complement each other. Second, changes in running technique can be maintained for at least 1 year after eight sessions of (laboratory-based) gait retraining (Bowser et al., 2018), suggesting only a few training sessions with faded real-time feedback can be used to modify the technique, while bandwidth feedback can be used after this initial phase to ensure the technique remains within a desired range. Finally, the feedback frequency for several existing wearable applications shown in Supplementary file II indicates that visual feedback usually involves continuous or self-determined feedback because the participant can self-determine when to look at a display. In contrast, auditory and haptic feedback are usually provided as bandwidth feedback. Visual feedback may therefore be a preferred method to combine with self-determined feedback, whereas auditory and haptic feedback may be best combined with bandwidth feedback.

5.2. Feedback modalities

Visual feedback is the most common feedback modality (Colley, Wozniak, Kiss, & Hakkila, 2018) and can be used in several ways (Supplementary file II). Although little research has been completed on the most effective way to provide visual real-time feedback (Sigrist, Rauter, Riener, & Wolf, 2013), this likely differs between variables and individuals. For example, although LED lights on shoes were effective at informing runners on their running pace relative to target pace, they were considered unsuitable for providing feedback about stride length and pronation (Colley et al., 2018). Visual feedback during running can overload visual perception and cognitive processing capacities, and when interaction with a device is required also distract from the environment, affect running technique (Seuter, Pfeiffer, Bauer, Zentgraf, & Kray, 2017) and lead to accidents (Kuru, 2016). Although it is therefore difficult to provide effective visual feedback during a “real-world” run, it can be an effective real-time feedback modality, in particular when used in combination with other feedback modalities and when it does not require frequent and long interactions.

Auditory real-time feedback can be provided as I) verbal information whereby the wearable/clinician provides spoken feedback, II) an auditory alarm whereby a sound without any modulation is played if a variable exceeds the predefined threshold, or III) using sonification whereby the error between actual and desired performance is indicated by varying auditory variables. All three types of auditory feedback have been effective at instantly modifying (running) technique (Eriksson, Halvorsen, & Gullstrand, 2011; Messier & Cirillo, 1989; Schaffert, Janzen, Mattes, & Thaut, 2019; Sigrist et al., 2013) and it has been shown that these acute effects can be maintained on retention tests without feedback (Schaffert et al., 2019; Sigrist et al., 2013). Examples of auditory feedback and their application in running wearables are provided in Supplementary file II. When used appropriately, auditory feedback requires no specific focus of attention and does therefore not have the disadvantages of distraction associated with visual feedback (Sigrist et al., 2013). The most effective way to provide auditory feedback also differs between variables and individuals (Mueller et al., 2017). With regards to different types of auditory feedback, a disadvantage of auditory alarms is that they provide no information on the degree to which the movement has to be corrected (Sigrist et al., 2013). Audification or sonification can provide such information, for example, by adding noise to music with further deviations from the target value (Lorenzoni et al., 2018). These latter forms of feedback are therefore generally preferred over auditory alarms.

Haptic real-time feedback is frequently provided as vibrotactile feedback. A recent systematic review (van Breda et al., 2017) concluded that vibrotactile feedback can maintain heart rate within the desired zone, but this conclusion was based on one study among one participant. No studies on vibrotactile feedback and running technique were identified. Although there are several applications of haptic feedback (Supplementary file II), the most effective way to provide this feedback during running has been subject of only limited research (Demircan et al., 2019) and requires further investigation.

Overall, all modalities can be used to modify performance instantly. In parallel, recent research (Agresta & Brown, 2015; Tate & Milner, 2017) suggests that laboratory-based auditory and visual real-time feedback can be effective at modifying the running technique. The most effective feedback modality differs however between variables and individuals (Ching et al., 2018; Eriksson et al., 2011; Jensen & Mueller, 2014). Real-time feedback is however only effective when the information is intuitive and correctly interpreted. Inappropriate use of real-time feedback hinders performance by reducing motivation, inducing distraction and leading to misinterpretation. Due to the small amount of research and conflicting findings, it is difficult at this point to provide general recommendations. Nevertheless, a combination of different feedback modalities is likely more effective than the application of one feedback modality (Sigrist et al., 2013) and generally also preferred by runners (Clansey et al., 2014; Eriksson et al., 2011; Vos et al., 2016). Regardless of modality, wearables need to provide feedback in an understandable way to facilitate use of the collected data as runners not always know how to use this without instructions (Kuru, 2016; Lazar et al., 2015).

5.3. Feedback content and motor learning

The recently proposed OPTIMAL theory of motor learning (Wulf & Lewthwaite, 2016) states that feedback is most effective at enhancing learning and performance when it promotes enhanced expectancies (and thereby intrinsic motivation), autonomy, and directs attention to the result of the movement rather than the movement itself. Learning a “new” running technique can be enhanced when these principles are applied in real-time feedback, whereas incorrect application may hinder learning. Table 3 therefore provides information on the relevance of these motor learning concepts for real-time feedback in running and implications for practice.

Table 3. Feedback content and motor learning principles.

Theory and evidence	Implications for practice
Competence
Competence refers to the feeling of experiencing oneself as capable and competent. Promoting perceived competence is important for motivation, learning and performance (Wulf & Lewthwaite, 2016). Providing positive feedback during/after successful performance, while ignoring less successful performances generally increases perceived competence and benefits learning and motivation (Chua, Wulf, & Lewthwaite, 2018; Wulf & Lewthwaite, 2016; Wulf, Lewthwaite, Cardozo, & Chiviacowsky, 2018). Continuously informing a runner of errors is therefore not optimal to increase perceived competence and hence motivation (Colley et al., 2018) and also not for learning because the runner is only informed about what is wrong and not how to correct it (Jensen & Mueller, 2014).	Provide positive encouragement when the variable of interest (e.g., heart rate, stride frequency) is in the desired range and refrain from continuously pointing out ‘errors’ to promote perceived competence.
Social-comparative feedback is a second strategy to promote performance and learning (Stoate, Wulf, & Lewthwaite, 2012; Wulf & Lewthwaite, 2016) and may be particularly relevant for individuals that compare themselves to others (Table 1).	Occasionally inform the runner that he/she is doing better than average (e.g., improving the technique or their performance faster compared to other individuals).
Decreasing perceived task difficulty is a third way to enhance competence and learning (Wulf & Lewthwaite, 2016).	Set a moderate bandwidth of what constitutes a good running technique or intensity, rather than a very small bandwidth in which the technique or intensity has to remain. Adaptive feedback strategies that set a lower target when an individual continuous to run outside of a reference bandwidth may prove beneficial to promote competence and facilitate compliance and adherence.
Autonomy
Autonomy reflects the ability to exercise control and numerous studies have shown that an enhanced perceived autonomy improves learning. Being able to choose when feedback is received, for example, led to enhanced learning in several discrete skills (Chua et al., 2018; Wulf & Lewthwaite, 2016; Wulf et al., 2018). Anecdotal evidence shows that runners also like to select the type of data provided as feedback (Kuru, 2016) and like to be in control of the extent to which they receive feedback (Mueller et al., 2010). Allowing runners to customise these aspects may help reduce the high rejection rate of wearables (Lazar et al., 2015; Nurkka, 2016; Rupp et al., 2016) and improve the attitude towards exercise (Kang, Binda, Agarwal, Saconi, & Choe, 2017). Further, higher levels of autonomy have also been associated with more frequent sports participation (Deelen et al., 2018).	Offer a variety of choices to increase perceived autonomy, for example, on the type, modality and frequency of real-time feedback.
Even if individuals are given choices that are irrelevant for the motor task, perceived autonomy and intrinsic motivation are enhanced (Iwatsuki, Navalta, & Wulf, 2019).	Also provide the runners with choices to modify less relevant variables such as the size and colour of the text in the display, the vibration pattern for haptic feedback or the auditory cues. Some wearables allow runners to select which metrics are displayed on the screen (Kiss et al., 2017) or to self-select a speed or cadence range within they would like to run and receive feedback if they are outside of this range (Aranki et al., 2018).
Autonomy supportive language leads to better motivation and learning compared to controlling feedback (Wulf & Lewthwaite, 2016).	Use autonomy-supportive language such as ‘try to increase your running speed for the last minute’ rather than controlling feedback such as ‘increase your running speed for the last minute’.
External focus of attention
The focus of attention can be broadly divided into an external focus on the intended movement effect, or internal focus, on the (coordination of) body parts or movement execution. An external focus generally leads to better performance and learning compared to an internal focus in a variety of skills, including running (Chua et al., 2018; Hill, Schucker, Hagemann, & Strauss, 2017; Schücker, Knopf, Strauss, & Hagemann, 2014; Schücker & Parrington, 2018; Schücker, Schmeing, & Hagemann, 2016; Wulf, 2013; Wulf & Lewthwaite, 2016; Wulf et al., 2018). It is however important to distinguish between an internal focus whereby the individual only monitors physical sensations or attempts to modify technique, with only the latter one being detrimental to performance and potentially learning (Schücker et al., 2014; Vitali et al., 2019).	Formulate feedback that promotes an external focus rather than internal focus on automated processes. Instructing a runner to increase knee flexion before ground contact may, for example, induce an internal focus, whereas instructing the runner to ‘land quietly’ may have the same biomechanical effect, but with a focus on the intended effect (external focus (Moore et al., 2019)).

6. Limitations and future directions

There are several limitations to this review and framework. First, we used a narrative search and may therefore have missed studies that could have been relevant, in particular for Table 1 and Figure 2. Due to the different topics addressed, a systematic search with clearly defined in- and exclusion was however considered unfeasible. Nevertheless, hand searching of reference lists and forward citation searching of included studies was used to minimise the potential of missing relevant studies. With regard to the framework, we acknowledge that a lack of time is also a common reason why individuals do not engage in, or discontinue with running (Clough et al., 1987; Fokkema et al., 2019; Janssen et al., 2017; Koplan et al., 1995). However, a perceived lack of time can often be related to cognitive errors (Locke, McKay, & Jung, 2019) and we contend that more personalised feedback can help to maintain or improve motivation and thereby help to make time for running. Further, factors such as participation in other sports, sleep, and daily life stress should also be considered when deciding on the most effective training programmes and hence feedback to reduce discontinuation.

7. Conclusion and practical applications

This paper proposed a framework that integrates insights and empirical evidence from different scientific disciplines to help clinicians, design engineers and researchers optimise real-time feedback in running with the overall aim of reducing discontinuation by reducing injury risk and improving performance and motivation (Figure 1). Practical applications to improve real-time feedback resulting from this framework are provided in Table 4.

Table 4. Conclusions and practical applications to optimise real-time feedback in running per category.

Motives
Individuals run for different reasons; Integrate the motives of the individual within the feedback to improve motivation and compliance.
Running technique
Real-time feedback on running technique has the potential to reduce injury risk and improve running economy; Novice individuals may benefit most from real-time feedback on their running technique because they are less likely do have developed a technique that is less prone to injuries and highly economical; Real-time feedback should be provided on a) components of running technique that have been associated with injuries and running economy (Figure 2), b) can be accurately measured, and c) are modifiable; A reference range based on cut-off values from individuals with similar gender, age and anthropometrical characteristics can be used to improve the technique more in line with a general “ideal” model that may reduce biomechanical loading and hence injury risk and also improve running economy; Small modifications in running technique over time are required for both novice and more experienced individuals to reduce injury risk and prevent large decreases in running economy associated with adopting a new running technique.
Workload
Rapid increases in running workload and a monotonous training program are not optimal for performance and also increase injury risk; Real-time feedback on the intensity and duration of the run based on a pre-determined training goal can help individuals exercise at an appropriate intensity for an appropriate duration and thereby reduce injury risk and improve performance and motivation; All workload measures have their own benefits and limitations and a combination of different measures will likely provide the best indication of the actual workload.
Modalities
The most effective feedback modality differs between variables and individuals; A combination of different feedback modalities is likely more effective than the isolated application of one feedback modality; Feedback needs to be provided in an understandable way to facilitate using the collected data as runners not always know how to use the collected data without further information.
Feedback frequency
Real-time feedback is likely more effective for modifying technique and workload and improving motivation than feedback after the run, but a combination of real-time and delayed feedback is optimal; Faded feedback can be used in the initial few weeks when modifying technique, while bandwidth feedback can be used after this initial phase to ensure the technique remains within a desired range; Visual feedback may be best to combine with self-determined feedback, whereas auditory and haptic feedback may be best combined with bandwidth feedback.
Motor learning
Perceived competence can be promoted by providing positive feedback, social-comparative feedback and by decreasing perceived task difficulty; Autonomy can be promoted by autonomy supportive language and offering a variety of choices, for example, on the type, modality and frequency of real-time feedback and also on less relevant variables such as the size and colour of the text in the display; An external focus can be promoted by providing feedback related to the movement effect.

Author contributions

BVH and SV conceptualized this manuscript. BVH drafted the manuscript. SV, JRV and JG provided feedback. SV obtained the funding for JRV and JG. All authors read and approved the final manuscript.

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplementary material

Supplemental data for this article can be accessed here.

Additional information

Funding

JRV was supported by the Netherlands Organization for Scientific Research (Grant P16-28 project 5), and JG by Interreg Vlaanderen-Nederland as part of the project Nano4Sports. The authors declare that they have no conflicts of interest and the funders had no role in the writing of this manuscript;Nederlandse Organisatie voor Wetenschappelijk Onderzoek [Grant P16-28 project 5];Interreg Vlaanderen-Nederland [Nano4Sports].