An insight into the use and assessment of lower limb running prostheses in sport with a disability: A mixed method approach

Abstract

In able-bodied sport, controversy has occasionally been caused through the adoption or introduction of sports technology. However, scant attention has specifically been paid to sport with a disability with respect to such concerns. This article therefore investigates the use of lower limb running prostheses (LLRPs) in competition by below-knee amputees. This study uses a four-phase mixed method approach to investigate the nature, use and assessment of LLRPs. The first phase conducted a statistical analysis of the sports time series data to ascertain the progression of the sport when considering the impact of sports technology. The second phase performed a stakeholder assessment of the sport using the Delphi method and provided a proposed series of guidelines for lower limb prostheses technology inclusion. The third phase assessed the behaviour of LLRPs in a competitive environment. Ultimately, the three previous phases provided information that led to the fourth phase. This phase investigated the use of a dynamic drop jump technique as an assessment strategy for further development in the future. This information would prove of intrinsic value when developing sports policy in the future.

Keywords:

• amputee
• design
• prostheses
• running
• sports technology

Public Interest Statement

Controversy has occasionally been caused when exciting new forms of equipment and technology are introduced into a sport. This can include everything from a ball or bat to aerodynamic bicycles and prosthetic limbs. This article investigates the recent adoption of high-performance running prostheses in sport with a disability. This study used a variety of qualitative and quantitative methods and established that the technology had made a noticeable impact when introduced. From this, a stakeholder evaluation then proposed new guidelines to help act as an assessment strategy for new technology in the future and to determine when such technology is unacceptable. Ultimately, a method was then proposed that would provide an objective means of assessing an athlete when wishing to use new forms of lower limb prosthetic limb designs. The information in this paper could assist governing bodies when forming sport with a disability policy in the future or when dealing with controversial forms of sports technology generally.

1. Introduction

Lower limb running prostheses (LLRPs) have provided the means for an amputee with a lower limb amputation to participate (Lewis, Buckley, & Zahedi, 1996) or facilitate competitive sport with a disability (Burkett, 2011). However, when designing prostheses for use in competitive sport, the emphasis is primarily on the athlete’s performance and results but not always with respect to the impact of its introduction.

LLRP running technology is a substitution for a biological limb that an athlete lost congenitally or via a traumatic incident. Thus, LLRPs are integral to the performance of an athlete (Nolan, 2008). Such technology is typically prescribed to an athlete based on their bodyweight coupled with selection of the appropriate stiffness category (Lechler & Lilja, 2008). Modern LLRPs are normally manufactured from a composite material (Nolan, 2008), are effectively a spring (Lewis et al., 1996) and of a passive nature (Nolan, 2008). The LLRP is unable to be more than 100% efficient (Nolan, 2008) seemingly putting it at a disadvantage when compared to the human foot region.

Sports and its use of technology have occasionally created controversy. Some of these have included issues surrounding tennis stringing patterns (Gelberg, 1998), the material of vaulting poles (Haake, 2009), the use of strapping in weightlifting (Holowchak, 2002), full body swim suits (Burkett, 2011), speedskate design (van Hilvoorde, Vos, & de Wert, 2007) and golf club design (Gardner, 1989). Historically though, little attention had been paid to the regulation of LLRP technology until 2008. It was at this point that bilateral amputee Oscar Pistorius attempted to qualify for the Olympic Games using LLRPs. The outcome of the ensuing investigation into Pistorius (Bruggemann, Arampatzis, Emrich, & Potthast, 2008) raised conflicting opinions on how prostheses in sport should be regarded generally and what contribution the technology initially assumed to be purely restorative contributed to an athlete’s performance. Ironically, accusations with respect to prostheses technology were later made by Pistorius himself and also American Jerome Singleton in post-race BBC interviews at the London 2012 Paralympic Games. Whilst this debate remains contested and conflicted (Grabowski et al., 2009), any concerns to date have mainly centred on Pistorius attempting to cross over from Paralympic to Olympic arenas. However, given that the Pistorius case highlighted debate surrounding the function of prosthetic limbs, plus observations that performance can differ greatly between designs (Nolan, 2008), there are viable claims for concern that technological disparity may be evident specifically with sport with a disability (Dyer, Noroozi, Redwood, & Sewell, 2010). However, such claims have not to date been investigated extensively. Such an investigation would be of value to a governing body if any development of sports policy was required in the future. As such, clarity into whether a problem exists would seem of timely value.

The evidence for the inclusion of any sports technology is often attempted to be resolved in academic areas of study such as sports philosophy (Lavin & Morgan, 2007) or sports engineering (James, 2010). However, applying these perspectives independently of each other does have limitations. Sports ethics can provide a wide range of viewpoints but only moralise “on paper” (Kretchmar, 1984). Likewise, sports engineering can provide empirical evidence to address a stated controversial issue, yet typically does not discuss the social ramifications of its findings in a sport (Bruggemann et al., 2008). Due to the discussed limitations of a purely qualitative or quantitative approach, a mixed method research (MMR) process was proposed as a viable alternative as part of this research process. This paper reviewed amputee sprinting performance to ascertain if any objective data provided any evidence of potential unfairness or significant technological impact. From this, further methods were employed to provide legislative guidance and a proposed solution to monitor technological impact in the future. To date, this kind of pragmatic approach has not been attempted within peer-reviewed literature.

2. Methods

MMR is based upon the concept of triangulation—“a combination of methodologies in the study of the same phenomenon” (Johnson, Onwuegbuzie, & Turner, 2007).

MMR has been defined as the application of different approaches when applied at any or all stages of a research project (Bazeley, 2004) or in a single study (Driscoll, Appiah-Yeboah, Salib, & Rupert, 2007). It has also been suggested as the logical alternative to the unproductive debate over the advantages or disadvantages of qualitative or quantitative research methods (Yvonne Feilzer, 2010) and has become increasingly popular as the means to combine the best elements from both (Doyle, Brady, & Byrne, 2009; Lingard, Albert, & Levinson, 2008; Morgan, 1998). MMR’s primary philosophical advantage is one of pragmatism (Yvonne Feilzer, 2010) and a high regard for reality (Johnson et al., 2007). It allows researchers to be more flexible and holistic in their investigative techniques (Onwuegbuzie & Leech, 2004).

In this study, a Development MMR approach was used, as defined by Onwuegbuzie and Leech (2004) and Bryman (2006). This is whereby the findings from one method help inform the other method prior to their design in either a sequential or subsequent parallel action format. A sequential four-phase method took place which is then illustrated in this paper. Institutional ethical approval was obtained for all phases of this research.

2.1. MMR phase 1

Prior to the research project being conducted, a primary research exercise was conducted to complement the literature review and to establish the validity of whether there could be any potential controversy surrounding the use of LLRP in sport with a disability. As a result, performance data were reviewed from the longest standing and consistently held running event of athletes with a lower limb amputation. This was deemed to be the men’s 100-m sprint performed at the Paralympic games from 1976 to 2012.

Performance data have been used as a means to imply some level of societal change in sports events (Balmer, Pleasence, & Nevill, 2011; Foster, James, & Haake, 2010; Haake, 2009; Munasinghe, 2001). As a result, an abnormal change in athletes’ performances may indicate sport-specific technological change (Haake, 2009).

Whilst several mathematical methods have been used to model athlete performances (Watts, Coleman, & Nevill, 2012, Foster et al., 2010; Munasinghe, 2001), few have been proposed to investigate the specific impact of sports technology (Balmer et al., 2011; Haake, 2009). The Balmer et al. (2011) method uses double logistical growth as a means to determine the impact of technological innovation. However, the Haake (2009) method is more specific as it considers the actual physics that applies to a sport and then uses any significant changes in event-specific physics to act as an indicator of technological impact. The Performance Improvement Index (PII) primarily assesses sports performance change from one date to another using the results of an event (Haake, 2009). It ultimately identifies proportions that may be attributed to sports technology and provides a metric which forms a direct comparison to other sports that rely on different metrics (such as speed, distance, or time). The PII has also been used to explore the impact of the First and Second World Wars upon running short, middle and long distance world records (Foster et al., 2010), and on the impact of technological innovation in Olympic field jumping events (Balmer et al., 2011). The PII cannot currently identify the exact proportion of impact of sports technology change, but it has been shown to corroborate anecdotal evidence of change such as the inception of new materials used for the pole vault or a change in the aerodynamic design of bicycles (Haake, 2009). As a result, it is a complementary tool to inform debate rather than to generate substantive findings.

When considering timed events over fixed distances, Haake (2009) defines the Performance Improvement Index as:$$\text{Index}=\left[{\left(\frac{t_1}{t_2}\right)}^2-1\right]\times 100$$

This formula has been derived as part of a linear regression from a larger formula (Haake, 2009). This formula addresses work done by a body overcoming aerodynamic drag when moving and for a fixed air density. It comprises a first timed performance t ₁ divided by a second performance t ₂. The rest of the formula converts the change between two performances and expresses it as a percentage.

This formula assumes events requiring motion to be dominated by aerodynamics. However, it should be noted that air resistance increases exponentially as speed increases (Kyle, 1991). The proportion attributed to the air resistance of a 100-m sprinter running at 10 m/s is going to be proportionally less than a cyclist performing a 4-km individual pursuit at 14 m/s. As a result, the magnitude of the PII index may be skewed if comparing events that result in different absolute and average speeds.

The mean averages of the fastest three runners in each successive Paralympic Games were used as a basis for assessment. These fastest three athletes would be those who would “podium” thereby winning bronze, silver or gold medals. By considering a mean of a group of athletes rather than outright world records or event winners, outliers or generation defining athletes were reduced in impact (which could skew any analysis). However, the full field of a final was not used as it was an issue that the early Paralympic Games saw incomplete eight person finals and a relatively novice/developing standard of competition. By limiting this sample to only the medallists, it was considered that those that may have been merely attending and having not needed to formally qualify to be of a competitive standard (unlike able-bodied athletes at the Olympic Games) was felt would reduce skew in the data. The three medallists provided a consistent and accurate snapshot of an events standard.

The results of amputee sprinting at the Paralympic Games were gathered and compared to the nearest able-bodied equivalent, the Olympic Games. Assuming prosthetic limbs are intended to be non-performance enhancing (Burkett, 2011), this comparison was undertaken to provide a frame of reference for any level of statistical change detected. Both competitions take place every 4 years, are accessible by the same countries of athlete origin and since 1992 have taken place at the same venue using the same facilities.

Three data-sets were compared:

•	Amputee sprinting (AS): the change from Paralympic Games to Paralympic Games over the 1976–2012 time period.
•	Able-bodied Modern Period (MP): the change from Olympic Games to Olympic Games over the 1976–2012 time period.
•	Able-bodied Inception Period (IP): the change from Olympic Games to Olympic Games over the 1896–1932 time period.

The AS to the MP comparison provided information that shows the performance of athletes over the same time period. The AS to IP comparison provided information that shows the impact of a new event developing at the highest elite level of competition. In the case of the able-bodied 100-m sprint, this took place nearly 100 years prior to the Paralympic Games equivalent.

2.1.1. MMR phase 1 results

The general trend progressions of the mean podium of all three time periods in this study are compared in Figure 1.

Figure 1. Performance improvement index of timeframes

It can be seen that at the start of the evaluated time period, the three traces are wide apart. However, by the end of the 36-year duration they have narrowed significantly. Whilst the MP group has seen relatively little progression, the AS mean time has been subjected to a substantial reduction in the mean podium performance.

The “games to games” PII changes of AS and able-bodied MP and IP over their 36-year periods can be compared together as shown in Figure 2.

Figure 2. Performance improvement index of timeframes

It should be noted that the gap in IP’s trace was due to there being no Olympic Games held in 1916. The magnitude of the change in the AS (T44) event is shown by the steep peak in 1988 (games no. 3). After this point, the AS rate reduces in magnitude until 2012. The IP timeframe also sees a slight increase at the end of its 30-year period despite relative stability prior to it. The MP period seemed to be in a state of relatively marginal improvement.

2.1.2. Discussion of MMR phase 1

Able-bodied sprinting has generally shown consistent, yet marginal improvements in performance. Amputee sprinting on the other hand has not directly followed the trends of able-bodied sprinting, neither with its original inception into the Olympic Games nor over the same timeframe from 1976 to 2012. A spike in 1988’s amputee sprinting results was clearly identified whereas able-bodied sprinting appeared much more stable. Whilst the PII value cannot specifically identify whether this is solely due to the known change in prosthetics technology, such information does corroborate the published introduction of LLRPs (Nolan, 2008). In addition, the magnitude of change is extremely large compared to any other Paralympic games to games increase over the evaluated period. This would suggest it is entirely plausible that the root cause is specifically related to changes in disability sports and technology. The rate of progression, whilst positive, has then generally decayed since 1988. An events decaying trend in performance improvement has also been witnessed before in the able-bodied 100-m world record (Foster et al., 2010).

It is shown that a PII of 21% occurred from 1984 to 1988. This is not as high as other values generated by Haake (2009) describing the changes in javelin technology (95% over 76 years), pole vault design (86% over 94 years) or the cycling one hour event (221% over 11 years), but these were all over much longer timeframes. Considering that the maximum change in able-bodied sprinting would be 24% over a 100-year period (Haake, 2009), the level of progression shown over a 4-year window could be regarded as somewhat unusual.

It has been demonstrated that of a typical historical performance improvement change, some of this can be attributed to clothing (Haake, 2009) or the social impact of war (Balmer et al., 2011), but the able-bodied athletes samples in this paper do not reflect any of the major spikes in PII that the amputee-based racing had demonstrated. Acknowledging that there were no changes in disability classification at the Paralympics between 1984 and 1988 which could account for the sudden rise in performance, this suggests that there were other causative factors involved which are unique to disability sprinting. Running performances themselves in the T44 event has continued to improve up until 2012.

While the methods used cannot conclusively point to the introduction of LLRPs in 1988 as being the root cause of change, such a distinct change in performance suggests technological change is a factor of importance in competitive running with an amputation. On this basis, the researcher determined that further investigation of the role and performance of LLRPs was warranted. Therefore, progression of the MMR process was continued.

2.2. MMR phase 2

With phase 1 establishing the need, the MMR process was continued to investigate and potentially propose a stakeholder driven set of values for how such technology is both perceived and used in running with a disability. The outcome of this study would provide a series of guidelines that would then be refined via further research phases.

Phase 1 used a consensus-based technique to identify the current perceptions of LLRPs. The research’s emphasis on providing insight into a novel area required the consensus technique to provide the insight and a catalyst that subsequent quantitative approaches would then be used (in parallel) to ultimately propose a solution to the overarching research question. A fuller detail of this study has been published in Dyer, Noroozi, Sewell, and Redwood (2011).

The method selected to obtain the perspectives of a number of different groups and stakeholders and experts was a consensus-based methodology, referred to as the Delphi Technique. It seeks to elicit expert (Martino, 1983), peer (Thompson, MacAuley, McNally, & O’Neill, 2004) or informed individuals (McKenna, 1994) opinion in a systematic manner. It is used to assist with decision-making and can also be applied when there is incomplete knowledge about a problem or phenomenon (Skulmoski, Hartman, & Krahn, 2007). It has also been employed whereby answers are vague, or subject to many interpretations (Chang & Kim, 2003) or as a way to determine priorities and alternative futures (Beech, 1999).

The Delphi method is a group technique with the aim to obtain the most reliable consensus of views of a group of purposively selected key informants, stakeholders or experts by means of a series of questionnaires which become progressively more focused. The process also involves controlled feedback to the respondents whilst maintaining their anonymity (Gupta & Clarke, 1996). The iterative process involves a series of “rounds” of questioning to the same panel. The subsequent findings from each previous round are then communicated back to the respondents and the scope for variation reduced in the next round in order to achieve convergence of opinion (Kennedy, 2004). The purpose of the technique is to produce consensus among a group of informed individuals using flexible methods, reiteration and statistical results. Whilst both interviews and focus groups could also be used to achieve the same objectives, the Delphi process has its unique advantages of its anonymity coupled with an ability to still achieve consensus on the issues at question.

The Delphi process in this study ran for three rounds. The response rate of the Delphi process was deemed as high. This rate was 100% from the initial contact email through to round one, 90% from round one to round two and 100% from round two to round three. It was felt though that since a technique would likely produce predominantly qualitative outcomes, this technique would likely require quantitative data reinforcement in subsequent phases as part of the MMR development approach.

2.2.1. Expert panel selection

The key principle of the Delphi technique is the recruitment and composition of an expert panel. The literature on the Delphi technique does not recommend one particular method to determine a credible number or type of “expert” used as this will depend on the topic under investigation and the context in which it is being undertaken. For doctoral studies, for example, expert panel size has varied from 8 to 345 participants (Skulmoski et al., 2007). As Bowles (1999) indicated, “Expertise is a valid construct but it is not easy to identify who possesses it”. In its use as a forecasting tool, a typical range of 4–20 experts have been used with occasional occurrences with panels as big as 98 (Rowe & Wright, 2001).

The experts for this study were selected through purposive sampling on the basis of their involvement with sport with a disability. Institutional ethical approval was obtained for the study before any contact of the respondents took place.

The number of the panel members included: prosthetists (3), disability sport academics (8), disability sport athletes (4), disability sport governing body member (1) and disability sport spectators (5). All expert panellists had direct experience of disability sport either through participation with or by academic publication within. In total 21 experts were selected for the panel.

2.2.2. Definition of consensus

The notion of consensus is difficult to define and thus problematic. Published research practice in the area suggests that the Delphi process continues through a number of rounds until a level of consensus is reached. It has been suggested that this is set prior to the study being undertaken (Fink, Kosecoff, Chassin, & Brook, 1984). A two-thirds majority was used for this study. This figure requires twice the level of positive agreement than for the views opposing it and has been recommended a suitable level for defining consensus (Fink et al., 1984).

For the present study, two key conditions were put in place to act as definitions of consensus:

(i)	A two-thirds majority is obtained from a line of questioning (66.6%).
(ii)	That the panellists failed to change their opinions on two consecutive rounds.

These conditions were defined in concordance with common principles which emerged as the literature of the Delphi technique was reviewed.

2.2.3. Delphi round 1

Round 1 asked three open-ended questions based on the core aims of the study. It was to seek the expert’s views on:

(i)	What role they felt a lower limb sports prostheses played in the context of competitive running.
(ii)	What they saw as fair or unfair when using a sports prostheses.
(iii)	What limitations they felt should or should not exist on such technology.

On completion of round 1 which achieved a 100% response rate the data were analysed and 17 themes were identified through a process of open coding. The limitations of the open coding process is that it has no preconceived process to follow (Walker & Myrick, 2006) so there is a risk of bias (Moghaddam, 2006). Any bias can be reduced when using the Delphi technique as it uses the panel to form consensus and conclusions rather than the researcher themselves.

Round 1’s data were ultimately developed into a set of 12 closed questions for inclusion in round 2 with the aim of gaining consensus.

2.2.4. Delphi rounds 2–3

Round 2’s closed ended questions used a 4-point modified Likert scale. The terminology used in the round 2 comprised the options “strongly agree”, “agree”, “disagree” and “strongly disagree”. This method was designed as a scale for assessing the respondent’s attitudes (Clason & Dormody, 1994). A neutral fifth choice was deliberately omitted in order to direct respondents towards a clear opinionated decision. A comments box was provided to allow the respondent to elaborate if they felt uncomfortable with this option.

As a result of round 2, several of the themes were removed from subsequent rounds when consensus had already been obtained. This process kept round 3 as short (and therefore as quick to complete) as possible. Upon conclusion of rounds 2 and 3, a series of statements, agreed by consensus of the stakeholder group were created.

2.2.5. MMR phase 2 results

Round 1 of the Delphi involved a qualitative coding analysis. From the open coding process, 17 common themes were identified. Each theme provided the basis for a question that was developed for the following round in order to elicit panellists’ responses.

Of the 12 close-ended questions in round 2, consensus of greater than 66.6% was obtained in 7 of the questions/statements. Five additional areas failed to gain enough consensus so these were further pursued into round 3 by question reformulation which were based upon the three themes from round 1. Round 3’s questions were refined based on both the comments and closed answers of round 2. After round 3, the consensus from the statements offered from rounds 1–3 were formulated into summary guidelines. These were:

The findings of the Delphi study were summarised as:

(1)	Lower limb prosthesis used for competitive running are classed as equipment and will be formally legislated.
(2)	Prosthetics technology not formally approved by the IPC should be submitted for evaluation to them ahead of any use of such designs in competition.
(3)	Lower limb running prosthesis are restorative in nature.
(4)	Lower limb running prosthesis will restore functionality to the user of no greater magnitude than that exhibited by the same region of the athlete’s biological lower limb.
(5)	Lower limb running prosthesis will contribute to a limb and stride length of no greater magnitude than that exhibited by the athlete’s biological lower limb.

2.2.6. Discussion of MMR phase 2

Points 1, 2 and 3 define the core philosophy behind LLRP’s proposed future use in disability sport. Points 4 and 5 are proposals for LLRP’s intended performance thresholds. These points are problematic as they do not resolve the participation of bilateral amputees. It is therefore suggested that the viability of a combined classification would need to be investigated further. Current International Paralympic Committee legislation addresses the maximum permitted prosthetic lengths, but the Delphi investigation notably from the athlete stakeholders suggested that such limitations should be taken when under racing conditions. Whilst the stride length has been investigated (Buckley, 1999), such research does not appear to place athletes under actual race-based conditions (with its associated emotional stresses and multiple competitors). As a result, further investigation into stride behaviour is warranted and should be undertaken before any proposal to monitor LLRP technology is proposed based on the Delphi’s guidelines alone. Point 4 proposed that same region limb-to-limb comparison should be undertaken i.e. the prosthetic prostheses is compared mechanically to a region of the limb it has replaced. This is feasible with single amputees but not possible with double amputees. Such limitations will require further investigations in the future with the simplest solution to determine whether single and double amputees are mechanically different and therefore for reasons of fairness should be raced separately. Such evidence has been inferred (Noroozi et al., 2012).

Whilst the Delphi study findings proposed direct limb-to-limb comparison, it did not specify whether the LLRP should in fact be attached when doing so. As a result, more insight into LLRP technology’s behaviour when running would seem to be useful is reducing the ambiguity of Point 4. In addition, there are no studies to date that investigate the characteristics of stride behaviour when under competitive conditions as required by point 5. Therefore, the research process next progressed through further investigation into points 4 and 5 of the Delphi study as part of the development ethos of MMR, before then proposing an assessment strategy.

2.3. MMR phase 3

The limitations of laboratory-based studies are that it is not known if such an environment is reflective of those experienced under actual competitive conditions. Such environments may alter an athlete’s behaviour due to the level of emotional stress or the inability to cope (Nicholls & Polman, 2007). It has been suggested that it is beneficial to ascertain if conditions simulated in the laboratory actually reflect those in real life (Lansley, Winyard, & Bailey, 2011) and this is of concern in sport with a disability (Curran & Frossard, 2012). This is proposed as the pursuit of ecological validity (Lansley et al., 2011).

To assess an athlete’s stride behaviour when under competitive running conditions, a quantitative-based analysis using video footage was undertaken. Video footage is the most feasibly accessible source of information. Real-time analysis or observation is not realistic due to the speed and number of athletes involved in any one event. In addition, TV footage utilises the best field of vision of an event that would not be easily possible as an observer. The footage used for this study was derived from public domain sources including Paralympics TV (http://www.paralympic.org/Videos) via Youtube (www.youtube.com). The identity of the athletes in each piece of footage is a matter of public record.

There are three competitive running distances that occur in the current Paralympic Games format. These events are the 100, 200 and 400-m running distances. However, the issue with such events and subsequent footage is that due to the switch between and panning of multiple video cameras, the same athletes do not always remain in shot or in clear view. This makes any assessment of their stride characteristics problematic. The only event which minimises such issues is the 100-m sprint event. As a result, only the 100-m event contested by below-knee amputees was feasible for evaluation in this study.

The video footage was assessed to ensure that its televised recording speed was the same as the actual events results and it was also assessed for its visual quality to allow clarity of an athlete’s ground impact. This ultimately meant only HD quality pieces had the desired resolution. The footage was then imported into the Quintic Biomechanics 9.0 software (Quintic Consultancy Ltd, Coventry, UK) which allowed frame by frame evaluation at the footages maximum specification of 0.04 s increments (25 frames per second).

Whilst several videos were found to be available, only seven HD quality pieces of video footage had the visual resolution it was felt to be examined in close enough detail to evaluate the footfall data accurately. Of these, four pieces of footage were of qualification heats of both the 2008 and 2012 Paralympic Games. However, these were discounted from this analysis as it was seen that several athletes intentionally slowed down before the finish line. It is assumed that this took place due to either an athlete had estimated that their qualification was already secured or that they felt this had not been achieved and therefore gave up. This meant their stride would sometimes be seen to visibly slow down or shorten in stride length towards the end of an event. This made such data not representative of the events maximal effort and was therefore rejected. In addition, the athlete’s reaction time to the starting pistol could not be evaluated as this exact moment was not within the unit of measurement of the analysis software or the footage.

The three suitable races evaluated for step frequency and symmetry were:

•	2008 Paralympic Games T44/43 100 m Final (http://www.youtube.com/watch?v=UDDhZx54Jy4).
•	2011 IPC World Athletics Championships T44/43 100 m Final. (http://www.youtube.com/watch?v=LTxypZ71-30).
•	2012 Paralympic Games T44/43 100 m Final. (http://www.youtube.com/watch?v=mcdUsMULNzo).

When reviewing the footage, a definition of ground impact was needed to record when a step had taken place. A ground impact was determined whereby the foot is seen to contact the ground just prior to the lower limb beginning to bend at the knee or the prosthesis is seen beginning to compress.

The greatest possible error of this evaluation is defined as half of the measurement unit. Therefore, potential errors in the step symmetry data were 0.02 s. The tolerance interval (or margin of error) is defined as ±0.02 over the established measurements. However, due to the relatively large imprecision (due to the limitations of the footage), the largest error possible was defined here as one measurement increment of ±0.04. Therefore, only a change of greater than ±0.04 from the previous data point is proposed to be significant in this study to then be defined as lower limb asymmetry. Due to this relatively large tolerance, interval typical calculations such as the symmetry index would be misrepresentative of asymmetry so were not used in this study.

The athletes were classified as having three types of lower limb behaviour. These were designated as lower limb to limb symmetry (LS), lower limb to limb asymmetry (LA) and random asymmetry (RA). LS was defined as a limb-to-limb timing within the measurement precision. LA was defined as a consistent limb-to-limb timing imbalance of greater than 0.04 s. RA was considered a single-event limb-to-limb timing imbalance of larger than 0.04 s.

2.3.1. Results of MMR phase 3

When reviewing the footage, three athletes gave an appropriate level of visibility in the 2008 event. There were also six from 2011 and four from 2012.

The three athletes’ lower limb-to-limb timing footfalls (Y axis) and number of steps (X axis) in 2008 are shown in Figure 3.

Figure 3. 2008 World Championships athlete gait symmetry summaries for athletes: (a) Pistorius, (b) Singleton and (c) Fourie

Figure 3 illustrates the time taken for a foot’s impact on the track to the alternate foot’s impact upon the track. Both Pistorius and Singleton exhibited a lower limb symmetry within the acceptable tolerance range of the study. However, Fourie demonstrated significant step to step RA in the first few strides of his race. It took him four steps to reduce to a more typical level of the LS seen with the other athletes. Fourie’s mid-section of his race shows an extremely symmetrical period of gait. Singleton demonstrated relatively consistent LS during his event. However, the last three steps of his event were slightly slower in duration.

The six athletes’ lower limb-to-limb timing footfalls (Y axis) and number of steps (X axis) at the 2011 World Championships are shown in Figure 4.

Figure 4. 2011 World Championships athlete gait cycle symmetry summaries for athletes: (a) Singleton, (b) Pistorius, (c) Oliveira, (d) Fourie, (e) Leeper and (f) Peacock

In this event it can be seen that bilateral amputees Pistorius (second) and Leeper (fifth) exhibit very high levels of lower limb symmetry. Singleton has brief RA at the start and again with his finish. Oliveira’s run was only visible for the first half of the event. This aside, he exhibited RA at his start and sporadically throughout the first half of the event. Pistorius had relative LS but his last stride saw a one-off RA. Peacock had extremely consistent initial LA until the latter part of the race whereby his gait reflected LS.

At the 2012 Paralympic Games, four athletes produced clear line of sight for evaluation. Their lower limb-to-limb timing footfalls (Y axis) and number of steps (X axis) are shown in Figure 5.

Figure 5. 2012 Paralympic Games athlete gait symmetry summaries for athletes: (a) Peacock, (b) Browne, (c) Fourie and (d) Pistorius

Peacock showed great improvement in terms of his actual performance from 2011 by winning this event. However, his run still demonstrated significant RA. Unlike 2011, this took place towards the middle rather than the start of his race. Browne’s run is perpetuated by RA throughout his event. Fourie exhibits RA after his start and towards the final stages of his run. Pistorius exhibits LS typical of both his 2008 and 2011 events.

The number of RA’s of athletes from the 2008, 2011 and 2012 events are summarised in Table 1.

Table 1. 2011 World Championship performance

Athlete	Year	Place	Amputation type	No. of RA events (>0.04 s)
Singleton	2008	2nd	Unilateral	0
Singleton	2011	1st	Unilateral	3
Pistorius	2008	1st	Bilateral	0
Pistorius	2011	2nd	Bilateral	0
Pistorius	2012	4th	Bilateral	0
Oliveira	2011	3rd	Bilateral	5
Fourie	2008	5th	Unilateral	10
Fourie	2011	4th	Unilateral	5
Fourie	2012	3rd	Unilateral	6
Leeper	2011	5th	Bilateral	5
Peacock	2011	6th	Unilateral	13
Peacock	2012	1st	Unilateral	16
Browne	2012	2nd	Unilateral	9

The finishing position does not correlate to the number of RA events. Peacock has improved his finishing position performance in the 100 m between 2011 and 2012 significantly yet still displays a large level of random RA between his two lower limbs.

2.3.2. Discussion of MMR phase 3

This study demonstrated that in the limited number of case studies available, randomised asymmetry behaviour did take place in elite 100-m competition.

It is conceded that the limited 25 frames per second is not a high enough resolution to detect the exact level of asymmetry. However, the degree of consistent LA here seen in running is similar to those reported in controlled studies such as Sanderson and Martin (1996) which saw lower limb step timing asymmetry of 0.02 s at 3.5 m/s and 0.03 at 2.7 m/s. However, this study saw random asymmetry events of up to 0.08 s. From this it could be concluded that athletes under race conditions can create occasional asymmetry that may not be reflected when running at either slower speeds, at a steady state or in non-competitive environments. Differences in prosthetic length as the root cause as reported by Hafner, Sanders, Czerniecki, and Fergason (2002) are unlikely to be the reason when the outcome is randomised asymmetry lower limb timing.

Despite such asymmetry, there are several cases of randomised asymmetry performed by athletes in races. This does not seem to be a barrier to success as Peacock went from finishing sixth to first from 2011 to 2012 yet still exhibited a similar level of RA. However, what might be more important is where he exhibited the RA. In 2011, it was at the start whilst he was trying to accelerate. Yet in 2012, it took place when he was already closer to a steady state speed therefore it is proposed that the net loss in speed would be lower. The impact of randomised asymmetry within the 100-m event and its impact on running speed are recommended for further study in the future.

The actual cause of RAs is unknown. However, in some cases, a root cause can be identified by qualitatively assessing the race footage. For example, Singleton in 2008 and both Singleton and Pistorius in 2011 demonstrated RA in the last few steps of their events. The reason for this could conventionally be assumed to be fatigue yet when looking at the video, both athletes were involved with a lunge for the line as both attempted to win. It is proposed that the action of lunging for the line slows the stride slightly as the runner positions their torso. Singleton exaggerated this by so much in 2011 that he fell forwards after crossing the finish line.

When reviewing other athletes mid race asymmetry, the root causes could not be identified. It is proposed that the cases of RA need to be evaluated qualitatively alongside the quantitative data. The RA’s at the beginning and end of races could be attributed to a poor or slow start with the finish RA being attributed to fatigue, falls or torso lunging. However, it is possible that mid race RA’s may not be the fault of the athlete but equally in them dealing with their technology and the constraints of the event such as remaining within the lanes.

2.4. MMR phase 4

The Delphi study produced a framework of guidelines for LLRP technology and was refined with the subsequent experiments. The limitations of any proposed assessment strategy is that it is not realistic to expect an athlete to perform a maximal, race-specific effort of an event like the 100, 200 or 400-m reliably. In addition, this would not be realistic as only one run could be feasibly realistic yet statistical robustness would ideally demand several runs. As a result, any proposal is ultimately one with limitations.

It is suggested that a dynamic test which will allow those assessing such technology the ability to compare limb to limb performance (as proposed in the Delphi study), obtain statistical reliability (as any scientific method would require) and encompass the mechanical qualities of the lower limb (as indicated in this papers literature review) to be proposed as the best solution.

Whilst run testing has been performed by amputees on treadmills, jump tests have been correlated to sprint performance (Holm, Stålbom, Keogh, & Cronin, 2008), can provide independent leg assessment (Strike & Taylor, 2004) and provide a “float phase” i.e. with neither feet touching the ground, as per running (Umberto, Massimiliano, Daniele, & Matteo, 2006). This paper proposes the feasibility of using uni-lateral jump testing as the means to assess LLRP technology.

The drop jump is used either as a training method to improve the physical ability of jumping or is used as a diagnostic assessment to measure jumping strength. In this case, a unilateral drop jump would allow the performance of a prosthesis to be compared and limited by the naturally generated performance of the athlete’s biological limb. The test is proposed as a unilateral (rather than a bilateral) drop jump as this will allow the left and right legs to be separately evaluated and then compared when subjected to pre load and propulsive forces (Stalbom, Holm, Cronin, & Keogh, 2007). The load upon the limb is provided by a freefall phase of the drop rather than the athlete’s efforts. This minimises the ability to “cheat” the test as the effect of gravity cannot be manipulated.

The bilateral or unilateral drop jump test is typically undertaken by having a participant stand on a platform of a given height above the ground. Once commenced, the participant jumps down to the floor landing on either one or (more typically) two feet and then immediately executes a vertical or horizontal jump on the same foot/feet (depending on the technique method chosen). The achieved displacement is then measured as the magnitude of muscular power. The height of the jump creates the running events expected magnitude of ground impact (thereby generating a context-specific lower limb stiffness response). The landing phase could be used to calculate lower limb stiffness. The launch phase can then be used to form the basis of energy generation limb-to-limb comparison. The advantage of this technique is that it uses a manipulation of drop height and the person’s weight to provide a ground impact to create the lower limbs appropriate stiffness response, as opposed to the athlete’s effort. Alternatively, by standardising the participant’s weight, drop height and impact of gravity means, that the potential energy can be quantified and standardised (in Joules) using this technique. An additional advantage is that the secondary, vertical launch phase can assess the energy return of the overall limb although it was conceded that due to the passive nature of the current acceptable level of technology, this may not be currently relevant.

Within sport science, the drop jump has been performed by sports performers. These have included physically active individuals, (Stalbom et al., 2007; Schot, Bates, & Dufek, 1994; Bobbert, Huijing, & Schenau, 1987) adolescent athletes (Barber-Westin, Noyes, & Galloway, 2006), elite-level biathletes (Krol & Mynarski, 2012), Norwegian national-level triple jumpers (Viitasalo, Salo, & Lahtinen, 1998), basketball and hockey players (Rishiraj et al., 2012) and men from team sports (Baca, 1999).

Several studies have attempted to identify the correlation between sprinting ability and jump testing performance using able-bodied participants to ascertain its ability as a diagnostic or talent identification test. Some studies have shown a direct correlation with bilateral jump testing and short-distance sprinting (Maulder & Cronin, 2005); however, this has been disputed (Kukoli, Ropret, Ugarkovic, & Jaric, 1999). However, these two studies both agree that vertical jump displacement and sprinting performance correlate significantly.

Little evidence has been seen of drop jump use when performed directly on lower limb amputees. Lower limb amputees have however performed other jump test variants such as uni-lateral vertical jumping (Strike & Diss, 2005) and a one-foot vertical jump with approach with unilateral trans-tibial amputees (Strike & Taylor, 2004). In the 2004 study, the maximum height and flight time were reduced noticeably on the prosthetic side but this study’s findings should be taken with a degree of caution as it was only performed with two participants. No studies of athletes with a lower limb amputation performing unilateral or bilateral drop jumps are currently evident within the literature. This also means that any differences in lower limb damping between an artificial sprung prosthesis and the natural spring characteristics of a biological leg have also not been investigated.

Drop jumping in practise does have several concerns as to its use. The magnitude of the height used has been cited as modifying the technique used to perform it (Bobbert et al., 1987), likewise upper-limb motion has been shown to affect the generated impulse (Laffaye, Bardy, & Taiar, 2006) meaning arm motion to stabilise height or create comfort to the participant of the dropping height would have to be removed. If this is not controlled, the effect of the arms may influence the results affecting its correlation to sprinting. It has also been suggested that to increase the jump height up to 60 cm creates net joint reaction forces with sharp peaks (Bobbert et al., 1987), as well as observations of an altered jump technique performed by the participants (Viitasalo et al., 1998). It has been proposed that there is a potential for injury in the ankle if 10 or more jumps are performed (Delahunt, Monaghan, & Calufield, 2006) and that fatigue will lead to changes in movement patterns in the lower limbs (Weinhandl, Smith, & Dugan, 2011). These observations may be less desirable or raise ethical issues. However, in terms of test competence, it should be noted that athletes with a lower limb amputation already performed the long jump event at the 2008 Paralympic Games (www.paralympic.org/Sport/Results/) and empirical assessment of jumping events have taken place (Nolan, Patritti, & Simpson, 2012).

Ideally, the height for the drop jump should also be optimised through testing to create a lower limb stiffness response applicable to the athletes running event—for example, a 400-m runner would see less stiffness and therefore drop from a lower height than the 100-m sprinter equivalent. This would require further investigation but if successful, could form the basis of a regulatory test.

2.5. Summary of MMR process

Whilst the research methods used as part of the MMR approach in this paper could have been undertaken in many alternative ways, all four phases here provided novel insight. Phase 1 established the investigative need. It could be argued that this is not typically required in research since a negative result can be equally useful. However, by doing so provided a pre-emptive approach to future discussion—if both able-bodied and disability-based competitive sport are considered equal (such as at the Olympic and Paralympic Games), there is no reason why amputee-based running competition would not see technology-based controversy at some point in the future too. However, whilst the methods used in phase 1were not conclusive, the data did infer that technological change was a factor of importance. This key outcome therefore established the need for phase 2. The Delphi study then produced a novel framework which provided further insight. However, it was felt beforehand that the qualitative information provided would require further reinforcement using quantitative data. This was therefore undertaken in phase 3. This method of reinforcement could have been used for all points of the proposed framework but it was felt that there was enough information generated from phases 2 and 3 upon which to propose a solution for future investigation as the key outcome of phase 4.

Potential criticism surrounding the use of a development MMR approach was that the nature, length and value of each phase of research would not be known at the inception of the project. However, what this philosophy did provide was a responsive, flexible and tailored research process utilising the strengths of both qualitative and quantitative approaches. As a result, this method would be suggested as proving useful when dealing with practical and multidisciplinary problems in sports related issues.

3. Conclusion

A four-phase mixed method approach was undertaken to investigate the use of lower limb prostheses in competitive running with a disability. An investigative need was undertaken through the evaluation of historical performance data. Due to an unusual progression of athlete’s performances, further research was therefore warranted. A stakeholder study was then undertaken to establish both insight and a framework of acceptability for running when using lower limb prostheses. These findings were further clarified and enhanced with insight into the stride behaviour of athletes using prosthetics technology when under competitive conditions. Finally, a novel jump-based assessment method was proposed as a pragmatic means of monitoring athletes with a lower limb amputation performance. This information would be of intrinsic value in lieu of any required future policy regarding LLRPs.

Correction

This article was originally published with errors. The reviewing editor’s affiliations were incorrectly listed as “Xian Jiao Tong University (China) and Leeds University, China” but the correct affiliations should be “Xian Jiao Tong University, China and Leeds University, UK”. This version has now been corrected.

Additional information

Funding

Funding. The authors received no direct funding for this research.

Notes on contributors

Bryce Dyer

Bryce Dyer is currently Head of Research & Professional Practice in the Department of Design & Engineering at Bournemouth University in the UK. He is a chartered scientist and a chartered technological product designer. His current research interest is with the philosophy, design or performance analysis of technology used in elite sport. His recent research projects have investigated the use of prosthetic limbs in both running and kayaking and he has designed high-performance prostheses used by elite cyclists for both the 2012 and 2016 Paralympic Games. The research in this paper attempts to address the significant controversy surrounding the use of running prosthetic limbs that arose in 2008 and continued through to the London 2012 Paralympic Games. However, it builds on previous research by using a range of research methods to help provide practical solutions to such controversy in competitive sport.