Control of active lower limb prosthesis using human-in-the-loop scheme

Abstract

In many cases robots are directly in touch with humans, this interaction is necessary to design control systems that take into account a human-machine relation. Particularly important is the case when humans share the control with a robot. So, in order to validate control schemes, it is necessary to test them in an environment where the human-machine interaction exists, what could be even dangerous for humans. We present a model of human useful to test controllers, and propose a scheme for human-in-the-loop under shared control. We test the model and the scheme in a lower limb prosthesis that implies naturally a human-machine interaction. Apart from this research, we propose an efficient and low-cost controller for the prosthesis, based on a sensory exoextension and a neuro-decoder consisting of a multilayer perceptrons. The results show the effectiveness of human model to emulate the human interaction.

Keywords:

• Human-model
• human-in-the-loop
• shared-control
• prosthesis

PUBLIC INTEREST STATEMENT

The proposed method for prosthesis is useful to study the human correction of errors in automatic control. The participation of the human for fine adjustments due to disturbances. The active lower limb prosthesis has the advantage that the amputee can freely change the desired trajectory and does not depend on the healthy leg. It does not require a complex sensory system. Only hip reading status is required for mapping. The study would mainly benefit in the areas of medical robotic, automatic, and computational intelligent.

1. Introduction

Only in Mexico, according to Cruz (2016), there are 128 thousand amputations per year. With what arises the need to restore the motor capacity of amputees, and with it their autonomy. Scientific and technological advances increasingly allow the development of robotic devices with better performance, see, Martinez-Hernandez et al. (2018). However, more work is needed to make these devices not only tools but also able to completely replace missing limbs.

Prostheses can be classified according to the limb they replace in the upper and lower limbs. According to the actuation, the prostheses can be classified into passive and active. Ferreira et al. (2016) mention amputees who use passive limbs generally walk more slowly and expend significantly more energy, about 60% more according to Lawson et al. (2014). Active prostheses can restore locomotion functions and recognize human intention, see, Spanias et al. (2018). But the cost of a smart prosthesis is too high for most people, see, Wen et al. (2018). The reasons for the high cost are sensors and actuators, as well as control.

The most common control strategies are gait mode recognition, in combination with low impedance control and direct electromyography (EMG) control. Echo-control requires high mechanical impedance, it does not feel natural as it does not interact with the amputee, and is only useful for unilateral amputees. The finite state impedance control has the drawback of tuning many parameters as explained by Ferreira et al. (2016), while the EMG-based control is affected by the problem of fatigue, misalignment of electrodes and a low signal-to-signal ratio, see, Ryu et al. (2019); K. Zhang et al. (2019).

The acceptance of prostheses has been complicated by the fact that the control systems are not sufficiently intuitive and the sensory feedback is inadequate, as Yang and Liu (2021) comments in their work on hand prostheses. And as Wu et al. (2021) mentions, due to technological limitations, autonomous systems still cannot fully supply human participation. They suggest the definition of schemes where both elements interact to provide better performance together, creating more intelligent systems Dellermann et al. (2021). The development of this approach has been varied, based on forms of interaction Grønsund and Aanestad (2020); levels of human participation Mabrok et al. (2020); as well as application environments: labor Grønsund and Aanestad (2020), UAVs Feng et al. (2016), autonomous vehicles Li et al. (2020); Wu et al. (2021), economy, machine learning Dellermann et al. (2021), Goecks (2020), Mandel et al. (2017), and Perrusquía et al. (2021), social sciences Dautenhahn (1998), robotic prostheses Welker et al. (2021); Yang and Liu (2021) and rehabilitation Gao (2020).

Some researches have been developed in order to find human-robot interaction dynamics like in Roveda et al. (2020) where using a model-based reinforcement learning approach, they define an Artificial Neural Network to model the dynamics. This approach is not possible to modify directly each of the systems what compound the human. Shi et al. (2021) propose human-centred adaptive control of a lower limb rehabilitation robot based on an HRI dynamic model. An equivalent spring model in three-dimensional space is proposed to express the interaction force between the human and robot. Then, it is just defined one part of the human model, and it is not enough to describe a human in order to be possible to simulate, for example, a general human-machine environment avoiding the real, and most of the times dangerous, human participation, such that there is no a complete human model.

Human-in-the-loop (HITL) simply means a system completely or partially controlled by a human via a control manipulator Mabrok et al. (2020). HITL can be modeled as an input-output system. Mabrok et al. (2020) suggest three challenges in feedback control with HITL:

• Understanding the spectrum of driver types with HITL.

• Derive models of human behavior

• Incorporate human behavior into the formal feedback control methodology.

The importance of these schemes lies in the possibility of a quantitative evaluation of the impact of human behavior in dynamic systems Bae et al. (2019). They contribute to health care or reduction of energy consumption Mabrok et al. (2020). And they are mainly useful when robustness is a priority Yang and Liu (2021). From a control theory perspective.

Various HITL algorithms have been implemented in the area of prosthetics, orthotics, and lower extremity rehabilitation robots mainly for the off-line design of these devices, as well as in the configuration of controller parameters, for example, as in the work of J. Zhang et al. (2017) on optimization of exoskeletons and that of Welker et al. (2021) on optimization of ankle-foot prostheses, who comments on the disadvantages of such an approach. Since this scheme is recently investigated, there is not much research on human in the loop control for prostheses. Examples of these recent investigations are the works of Zhuang et al. (2019) and that of Yang and Liu (2021) where they carry out an implementation of HITL in hand prostheses.

In all these cases, the human has participated actively in the tests, which implies the submission to stress and risks typical of any experiment. Perrusquía et al. (2021) proposes a generalization of reinforcement learning to emulate human behavior. Grønsund and Aanestad (2020) consider humans as supervisors who are responsible for managing performance. Feng et al. (2016) designs human behavior based on cognitive architectures. Bae et al. (2019) regards the human as an intelligent, independent agent to maximize their idiosyncratic utility function. The nature of these schemes leads to the need for a model that mimics human behavior in loop human applications Mabrok et al. (2020).

According to Welker et al. (2021), robotic prostheses have not been sufficiently explored and there is still no definition of the basic principles of how humans and prostheses interact. It is not possible to design controllers based on a mathematical description of human-prosthesis dynamics Gao (2020). Feng et al. (2016) highlights models of human behavior based on cognitive architectures that include memory encoding and organization of elements in mental structures. Work is continuing on the human model, as in the work of Perrusquía et al. (2021), they propose a generalization of reinforcement learning that considers stimuli, skills, knowledge, and previous experiences to emulate human behavior. However, current models of the human are limited Mabrok et al. (2020).

• Human behavior has been considered as a random disturbance or as an error compensation mechanism, however humans cannot be considered as irrational decision makers that can be modeled as stochastic disturbances.

• HITL under shared control has not yet been implemented in active prostheses.

In order to study human interaction with active lower limb prostheses in efficient and safe ways, we propose a general scheme of HITL for active prostheses, which in turn avoids human involvement during controller testing. The contributions of the paper are

• We propose an explicit human model-based system, which allows us to separately modify any part of the system. The model can be implemented for any system adapting the human model.

• Theory analysis of the proposed HITL

• Application of the proposed HITL to active lower limb prosthesis

1. Human-in-the-loop with shared control

The human-in-the-loop with shared control proposed in this paper is illustrated in Figure 1. It has the system where the human influenced by the environment controls an exogenous system (a machine) in a shared way with autonomous control (the inner control of the machine).

Figure 1. General scheme of human-in-loop under shared control.

We consider that there exists a system (machine) that is controlled by human and an autonomous controller. That is, there exists a system

(1) $\dot{\mathit{x}}=h(\mathit{x},u),$(1)

if this can be rewritten in the affine form,

(2) $\dot{x}=f(\mathit{x})+g(\mathit{x})u$(2)

where $h(\mathit{x},u)$ is a nonlinear function that depends on the state, $\mathit{x}$, and the control $u_2$. And $f(\mathit{x})$ and $g(\mathit{x})$ are nonlinear functions that depend just in the state, $\mathit{x}$.

On the other hand, the human can be considered like

(3) $\dot{\mathit{y}}(u_1)=f_2(\mathit{y}(u_1),\mathit{x})$(3)

That means that human is not considered to be explicitly controlled. That’s the reason why there is no control signal in (3). Human is a system that just react in response to the perceived environment, described by the state, $\mathit{x}$, of the controlled system. Moreover, the state, $\mathit{y}$, of the human is a function of the control signal, $u_1$, what is applied to the controlled system. That simply means that human produces a signal that is considered as a control signal for another system, and that system produces a state, $\mathit{x}$, that provoke to human to evolve and produce $u_1$. That is, in general, the behavior of the system in close loop, for the human-machine interaction.

And if we consider that the machine has an inner controller that produces the control signal, $u_2$, and at the same time, it receives an outer control signal, $u_1$, to add both getting the total control signal,

(4) $u=u_1+u_2.$(4)

Then in the close loop, it’s defined an interaction by shared control. That is, the machine is controlled by an inner and outer control, provided by itself and by human.

1.1. Human model

The model of the human we propose consists of

1. Cognitive system. Within this system, the intention of the human is defined and transmitted to the motor system. Through an adaptation system, the human generates signals based on the perceived exogenous agents, which are transmitted to a control system that produces the extended actions necessary to achieve that the satisfaction error converges to zero. It can be simplified as a controller

(5) $\dot{u}=f(e(\mathit{x},\mathit{y}))$(5)

where $u$ is the control signal produced by human, and $f(e(\mathit{x},\mathit{y}))$ is a function that defines the controller dynamics, which is defined in terms of the satisfaction error $e(\mathit{x},\mathit{y})$, with $\mathit{x}$ and $\mathit{y}$ the states of the human and the exogenous system.

1. Motor system. This system acts as a link between humans and exogenous systems. To convey his intention through movements and the imposition of states on the joints. Practically, this is regarded as an usual robot, that is, it can be described as

(6) $\mathit{D}(\mathit{y})\ddot{\mathit{y}}+\mathit{C}(\mathit{y}\mathbf{,}\dot{\mathit{y}})+\mathit{g}(\mathit{y})=\mathit{u}$(6)

where the states, $\mathit{y}$ corresponds to the human-join states, what could regard the natural and extended actions at the same time, because the actions of the human are produced as a result of the states taken by each of the human-joints. Here, the control, $\mathit{u}$, is the produced signal by the cognitive system.

1. Sensory system. It is a system that perceives (identifies and/or observes) exogenous systems or their states. Then, clearly a convenient representation is an observer, the sensors or the signal conditioning stage. Additionally, it can contain an identifier system.

(7) $D(\overline{x})\stackrel{\mathrm{..}}{\overline{x}}+C(\overline{x},\stackrel{.}{\overline{x}})+g(\overline{x})=u$(7)

This equation represents an identified system, that describes an estimated dynamics of the exogenous system, where, $\bar{\mathit{x}}$ is the estimated or sensed state for $\mathit{x}$.

The proposed human model has three signals:

1. Natural signals. These correspond to the physiological signals of a natural behavior, for example, the bio-mechanical signals that it produces when walking.

2. Extended Signals. The signals that are produced for the manipulation of external agents, as in the manipulation of tools, are considered.

3. Intentional signals. They correspond to those associated with human desires.

1.2. Human-robot interaction for active prosthesis

In human-robot interaction, the human defines the desired state of the joints through the residual limb, that is, through the natural actions, but human does not apply extended actions to share the control with the autonomous control of the prosthesis. Moreover, there isn’t exist human feedback to present a human-machine scheme control where human just provides the human intention. In order to compare lately with a scheme where human provides a control signal by closing the loop. See, Figure 2.

Figure 2. Emulation of human-prosthesis interaction with a 4-DOF robot.

To emulate the control of the prosthesis attached to the residual limb, we design a robot that represents the human-prosthesis system. The complete human-prosthesis system can be regarded as a 4-DOF robot.

The proposed scheme can be used as a test platform that involves the human being in active participation. The scheme allows to emulate human participation without risk to the human. In addition, the model’s ability to learn allows it to be implemented with a real human in the same way that it can be implemented with any other plant, which can improve the performance of the model when learning from another human.

The 4-DOF robot is modeled as

(8) $D(q)\stackrel{\mathrm{..}}{q}+C(q,\stackrel{.}{q})+g(q)=\tau$(8)

where $\mathit{q}=[q_1,q_2,q_3,q_4]=\left[{\mathit{q}}_{\mathit{r}}\mathbf{,}{\mathit{q}}_{\mathit{l}}\right]$, ${\mathit{q}}_{\mathit{r}}=[q_1,q_2]$ is the state of the residual limb, ${\mathit{q}}_{\mathit{l}}=[q_3,q_4]$ is the state of the lost limb,

The residual limb is controlled by the human who perceives the prosthesis as a perturbation, which is defined ${\mathit{d}}_{\mathit{p}}=[d_1,d_2,0,0{]}^T,$ ${\mathit{d}}_{\mathit{r}}=[0,0,d_3,d_4{]}^T.$ So (8) becomes

(9) $\begin{array}{c}\mathit{D}\mathbf{(}\mathit{q})\ddot{\mathit{q}}+\mathit{C}\mathbf{(}\mathit{q}\mathbf{,}\dot{\mathit{q}}\mathbf{)}+\mathit{g}\mathbf{(}\mathit{q}\mathbf{)}={\tau }_{\mathit{r}}+{\mathit{d}}_{\mathit{p}}\\ \mathit{D}\mathbf{(}\mathit{q}\mathbf{)}\ddot{\mathit{q}}+\mathit{C}\mathbf{(}\mathit{q}\mathbf{,}\dot{\mathit{q}}\mathbf{)}+\mathit{g}\mathbf{(}\mathit{q}\mathbf{)}={\tau }_{\mathit{p}}+{\mathit{d}}_{\mathit{r}}\end{array}$(9)

where $\tau ={\tau }_{\mathit{r}}+{\tau }_{\mathit{p}}$, ${\tau }_{\mathit{r}}=[{\tau }_1,{\tau }_2,0,0]$ is the control for human hip and thigh ${\tau }_{\mathit{p}}=[0,0,{\tau }_3,{\tau }_4]$ is the control over the prosthesis.

1.3. PD control for human-robot system

Regarding the amputee-prosthesis system as the previously described 4-DOF robot, we base in the perspective of robotics to define a controller for all the complete systems.

Considering, dynamics for the DC motors are almost linear, we can use simple controllers for the joints. We used PD control for the joints (10),

(10) $\tau =k_p\tilde{q}+k_d\stackrel{.}{\tilde{q}}$(10)

where ${\mathit{k}}_{\mathit{p}}$ and ${\mathit{k}}_{\mathit{d}}$ are positive gains, $\tilde{\mathit{q}}={\mathit{q}}_{\mathit{d}}-\mathit{q}=[{\mathit{e}}_{\mathit{r}},{\mathit{e}}_{\mathit{p}}]$ is the tracking error, it includes the error for residual limb part, ${\mathit{e}}_{\mathit{r}}$, and for the prosthesis, ${}_p$, ${\mathit{q}}_{\mathit{d}}=[{\mathit{q}}_{\mathit{r}\mathit{h}\mathit{a}},{\mathit{q}}_{\mathit{l}\mathit{e}}{]}^T$ is the desired trajectory. We use first-order low pass filter to estimate the derivatives. Here, we use first-order low pass filter to estimate the derivatives

(11) $\dot{\mathit{q}}\approx \frac{\delta }{\delta +s}\mathit{q}\left(s\right),0<\delta <<1$(11)

In order to obtain good controller, ${\mathit{k}}_{\mathit{p}}$ and ${\mathit{k}}_{\mathit{d}}$, we need to identify the simplified model. For each joint, we use the following second order linear model,

(12) ${\ddot{q}}_i=-a_i{\dot{q}}_i+b_i{\tau }_i-d_i,i=1\cdots 4$(12)

where $q_i$ is the joint position, ${\tau }_i$ is the control, $a_i$ and $b_i$ are the parameters to be identified, $d_i$ is the coupling perturbation. (12) can be written in matrix form,

(13) $\left[\begin{array}{ccccc}{a}_1& b_1& 0& \cdots & 0\\ 0& 0& \hspace{0.17em}& \hspace{0.17em}& ⋮\\ 0& \cdots & 0& a_4& b_4\end{array}\right]\left[\begin{array}{c}-{\dot{q}}_1\left(k\right)\\ {\tau }_1\left(k\right)\\ ⋮\\ -{\dot{q}}_4\left(k\right)\\ {\tau }_4\left(k\right)\end{array}\right]-\left[\begin{array}{c}{d}_1\left(k\right)\\ ⋮\\ d_4\left(k\right)\end{array}\right]=\left[\begin{array}{c}{\ddot{q}}_1\left(k\right)\\ ⋮\\ {\ddot{q}}_4\left(k\right)\end{array}\right]$(13)

There are $n$ data. We define

$\begin{array}{c}\alpha \left(k\right)={\left[-{\dot{q}}_1\left(k\right),{\tau }_1\left(k\right)\cdots ,-{\dot{q}}_4\left(k\right),{\tau }_4\left(k\right)\right]}^T,\beta \left(k\right)={\left[{\ddot{q}}_1\left(k\right),\cdots ,{\ddot{q}}_4\left(k\right)\right]}^T\\ \mathit{A}=\left[\alpha \left(1\right),\cdots ,\alpha \left(n\right)\right],\mathit{B}=\left[\beta \left(1\right),\cdots ,\beta \left(n\right)\right]\\ \mathrm{\Theta }=\left[a_1,b_1\cdots ,a_4,b_4\right]\end{array}$

$k=1\cdots n.$

The object of the parameter identification is

(14) $J=\underset{\mathbf{\Theta }}{min}\sum _k^n{\parallel }^{\hspace{0.17em}}\beta \left(k\right)-\mathrm{\Theta }\alpha \left(k\right){\parallel }^2$(14)

where $\hat{\mathit{y}}\left(k\right)=\mathit{V}\mathbf{\Phi }\left[\mathit{x}\left(k\right)\right].$ The training data are $\mathit{y}\left(k\right)$ and $\mathbf{\Phi }\left[k\right].$ The optimal solution of $\mathbf{\Theta }$ is

(15) $\hat{\mathrm{\Theta }}={\left(\mathit{A}{\mathit{A}}^T\right)}^{-1}{\mathit{A}}^T\mathit{B}$(15)

We can write the error dynamic for each of the joints as

(16) ${\dot{\mathit{X}}}_{\mathit{i}}\mathbf{=}{\mathit{A}}_{\mathit{i}}{\mathit{X}}_{\mathit{i}}+{\mathit{B}}_{\mathit{i}}{\tau }_{\mathit{i}}$(16)

where $\mathit{X}$ is the state variable $X_i=[{\tilde{q}}_i,{\dot{\tilde{q}}}_i]$, ${\mathit{B}}_i=[0,-{\hat{b}}_i{]}^T$ and ${\mathit{A}}_i=\left[\begin{array}{cc}0& 1\\ 0& -{\hat{a}}_i\end{array}\right]$. The PD control (10) can be rewritten as ${\tau }_{\mathit{i}}\mathbf{=}{\mathit{k}}_{\mathit{i}}{\mathit{X}}_{\mathit{i}}$, closed-loop system becomes

(17) ${\dot{\mathit{X}}}_{\mathit{i}}\mathbf{=}{\mathit{A}}_{\mathit{i}}{\mathit{X}}_{\mathit{i}}+{\mathit{B}}_{\mathit{I}}{\mathit{k}}_{\mathit{i}}{\mathit{X}}_{\mathit{i}}\mathbf{=}\mathbf{(}{\mathit{A}}_{\mathit{i}}+{\mathit{B}}_{\mathit{i}}{\mathit{k}}_{\mathit{i}}\mathbf{)}{\mathit{X}}_{\mathit{i}}$(17)

In order to assure stability of (17), we chose ${\mathit{k}}_{\mathit{i}}=[k_{1i}{k}_{2i}]$, such that ${\mathit{A}}_{\mathit{i}}+{\mathit{B}}_{\mathit{i}}{\mathit{k}}_{\mathit{i}}$ is negative definite, i.e., the following polynomial has negative roots,

$\begin{array}{c}det(\lambda \mathit{I}-({\mathit{A}}_{\mathit{i}}+{\mathit{B}}_{\mathit{i}}{\mathit{k}}_{\mathit{i}}))=det\left(\lambda \mathit{I}+\left[\begin{array}{cc}0& 1\\ -{\hat{b}}_i{k}_{1i}& -{\hat{a}}_i-{\hat{b}}_i{k}_{2i}\end{array}\right]\right)\\ ={\lambda }^2+({\hat{a}}_i+{\hat{b}}_i{k}_{2i})\lambda +{\hat{b}}_i{k}_{1i}\end{array}$

where the roots are

$\begin{array}{c}{\lambda }_1=\frac{-({\hat{a}}_i+{\hat{b}}_i{k}_{di})}{2}+\frac{1}{2}\sqrt{{({\hat{a}}_i+{\hat{b}}_i{k}_{di})}^2-4{\hat{b}}_i{k}_{pi}}\\ {\lambda }_2=\frac{-({\hat{a}}_i+{\hat{b}}_i{k}_{di})}{2}-\frac{1}{2}\sqrt{{({\hat{a}}_i+{\hat{b}}_i{k}_{di})}^2-4{\hat{b}}_i{k}_{pi}}\end{array}$

So we need the controller gains

(18) $k_{2i}>-{\hat{a}}_i/{\hat{b}}_i,k_{1i}>0$(18)

Here PD gains are based on the estimated parameters.

1.4. PID control for human-robot system

In the human-in-the-loop (HITL) as in Figure 1, there are two controllers interacting to control the prosthesis. The human signal is given priority over the tracking control signal. We define the human as an intelligent system that acts on a system based on knowledge acquired by interaction. The human applies actions based on the errors obtained by comparing what he wants and what he observes.

So HITL also needs a human model, which considers the following:

1. Past. The human is based on what he has learned from the system. He accumulates the experience, and takes an action proportional to this accumulated.

2. Present. He acts to the response of the stimulus, and observes the response with each stimulus, to minimize the error in each application.

3. Future. The human interacts more with the system, he learns his behavior so that he can anticipate his action.

The above three actions can be modeled as,

(19) $U(s)=[k_p+k_i\frac{1}{s}+\frac{N}{s+N}{k}_{d}s]E(s)$(19)

It has the form of PID control. We need to specify the parameters of $k_p$, $k_i$, $k_d$, and $N$.

The human model can be written as

$\left[\begin{array}{c}{q}_1\\ q_2\\ u_2\end{array}\right]=\left[\begin{array}{c}f\mathrm{m}\mathrm{o}\mathrm{d}\left(t,f\right)\\ g\left(t\right)\\ k_p+k_i\frac{1}{s}+k_d\frac{N}{1+N\frac{1}{s}}]e_s\end{array}\right]$

For $q_1$ and $q_2$ we have

$\left[\begin{array}{c}{q}_1\\ q_2\end{array}\right]=\left[\begin{array}{c}f\mathrm{m}\mathrm{o}\mathrm{d}\left(t,f\right)\le f^2\\ g\left(t\right)\le max\left\{q_{2h_2}\right\}\end{array}\right]$

where $q_{2ha}$ corresponds to the anthropomorphic femur angle desired with a bounded influence of the environment, what is always bounded. So $q_1$ and $q_2$ are bounded states.

$q_1$ and $q_2$ are variables measured directly by the amputee, they are in $(0,2\pi )$. The active prosthesis estimates the states to produce gait based on ${\mathit{q}}_{\mathit{r}}=[q_1,q_2]\in D,$ which are the residual signals from the residual limb.

We define the discrete sets

$\begin{array}{c}C=\left\{q_{1_{Ti}},q_{2_{Ti}},q_{3_{Ti}},q_{4_{Ti}}\right\}\\ F=\{0<q_3<2\pi ,0<q_4<2\pi \}\\ D=\{0<q_1<2\pi ,0<q_2<2\pi \}\end{array}$

We use mapping $M$ to describe an anthropomorphic trajectory followed by the amputee,

(20) $M:D\to F$(20)

Since the model of mapping $M$ is not available, we use the following neural network to estimate it,

$\begin{array}{c}{\sigma }_{1i}=\sigma (W_i\hat{q}),{\sigma }_{2i}=\sigma ({\mathit{V}}_{\mathit{i}}{\sigma }_{1i})\\ q_{lei}=\mathit{M}{\sigma }_{2i},L=\frac{1}{2}{(q_{lei}-{\bar{q}}_i)}^2\end{array}$

where $i=1,2$, $\mathit{M}$, $\mathit{V}$ and $\mathit{W}$ are the output, hidden and input layer weight matrices, respectively, and $\hat{\mathit{q}}=[q_r^T,1]$, ${\bar{q}}_i$ is based on the data set of Winter (2009) to train the network.

For training, the Backpropagation algorithm was applied using a descending gradient.

(21) $\begin{array}{c}{W}_{it}=W_{i(t-1)}-\eta \frac{\mathrm{\partial }L}{\mathrm{\partial }{W}_i}\\ \frac{\mathrm{\partial }L}{\mathrm{\partial }{W}_i}=diag({\sigma }_{1i}^{\prime })V_i{\sigma }_{2i}^{\prime }M\hat{q}\end{array}$(21)

$V_i$ uses the same form as (21).

2. Stability analysis of human-in-the-loop with shared control

2.1. Stability analysis of PD

When we apply the controller (10) to each joint (12), we need to assure the closed-loop system be stable. We rewrite (12) with state-space form

(22) $\begin{array}{c}{\dot{x}}_1=x_2\\ {\dot{x}}_2=-a_i{x}_2+b_i{\tau }_i-d_i\end{array}$(22)

where $x_1=q_i,$ $x_2={\dot{q}}_i$, ${\tau }_i=k_p{\tilde{q}}_i+k_d{\dot{\tilde{q}}}_i,$ ${\tilde{q}}_i=q_i^d-q_i.$

In regulation case, ${\dot{q}}_i^d=0,$ we define

(23) $\begin{array}{c}{\bar{x}}_2={\dot{q}}_i^d-{\dot{q}}_i=x_2,{\bar{x}}_1=q_i^d-q_i\\ {\tau }_i=k_p{\bar{x}}_1+k_d{x}_2\end{array}$(23)

We consider the following Lyapunov candidate function

(24) $V_1=\frac{1}{2}{\bar{x}}_2^2+\frac{b_i{k}_p}{2}{\bar{x}}_1^2$(24)

where $b_i,k_p>0.$ From (23)

(25) $V_1^{\cdot }=x_2{\dot{x}}_2-b_i{k}_p{x}_2{\bar{x}}_1$(25)

Using (22)

(26) $\begin{array}{c}{\dot{V}}_1=x_2(-a_i{x}_2+b_i{\tau }_i-d_i)-k_p{\bar{x}}_2^T{\bar{x}}_1\\ =-a_i{x}_2^2+b_i{x}_2\left(k_p{\bar{x}}_1+k_d{x}_2\right)-x_2{d}_i-b_i{k}_p{x}_2{\bar{x}}_1\\ =-\left(a_i-b_i{k}_d\right)x_2^2-x_2{d}_i\end{array}$(26)

Because

(27) $-x_2{d}_i\le \frac{1}{2}\left(x_2^2+d_i^2\right)$(27)

(26) becomes

(28) $\begin{array}{c}{\dot{V}}_1\le -\left(a_i-b_i{k}_d-\frac{1}{2}\right)x_2^2+\frac{1}{2}{d}_i^2\\ \le -\left(a_i-b_i{k}_d-\frac{1}{2}\right)x_2^2+{\bar{d}}_i\end{array}$(28)

where ${\bar{d}}_i$ is the upper bound of $\frac{1}{2}{d}_i^2,$ $\left|\frac{1}{2}{d}_i^2\right|\le {\bar{d}}_i.$

We should choose the gain $k_d$ such that

$\frac{a_i-\frac{1}{2}}{b_i}>k_d>0$

then the closed-loop system is uniformly bounded, $V_1$ convergences to ${\bar{d}}_i.$ Here $a_i$ and $b_i$ are obtained from (13)

In tracking case ${\dot{q}}_i^d\ne 0,$ we use the same definitions as (23), and define an auxiliary variable

(29) $r={\bar{x}}_2+\lambda {\bar{x}}_1,\lambda >0$(29)

The PD control becomes

${\tau }_i=Kr=K{\bar{x}}_2+\lambda K{\bar{x}}_1$

So $k_p=K,$ $K_d=\lambda K,$ and

(30) $\begin{array}{c}\stackrel{\cdot }{r}={\stackrel{\cdot }{\bar{x}}}_2+\lambda {\stackrel{\cdot }{\bar{x}}}_1={\stackrel{\cdot \cdot }{q}}^d-\stackrel{\cdot \cdot }{q}+\lambda {\stackrel{\cdot }{\bar{x}}}_1\\ ={\stackrel{\cdot \cdot }{q}}^d-\left(-a_i{x}_2+b_i{\tau }_i-d_i\right)+\lambda {\stackrel{\cdot }{\bar{x}}}_1\end{array}$(30)

The proposed Lyapunov function is

$V_2=\frac{1}{2}{r}^2$

The derivative of it is $\underset{2}{\overset{\cdot }{V}}=r^T\stackrel{\cdot }{r}$,

$r\dot{r}=r({\ddot{q}}^d+a{x}_2-b\tau +d)+\lambda r{\dot{\overline{x}}}_1=-r^{2}bK+r(d+{\ddot{q}}^d+a{x}_2+\lambda {\dot{\overline{x}}}_1)=-r^{2}bK+r(d+f)$

where $f={\ddot{q}}^d+a{x}_2+\lambda {\dot{\overline{x}}}_1.$ Because

$r(d+f)\le \frac{1}{2}{r}^2+\frac{1}{2}\bar{f}$

where $\left|\frac{1}{2}(d+f)\right|\le \bar{f}.$ So

${\dot{V}}_2\le -\left(bK-\frac{1}{2}\right)r^2+\bar{f}$

We should choose the gain $K$ such that

$K>\frac{1}{2b}$

then the closed-loop system is uniformly bounded, $V_2$ convergences to $\bar{f}.$

From the Lyapunov analysis, we concluded that the PD can stabilize the system. By the identification of the system, we can obtain good control gains.

So, amputee-prosthesis system is stable under previous consideration.

2.2. Stability analysis of PID

As described by (12), the prosthesis can be considered as a 2-DOF robot, subject to a perturbation due to the coupling with the residual limb, described in likewise as a 2-DOF robot subject to the perturbation due to the prosthesis coupling. And more over, each of the joints of the prosthesis can be considered as independent systems perturbed by the interconnections of the other links.

(31) ${\ddot{q}}_i=-a{\dot{q}}_i+b{u}_{2i}-d_i$(31)

defining $e={\bar{q}}_i-q_i$, $\dot{e}={\dot{\overline{q}}}_i-{\dot{q}}_i$. This implies $q_i={\bar{q}}_i-e$, and ${\dot{q}}_i={\dot{\overline{q}}}_i-\dot{e}$. Then, the error dynamics can be described as

(32) $\ddot{e}=\ddot{\overline{q}}+a(\dot{\overline{q}}-\dot{e})-bu+d$(32)

where the dynamics of the human controller can be written as

(33) $\ddot{u}=(k_p+k_{d}N)\ddot{e}+(N{k}_p+k_i)\dot{e}+N{k}_{i}e-N\dot{u}$(33)

So the system in closed loop can be described as

$\begin{array}{c}\ddot{e}=-a\dot{e}-bu+\bar{d}\\ \ddot{u}=-{\alpha }_1\dot{e}-{\alpha }_2\dot{u}-{\alpha }_{3}u+{\alpha }_{4}e+{\alpha }_5\bar{d}\end{array}$

where ${\alpha }_1=(k_p+k_{d}N)a-N{k}_p+k_i$, ${\alpha }_2=N$, ${\alpha }_3=b(k_p+k_{d}N)$, ${\alpha }_4=N{k}_i$, ${\alpha }_5=k_p+k_{d}N$, $\overline{d}=\ddot{\overline{q}}+a\dot{\overline{q}}+d$

We extend the idea of Thenozhi and Yu (2014) for the PID with derivative filtering. The equilibrium point for the error dynamics must be at $e=0$, $\dot{e}=0$ and $\ddot{e}=0$. From (32)

(34) $u_s=\frac{\ddot{\overline{q}}+a\dot{\overline{q}}+d}{b}$(34)

Defining $u=u-u_s$, so the equilibrium point results $u=0$. As the error converge, so $\ddot{\overline{b}}$ and $\dot{\overline{q}}$ are bounded and considering the perturbation, $b$, bounded too, we can consider constant the control at the equilibrium, so $\dot{u}=0$ at equilibrium. Finally, the equilibrium point is defined at $(0,0,0,0)$.

In order to analyze the stability of the system, we propose the candidate Lyapunov function

(35) $V=\frac{1}{2}(\dot{u}-e{)}^2+\frac{1}{2}(u+\dot{e}{)}^2+\frac{1}{2}{u}^2$(35)

with $k_1={\alpha }_1+1$ and ${\alpha }_2={\alpha }_3-k_1$. Assuming positive constants, $k_1$ and $k_2$ so $V>0$.

Then

(36) $\dot{V}=(\dot{u}-e)(\ddot{u}-\dot{e})+k_1(u+\dot{e})(\dot{u}+\ddot{e})+k_{2}u\dot{u}$(36)

substituting the error and control dynamics we get

(37) $\begin{array}{cc}\dot{V}& =-{\alpha }_2\dot{u}+({\alpha }_2+{\alpha }_4)\dot{u}\bar{d}\\ \hspace{0.17em}& -a({\alpha }_1+1){\dot{e}}^2-(a+b)({\alpha }_1+1)\dot{e}u\\ \hspace{0.17em}& +({\alpha }_1+1)\dot{e}e-b({\alpha }_1+1)u^2\\ \hspace{0.17em}& +{\alpha }_{3}ue+({\alpha }_1+1)u\bar{d}-{\alpha }_4{e}^2-{\alpha }_{5}e\bar{d}+({\alpha }_1+1)\dot{e}\bar{d}\end{array}$(37)

so

(38) $\begin{array}{cc}\dot{V}& \le -{\dot{u}}^2(\frac{1}{2}{\alpha }_2-\frac{{\alpha }_4}{2}-\frac{{\alpha }_5}{2})\\ \hspace{0.17em}& -{\dot{e}}^2[(1+{\alpha }_1)(\frac{a}{2}-\frac{b}{2}-1)]\\ \hspace{0.17em}& -u^2[(1+{\alpha }_1)(b-\frac{1}{2}(a+b)-\frac{1}{2})-\frac{{\alpha }_3}{2}]\\ \hspace{0.17em}& -e^2\frac{1}{2}({\alpha }_4-{\alpha }_2-{\alpha }_1-{\alpha }_3-{\alpha }_5-1)\\ \hspace{0.17em}& -{\bar{d}}^2(-{\alpha }_5-{\alpha }_1-1)\end{array}$(38)

The system is stable if

$\begin{array}{c}{\alpha }_2\ge {\alpha }_4+{\alpha }_5,0\le ({\alpha }_1+1)(a-b-2)\\ \frac{{\alpha }_3}{2}\le (1+{\alpha }_1)(b-\frac{1}{2}(a+b)-\frac{1}{2})\\ {\alpha }_4\ge {\alpha }_1+{\alpha }_2+{\alpha }_3+{\alpha }_5,0\ge {\alpha }_5+{\alpha }_1+1\end{array}$

3. Human-in-the-loop control for active lower limb prosthesis

In order to test the proposed scheme, we define an amputee-prosthesis system.

Here, the human model corresponds to an amputee with a lower residual limb made up of the hip and thigh, and with the ability to control this residual limb according to the desired intention under the influence of the environment.

For a complete model, we consider an active transfemoral prosthesis rigidly connected to the stump of the residual limb where humans share control in two ways, 1) by connection of the stump, 2) by extended actions. The complete assemble can be regarded as a model where an active prosthesis includes the dynamics of human system or a human who has attached a prosthesis. See, Figure 3.

Figure 3. Lower limb prosthesis.

For this implementation, we define:

1. Natural signals. The human performs the usual movement (through the residual limb) to reach a desired state when walking. These signals, $q_{2h},q_{3h}$ and $q_{h4}$, are defined as the trajectories followed by the joints during gait based on the work of Winter (2009). In addition, a synthetic signal is considered that is interpreted as the trajectory in the vertical direction of a fixed point on the hip, $q_{1h}$, during the walk, and is defined as

(39) $q_{1h}=f\mathrm{m}\mathrm{o}\mathrm{d}(t,f)$(39)

where $f\in 1,2]hz$ is the walking frequency and $\mathrm{m}\mathrm{o}\mathrm{d}(t,f)$ is as usual the mathematical modulo function, what obtains the remainder when we divide $t$ over $f$.

1. Extended. These signal are obtained by the human control, and, in general, is defined as a function, $f$ of the satisfaction error, ${\mathit{e}}_{\mathit{s}}$

(40) $u_2=f({\mathit{e}}_{\mathit{s}})$(40)

where

(41) ${\mathit{e}}_{\mathit{s}}=({\mathit{q}}_{\mathit{l}\mathit{h}\mathit{a}}+\mathit{k})-({\mathit{q}}_{\mathit{l}})$(41)

with ${\mathit{q}}_{\mathit{l}}=[q_3,q_4]$, the state of the prosthesis.

1. Signal of intention. It corresponds to the state of the prosthesis desired by the human.

The environment is considered as a system that generates a Boolean signal, as a differentiating means of two environmental conditions.

(42) $S_a(b)=\left\{\begin{array}{cc}\mathit{k}& \mathrm{i}\mathrm{f}b=1\\ \mathbf{0}& \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$(42)

where $k$ is the gain of influence in the system. So assuming interaction with environment and a unitary gain of influence, we have $S_a=k=1$.

There are exogenous systems that humans can interact with. These systems have a behavior that does not depend at all on the existence of the human being in the system. The exogenous system is an autonomous and independent system. The trajectories of exogenous systems will be defined as exogenous behaviors.

Robotic systems, even other humans, are examples of exogenous systems. The environment is an exogenous system that modifies or motivates the generation of signals. That is, human signals are personal, which means that they differ between human and human, since during his life he is immersed in a unique environment, which provides the particularities of the genesis of his signals. Since each system is considered autonomous, there is a controller for each system that defines a certain behavior.

Figure 4 shows the proposed scheme of human-in-the-loop control. Here, the human being influenced by the environment interacts with the prosthesis through the stump and extended actions.

Figure 4. Human-in-the-loop control for prosthesis.

The human intention is defined as

${\mathit{q}}_{\mathit{h}}=[q_{1h},q_{2h},q_{3h},q_{4h}{]}^T$

where $q_{ih}$ are the hip, thigh, knee and ankle.

The environment influence is defined as $s_a=k$. This signal is added to the desired intention to modify the intention under environment interaction. The intention is

${\mathit{q}}_{\mathit{h}\mathit{a}}=[q_{rha},q_{lha}{]}^T,{\mathit{q}}_{\mathit{l}\mathit{h}\mathit{a}}=[q_{3ha},q_{4ha}{]}^T$

where $q_{rha}$ and $q_{lha}$ are the desired human hip and thigh. $q_{lha}$ is the desired state of knee and ankle of the prosthesis.

The closed-loop representation with human control is shown in Figure 5.

Figure 5. HITL for active lower limb prosthesis.

4. Experimental results

The experimental results was obtained by simulation. The mechanical environment of the human-machine system was developed in SimMechanics over Matlab, exporting designed elements previously designed in Catia based on anthropomorphic dimensions for optimal gait. The control system was implemented in Simulink connected to SimMechanics. The integration method used was ode45, with maximum step size of $0.01s$.

The implementation of prostheses makes human-machine interaction inevitable. In the first test, we consider the case when the human does not have an intervention beyond the imposition of states to the system.

4.1. Open loop HITL control

When the human participates, he does not execute actions additional to the natural ones, and there is not a feedback of the states of the prosthesis influencing its control. We defined it as open loop with the human interaction.

The implementation and simulation of the HITL control are for the knee and the ankle for the prosthesis attached to the human, physically represented by the hip and the thigh as the residual limb such that these provide the more significant information of the desired trajectory for gait. The autonomous control is a PD trajectory tracking control. A comparison of tracking trajectory of the gait pattern obtained by the neuro-decoder is shown in Figure 6

Figure 6. Comparison of trajectories in tracking control.

The neural decoders receive the signals from the hip and thigh, and return the trajectory of the knee. They apply out-of-range signal to the decoder to emphasize the need for human participation, the behavior shown in Figure 7. The experiment considered a variation of the range of the signal for which the neural decoder was designed. Instead of having a $[0,2\pi ]$ signal, we defined a $[0,4\pi ]$ signal, see, Figure 8. The tracking errors are shown in Figure 9. Comparison of the human signal and the signal achieved by autonomous control is presented in Figure 10.

Figure 7. Comparison of the filters to estimate ${\dot{\theta }}_{d3}$.

Figure 8. Comparison of the decoded signal.

Figure 9. Tracking error of neuro-decoder generated trajectories.

Figure 10. Tracking error in open loop.

Neural networks provide the robustness of the system. Since the value is outside the design parameters of the network, it is very possible that the trajectory achieved is not the one expected by the amputee. However, we consider that there is no such intervention to observe the behavior of the system. The error increases, because the neuro-decoding system cannot adequately identify the intention, when the parameters are outside the design ranges.

4.2. Close-loop HITL control

The system receives intention signal, which is out of range. An environmental signal is included to justify this signal. The human intention signal is taken into account, which is under HITL scheme. The extended human signal is applied to the prosthesis-controller system.

In Figure 11, the result of the intention tracking based on PID model is shown. Due to the difference in the initial conditions, the participation of the human is small. It only seeks to compensate the signal that was added to the reference. In the second cycle, its participation is greater, because the tracking control fails in the decoding.

Figure 11. PID model in the HITL control.

Figure 12 shows the human must influence with greater effort, managing to better track the intention signal but with a lag. This delay comes from human observation to act. It can be solved by adapting an interaction prediction system through intelligent mechanisms.

Figure 12. Human control signal applied in the HITL control.

4.3. Human interaction

The human-robot interaction force is defined in this implementation as the extended actions. The energy used to control the prosthesis is observed through the control signal applied by the human as the force need in the shared control in order to achieve the human desired state. Figure 12 shows the human control needed to correct the state of the prosthesis in order to get the correct desired gait.

So, according to the observations, the human model proposed is useful to study the payload when it shares the control in the proposed scheme. And is clearly than the autonomous tracking control is not the best option to interact with human, so it is not so convenient to do a real-human implementation with this scheme using that control. First to do a real implementation, we could use the proposed human model to design and test better autonomous controller in order to prevent dangerous test for human.

It must be notice, that even when the human signal control, human-robot interaction force, looks high, it is not applied with that magnitude, this is because motors have a previous saturation stage, that is, to prevent overload in motors, the control signal passes through a saturator that limits the signal to $-10$ to $10V$. That means, human perceive, and actually apply, a higher force than applied to joints. This suggests again that it is better to choose another different control than tracking control, according to literature a possible option to improve this behavior is impedance control.

5. Conclusions

The proposed HITL scheme for prosthesis is useful to study the human correction of errors in autonomous controller dynamics due, for example, to modeling, identification or synchronization errors. As well as the participation of the human for fine adjustments due to disturbances such as environmental ones. In such a way as to guarantee his satisfaction, comfort and safety.

The active lower limb prosthesis has the advantage that the amputee can freely change the desired trajectory and does not depend on the healthy leg, like the eco-control schemes. It does not require a high computational cost treatment like the EMG-based implementation. It does not require a complex sensory system. Only hip reading status is required for mapping.

To improve the model, it is possible to add an adaptive and learning capacity, even psychological. It is still necessary to study stability, and furthermore, an inherently stable model should be considered to make it more suitable to emulate a real human.

In the future, we are going to extend human model in order to incorporate more physical and psychological behavior, and add more DOFs to the motor system, also include hip and thigh as residual limb. This will improve balance of walking process.

Acknowledgement

Authors would like to thank the Department of Automatic Control of CINVESTAV-IPN for providing the experimental devices.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Additional information

Funding

This work was supported in part by CONACYT under the grant Consejo Nacional de Ciencia y Tecnología CONACyT-A1-S-8216 and by CINVESTAV under the grants, SEP-CINVESTAV-62 and CNR-CINVESTAV

Notes on contributors

Iván Hernández

Wen Yu received the B.S. degree in automatic control from Tsinghua University, Beijing, China in 1990 and the M.S. and Ph.D. degrees, both in Electrical Engineering, from Northeastern University, Shenyang, China, in 1992 and 1995, respectively. From 1995 to 1996, he served as a lecturer in the Department of Automatic Control at Northeastern University, Shenyang, China. Since 1996, he has been with CINVESTAV-IPN (National Polytechnic Institute), Mexico City, Mexico, where he is currently a professor with the Departamento de Control Automatico. From 2002 to 2003, he held research positions with the Instituto Mexicano del Petroleo. He was a Senior Visiting Research Fellow with Queen’s University Belfast, Belfast, U.K., from 2006 to 2007, and a Visiting Associate Professor with the University of California, Santa Cruz, from 2009 to 2010. He also holds a visiting professorship at Northeastern University in China from 2006. Dr.Wen Yu serves as associate editors of IEEE Transactions on Cybernetics, Neurocomputing, and Journal of Intelligent and Fuzzy Systems. He is a member of the Mexican Academy of Sciences.

Wen Yu

Iván Hernández received the B.S degree in Electronical Engineering from Metropolitan Autonomous University (UAM), CdMx, Mexico in 2014 and the M.S. degree in Electrical Engineering from Center for Research and Advanced Studies of the National Polytechnic Institute (CINVESTAV-IPN), CdMx, Mexico in 2019. He received the medal for university merit from UAM in 2014. He has taught at various institutions since 2005. He is currently working toward the Ph.D. degree in Automatic Control in CINVESTAV-IPN. His main research of interest focuses on biomechatronics, robotics, artificial intelligence, nonlinear control.