EEG-modulated robotic rehabilitation system for upper extremity

ABSTRACT

This paper presents a novel electroencephalogram (EEG)-triggered upper extremity training system. Motor imagery EEG of upper extremity movements is adopted to trigger the Barrett WAM to perform rehabilitation training for patients with stroke. We focus on fully exploring the patient's movement intention and attention from movement imagination EEG and controlling the WAM robot to perform training effectively. A position controller based on fuzzy logic is presented for the rehabilitation system to drive the WAM robot smoothly. Experimental results with seven participants are reported to show the feasibility and effectiveness of the robotic therapy system.

KEYWORDS:

• Neurorehabilitation robot
• stroke therapy
• fuzzy logic
• motor imagery

Introduction

Brain-computer interface (BCI) systems allow users to adopt the activity of the brain to control the environment [1,2]. In recent years, new solutions are being extensively sought to exploit the BCI advantages. Most of the work in this area only applies BCI to replace dysfunctional body parts or control external devices.

Due to the ageing of the world population, the number of people affected by stroke will increase substantially over the coming years. Stroke affects the quality of life of the patients in the workplace, home, and community.

In the past few years, many research institutes have developed robotic rehabilitation systems [3,4]. Depending on the patient's condition, programmable levels of assistance can be provided by a robot [5,6]. Moreover, the patient's rehabilitation process can be tracked and suggestions to the therapists can be made whenever necessary.

The existing robotic rehabilitation systems supply stroke patients with effective tools. However, the patient can usually only passively follow the recorded trajectories and the patient's attention processes and motor initiatives may not be fully explored. Moreover, for robot-aided rehabilitation therapy, the improvement in motor control of the impaired up limb may not result in a consistent improvement of the functional abilities of the stroke patients.

While robot-aided rehabilitation therapy constitutes the key factor in rehabilitation training, it plays a more and more important role in the rehabilitation of stroke [7,8]. Combing noninvasive BCI with robot-aided rehabilitation can be expected to have a major impact on rehabilitation therapy.

Jiang et al. [9] developed a BCI system to detect the gait initiation from movement related cortical potentials. Electroencephalograph (EEG) signals from 9 healthy subjects were processed to adopt as templates. Artifacts were removed using independent component analysis and gait initiation was detected by a matched filter. The experimental results demonstrated that the intention of gait initiation could be extracted from EEG signals. This research laid a foundation for designing new gait therapy strategies. In [10], a hybrid BCI system was utilized to drive functional electrical stimulation (FES) for hand rehabilitation in subjects with tetraplegia. The BCI system detected the participants’ intent and drove the FES device accurately. The participants received the BCI-based rehabilitation training in addition to traditional therapy. This paper showed the feasibility of BCI-FES using only two brain rhythms, and the sensory imagery could be enhanced by visual feedback. In [11], a hybrid upper limb exoskeleton was developed for neurorehabilitation. EEG-based BCI was used to trigger the movement execution. Movement imagination tasks classification and movement intention detection were utilized to interact with the exoskeleton. The experimental results demonstrate that a hybrid upper limb exoskeleton in combination with BCI could be adopted for rehabilitation exercises.

In addition, Shin et al. [12] presented an EEG-based brain mapping system. EEG topographical maps were compared during motor execution and movement imagination tasks. This research laid a foundation for realizing a neurofeedback system for children with cerebral palsy or stroke patients. You et al. [13] and Jiang et al. [14] studied the virtual reality (VR) intervention on locomotor recovery in patients with stroke. These papers provided evidence for neuroplasticity and associated locomotor recovery after VR.

In this research, a robotic upper extremity neurorehabilitaion system based on EEG of motor imagery is designed. The rehabilitation system translates the motor imagery of upper limb movements into commands to trigger the WAM robot to stretch the impaired upper extremity to perform passive recovery training. We focus on extracting the subject's movement intentions from motor imagery EEG and controlling the WAM robot to perform rehabilitation therapy effectively, rather than to decode multiple degree-of-freedom (DoF) upper limb movements from EEG signals. Therefore, not only can we get high classification accuracy of upper limb movement imagination EEG, but also design multiple trajectories to perform rehabilitation exercises according to the recommendations of the therapists.

Subjects and methods

Experimental setup

The experimental setup is shown in Figure 1(a). The multi-channel EEG amplifier is responsible for acquiring motor imagery EEG. Several EEG studies have demonstrated that imagination of upper limb movements activates primary sensorimotor areas, which gives rise to an amplitude suppression or an amplitude enhancement of EEG signals [15,16]. In this paper, regions of interest (ROIs) were chosen around the primary sensorimotor cortex (Figure 1(b)). EEG signals were acquired using two bipolar channels around C3 and C4 according to the international 10–20 standard electrode placement. The sample rate is 128 Hz.

Figure 1. BCI-based upper extremity rehabilitation system (a) and EEG electrode positions (b).

Furthermore, the master PC is in charge of recording and processing the motor imagery EEG signals in order to supply the WAM Arm with high-level command. A slave PC is responsible for receiving the high-level command from master PC, running the control loop and providing the WAM robot with low-level command. Following the generated low-level command, the WAM robot will help the subject to perform various exercises.

Interaction between master and slave PC is based on TCP-IP protocol. The master PC running the BCI software is the server. The slave PC which is responsible for controlling the WAM Arm acts as client. After each BCI stimulus, the EEG signals are processed and a command is generated. Then, the high-level command is sent to the slave PC and the communication is implemented via TCI-IP sockets.

Control architecture

The control architecture of the BCI-based rehabilitation system is illustrated in Figure 2. The pattern recognition algorithm [17] for motor imagery EEG is realized in the master PC. Fuzzy-based position controller runs on the slave PC. According to the command sent by the master PC, the slave PC controls the WAM robot to help the subject to perform recovery training using the sinusoidal trajectory.

Figure 2. Control scheme of robot-aided upper-extremity rehabilitation system.

Position controller

The block diagram of the PID position controller based on fuzzy logic in joint space is shown in the bottom part of Figure 2. The regulator has separate fuzzy logic for $P$, $I$ and $D$ parts. The inputs of the fuzzy logic are ${\theta }_e$ and ${\theta }_{ec}$, where ${\theta }_e$ is considered as joint position error and ${\theta }_{ec}$ is its error change. The output of the fuzzy logic is $\Delta K_P$, $\Delta K_I$ and $\Delta K_D$. The torque applying to WAM is:(1) $\begin{array}{l}\tau =(K_P+\Delta K_P)({\theta }_d-\theta )+(K_I+\Delta K_I)\\ {\displaystyle {\times \int }_0^t({\theta }_d-\theta )}dt+(K_D+\Delta K_D)\frac{d({\theta }_d-\theta )}{dt}\\ +\hspace{0.17em}M\cdot Gscale\cdot g,\end{array}$(1) where $K_P$, $K_I$ and $K_D$ are the parameters for the traditional PID controller; $\Delta K_P,$ $\Delta K_I$ and $\Delta K_D$ are the output from the fuzzy logic system; ${\theta }_d$ and $\theta$ is the WAM Arm desired and actual joint position; $M$ is the link mass vector of the WAM; $Gscale$is the gravity compensation coefficient. The membership functions of the inputs and outputs for fuzzy-based PID controller are shown in Figures 3–7. Tables 1–3 illustrate the inference rules for $\Delta K_P,$ $\Delta K_I$ and $\Delta K_D$.

Figure 3. Membership degree function of ${\theta }_e$.

Figure 4. Membership degree function of ${\theta }_{ec}$.

Figure 5. Membership degree function of $\Delta K_P$.

Figure 6. Membership degree function of $\Delta K_I$.

Figure 7. Membership functions of $\Delta K_D$.

Table 1. Inference rule table of $\Delta K_P$.

	${\theta }_{e\text{c}}$
${\theta }_e$	NB	NM	NS	ZE	PS	PM	PB
NB	NB	NM	NM	ZE	ZE	ZE	ZE
NM	NM	NM	NS	NS	NS	ZE	ZE
NS	NM	NM	NS	NS	ZE	ZE	ZE
NE	ZE	ZE	NS	ZE	ZE	ZE	ZE
PS	ZE	ZE	NS	ZE	PS	PM	PM
PM	ZE	ZE	ZE	PS	PM	PM	PB
PB	ZE	ZE	PS	PS	PM	PB	PB

Table 2. Inference rule table of $\Delta K_I$.

	${\theta }_{e\text{c}}$
${\theta }_e$	NB	NM	NS	ZE	PS	PM	PB
NB	NB	NM	NM	ZE	ZE	ZE	ZE
NM	NM	NM	NS	NS	NS	ZE	ZE
NS	NM	NM	NS	NS	ZE	ZE	ZE
NE	ZE	ZE	NS	ZE	ZE	ZE	ZE
PS	ZE	ZE	NS	ZE	PS	PM	PM
PM	ZE	ZE	ZE	PS	PM	PM	PB
PB	ZE	ZE	PS	PS	PM	PB	PB

Table 3. Inference rule table of the $\Delta K_D$.

	${\theta }_{e\text{c}}$
${\theta }_e$	NB	NM	NS	ZE	PS	PM	PB
NB	NB	NB	NM	NS	ZE	ZE	ZE
NM	NB	NM	NM	NS	NS	ZE	ZE
NS	NM	NM	NS	NS	ZE	ZE	ZE
ZE	NS	NS	NS	ZE	ZE	ZE	ZE
PS	ZE	ZE	NS	PS	PS	PM	PM
PM	ZE	ZE	ZE	PS	PM	PM	PM
PB	ZE	ZE	ZE	PS	PM	PB	PB

Subjects

Seven healthy adults (four females, three males, aged between 22 and 29) took part in the experiment. All the subjects were volunteers from Southeast University in Nanjing, China. All participants gave their written consent. The study was approved by the local ethics committee.

Experimental paradigm

The subjects faced two 19'' monitors at a distance of 1.2 m. The first 2 s it was quiet. At t = 2 s, the word ‘Attention’ appeared for 1 s to indicate the start of the trial. Then, the master PC displayed the animation of left or right upper extremity movement between t = 3 and 7 s. At the same time, the subject was instructed to imagine movement of upper extremity. Next, the online pattern recognition algorithm running on the master PC processed the motor imagery EEG and the high-level command was sent to the slave PC via TCI-IP sockets. Finally, the slave PC controlled the WAM to help the subject perform rehabilitation exercises.

The experiments consisted of two sessions for each subject. Each session was conducted in three different conditions: without a participant, passive movement of a healthy participant, and with disturbance. Each case consisted of 10 trials for each of the two cases (see Table 4).

Table 4. Two different cases of passive exercises.

Conditions	Movement type	Position controller	Movement trajectory
I	Horizontal flexion/ extension exercises	Traditional PID controller	Sinusoidal trajectory
II	Horizontal flexion/ extension exercises	Fuzzy PID controller	Sinusoidal trajectory

For the left upper extremity exercises, if the motor imagery of the left upper extremity is decoded correctly, the affected left upper extremity will be manipulated by the WAM Arm to perform rehabilitation training. In case the motor imagery of the right upper extremity is recognized, feedback of virtual right upper extremity movement is given by the master PC so that the motor imagery process can be reinforced. On the contrary, for the right upper extremity rehabilitation exercise, if the motor imagery of the left upper extremity is detected correctly, virtual left upper extremity movement is provided by the master PC in order to enhance the process of motor imagery. The WAM Arm manipulator will assist the right upper extremity to carry out training on condition that the movement imagination of the right upper extremity is recognized.

Experimental results

Horizontal flexion/extension exercises were performed on the EEG-based upper limb robotic neurorehabilitation system. Three different conditions, without a participant, with a passive participant and with disturbance, were studied. Moreover, sinusoidal movement trajectory was adopted to demonstrate the effectiveness of the presented fuzzy-based PID position controller.

The BCI performance was evaluated using the classification accuracy of motor imagery task calculated during online rehabilitation training. Figure 8 illustrates the average classification accuracies obtained for all subjects in three different cases. It can be observed from the results that the method of combining harmonic wavelet with AR model is an effective tool to classify the motor imagery EEG.

Figure 8. Average classification accuracy of each participant.

The trajectory tracking performances of fuzzy and traditional control methods in regard to the WAM Arm's third joint were discussed. The movement range of the upper limb was defined as 0.45 rad in flexion and −0.45 rad in extension. Parameters $K_P$, $K_I$ and $K_D$ of the conventional PID control method were selected as 600, 2.5 and 10, respectively. The output variables of the fuzzy position controller were set as $K_P$ ∈ [−100, 100], $K_I$ ∈ [−0.5, 0.5] and $K_D$ ∈ [−2.5, 2.5].

Figures 9–11 present the results of fuzzy and conventional PID control methods in three different conditions. The results demonstrate that the proposed position controller based on fuzzy logic achieves better performance in terms of its small trajectory tracking error and smooth moving velocity than conventional controller. The means and standard deviations (MSD) of the trajectory tracking errors and the maximum of the absolute error (MAE) were used to evaluate the two position controllers. Table 5 shows the corresponding MSD and MAE values of the two control methods. In three different cases, it is obvious that the fuzzy PID control method achieves smaller MSD and MAE values than the conventional PID controller and better position control accuracy are obtained.

Figure 9. WAM Arm's third joint position control in horizontal flexion/extension exercises without a participant.

Figure 10. WAM Arm's third joint position control in horizontal flexion/extension exercises with a healthy participant.

Figure 11. WAM Arm's third joint position control in horizontal flexion/extension exercises with passive movement of participant and disturbance.

Table 5. Third joint position control performance of classic PID and fuzzy PID.

	MSD			MAE
	Without subject	With subject	With disturbance	Without subject	With subject	With disturbance
Traditional PID	−3.5E-05 ± 0.0008	6.1E-06 ± 0.0043	1.6E-04 ± 0.0025	0.0032	0.0082	0.0180
Fuzzy PID	−1.0E-07 ± 0.0004	−1.2E-06 ± 0.0005	6.3E-05 ± 0.0016	0.0016	0.0018	0.0084

The joint torque of the two control methods in the horizontal flexion/extension rehabilitation exercises are shown in Figures 12 and 13. In the passive movements of a healthy participant, it is clear that the controller based fuzzy logic has less vibration than the classic controller. From Figure 13, we can see that a large torque overshoot occurs in the classic PID controller when a disturbance is generated. Nevertheless, small vibration is achieved by the fuzzy-based controller.

Figure 12. Torque of the third joint in horizontal flexion/extension exercises with passive subject.

Figure 13. Torque of the third joint in horizontal flexion/extension exercises with disturbance.

Compared to the electromyography (EMG) or vision based rehabilitation system [18,19], the proposed system not only can be expected to fully explore the movement intention of subjects, but also can stretch the upper limb to perform recovery exercises smoothly with multiple trajectories recommended by therapists. Moreover, we can also combine the EMG or vision into our EEG-based system and carry out more research on the EEG processing for the purpose of predicting the recovery of the upper limb.

Conclusions

In this paper, we describe a BCI-based upper limb robotic rehabilitation system. Experimental results on seven healthy subjects show that: (1) The rehabilitation system can fully explore the movement intention of subjects; (2) Even in the condition of unpredictable disturbance, the PID position control strategy based on fuzzy logic can control the WAM robot stably and smoothly to help the participants to carry out recovery training; (3) Combining robot-aided training with BCI to make future rehabilitation therapy more effective is feasible. In addition, the present study mainly focused on fully exploring the subject's movement intention and attention from movement imagination EEG and designing a position controller to drive the WAM robot smoothly. Important future work is to utilize the proposed BCI-based upper limb robotic rehabilitation training system to stoke patients to further evaluate its applicability and effectiveness. Other methods to monitor human state information will be intergraded into the rehabilitation system. For example, electrocardiogram (ECG) signals can be utilized to monitor the patient's heart rate to detect their exhaustion, and EMG signals can be used to understand the recovery process of stroke patients.

Acknowledgments

This work was supported by the National Key Research and Development Program of China under grant number 2016YFB1001303, the National Natural Science Foundation of China under grants number 61673114, 91648206, 61325018, 61673105 and 61502096, the Natural Science Foundation of Jiangsu Province under grants number BK20141284 and BK20141426, and the Fundamental Research Funds for the Central Universities under grant number 2242016R30025.

Disclosure statement

The authors declare no conflict of interest.

Additional information

Funding

This work was supported by the National Key Research and Development Program of China [grant number 2016YFB1001303]; the National Natural Science Foundation of China [grants number 61673114], [grants number 91648206], [grants number 61325018], [grants number 61673105], [grants number 61502096]; the Natural Science Foundation of Jiangsu Province [grants number BK20141284], [grants number BK20141426] and the Fundamental Research Funds for the Central Universities [grant number 2242016R30025].