Biomechanical fidelity assessment of a pick and place task with haptic feedback in virtual reality

Keywords:

• Haptics
• virtual reality
• biomechanics

1. Introduction

The development of virtual reality technologies has accelerated in the last years, leading to increasingly powerful and affordable solutions. It particularly allows low-cost solutions for studying the ergonomics of workstations upstream of the design chain, thanks to the interaction of an operator and a digital mock-up (Jayaram et al. 2006). One of the key points to be fulfilled is to ensure that the results obtained in the simulator are transferable to the real task. It is therefore necessary to ensure biomechanical fidelity, or bio-fidelity, i.e., the degree of accuracy of speed, force and feeling between a simulation and the same real task. Most of the studies dealing with the movement of low mass objects were conducted without the use of haptic interfaces (HI). However, in order to obtain a high level of biomechanical fidelity and/or for the displacement of moderate or high mass objects, it is necessary to introduce an haptic interaction (Pontonnier et al. 2013).

This study presents the evaluation of the biomechanical fidelity of a pick-and-place task interacting with a 6 degrees of freedom HI and a dedicated inertial and viscous friction compensation algorithm.

2. Methods

2.1. Experimental data

9 subjects (7 males and 2 females, aged 26 ± 3 y.o., height 1.77 ± 0.11m, weight 73 ± 13kg, all right-handed) participated to this study, giving their informed consent after a presentation of the protocol (validated by the INRIA national ethics committee). The experimental setup (Figure 1) consisted in a real worktable and a virtual table with 3 targets. Subjects performed a total of 48 sequences of 6 pick and place movements of a real object or a virtual object through a HI (Table 1) after 5 minutes of free testing the devices (VR and HI). HI was a Virtuose6D from Haption (Figure 1) and was used with and without a custom intrinsic inertial and viscous friction compensation of the HI (Hilt et al. 2017). The viscous friction was set to 3 N.s/m for the two horizontal axes. An audio signal indicated the beginning and the end of the task, imposing the movement speed. 2 mass levels and 2 movement speed (MS) levels were used. Each condition was repeated twice. Conditions were randomized except ones with real objects, which were performed first in order to subjectively evaluate the interaction with the HI. Head, trunk and right arm motions of the subject were recorded thanks to a VICON optoelectronic system (12 cameras, 100 Hz). A set of 18 markers derived from the ISB recommendations and 8 markers (4 placed on the real object and 4 from the 3 D-glasses) was used. 8 electromyographic (EMG) electrode pairs placed on the right arm (upper trapezius, deltoid anterior, median and posterior, biceps, triceps, extensor and flexor carpi ulnaris) were used. A biomechanical model was developed in CusToM library (Muller et al. 2019) to perform the inverse kinematics analysis. Geometry of the model was scaled to the subjects. A questionnaire was used to evaluate subjectively each condition.

Figure 1. Experimental setup.

Table 1. Experimental conditions and levels.

Conditions	Levels
Interaction	Real	HI		HI with compensation (HI-C)
Mass	0.47kg		1kg
Movement speed	1.5s per pick and place task		2.5s per pick and place task
Sequence	Clockwise (1-2-3)		Anticlockwise (1-3-2)

2.2 Indicators

After each sequence, the subject was asked to estimate the perceived exertion (RPE, Borg 1998), the subjective realism on a Likert scale (1-Low to 7-High) (R) and the perceived load (-3-lighter to +3-heavier) (PL) for haptic interaction.

The range of motion (ROM) of the following joints was computed from an inverse kinematics analysis of each task: shoulder elevation plan (SEP), shoulder elevation (SE), shoulder internal rotation (SIR), elbow flexion (EF), forearm pronation (FP), wrist flexion (WF) and wrist abduction (WA).

EMG were processed with a detrend and a root-mean square (time window: 200 Hz) algorithm, before being averaged for each task: upper trapezius (TRA), deltoid anterior (DA), deltoid median (DM), deltoid posterior (DP), biceps (BIC), triceps (TRI), extensor carpi ulnaris (ECU) and flexor carpi ulnaris (FCU). These results were analysed through a multifactorial analysis of variance ANOVA with a Greenhouse Geisser sphericity correction with Tukey’s HSD post hoc test.

3. Results and discussion

Subjective results were very similar to the results presented in (Sagardia and Hulin 2017) and in (Hilt et al. 2017). The increase in speed led to a decrease in subjective realism (-0.2 ± 0.1) and perceived effort (-0.2 ± 0.1) with the interaction of the HI. The viscous friction of the HI can explain the deterioration of the subjective results. The compensation method did not bring a significant improvement because of a low tuning of the method.

Table 2 shows the significant results for the ROM, mean root mean squared EMG and RPE. The increase in mass led to an increase in the effort during the task. This led to an increase in RPE and in the EMG level of the muscles involved in the movement of the object and especially lifting it, (trapezius, anterior and median deltoid, biceps and flexor carpi ulnaris). The increase of the activation of the extensor carpi ulnaris can be explained by an increase of the rigidity of the wrist due to co-contraction for heavier mass. Indeed, muscle co-contraction provide stiffer joints and increase the precision of the movement. Higher speed led greater ROM for all joints and an increase in the activity of the flexors and extensors muscles. Indeed, effects of inertia are dominant with respect to gravity in such cases (Papaxanthis et al. 2003).

Table 2. Differences between low and high level of each condition for each indicator (mean ± std). (Grey: p > 0.05, blank: p < 0.05, yellow: p < 0.01, orange: p < 0.005 and red: p < 0.001).

	Mass	MS	Real ➙ HI	Real ➙ HI-C
SEP (°)	−0.5 ± 0.2	2.5 ± 0.5	−3.3 ± 1.0	−3.6 ± 1.0
SE (°)	−0.6 ± 0.3	1.5 ± 0.3	−2.9 ± 5.8	−1.1 ± 5.7
EF (°)	−1.2 ± 0.6	5.8 ± 0.7	1.2 ± 0.9	0.9 ± 1.0
FP (°)	0.3 ± 0.2	1.0 ± 0.3	−3.0 ± 1.0	−3.8 ± 1.0
WF(°)	0.1 ± 0.5	3.9 ± 0.7	1.1 ± 0.7	1.0 ± 0.8
WA (°)	−0.3 ± 0.2	1.7 ± 0.4	−2.8 ± 0.9	−3.0 ± 0.8
TRA (mV)	12 ± 5	−3 ± 2	−3 ± 4	−4 ± 4
DA (mV)	12 ± 2	0 ± 2	−15 ± 5	−15 ± 5
DM (mV)	17 ± 3	6 ± 3	−19 ± 5	−19 ± 6
DP (mV)	3 ± 3	7 ± 3	8 ± 7	10 ± 5
BIC (mV)	34 ± 8	34 ± 7	14 ± 12	15 ± 10
TRI (mV)	0.1 ± 1.5	7 ± 2	−7 ± 4	−8 ± 4
ECU (mV)	65 ± 12	55 ± 15	−33 ± 56	−45 ± 60
FCU (mV)	37 ± 11	26 ± 10	36 ± 12	50 ± 18
RPE	2.4 ± 0.5	0.1 ± 1.0	1.7 ± 2.0	0.7 ± 1.1

Finally, interaction with HI and compensated HI led to a lower realism (5.1 ± 0.4 for HI and 4.9 ± 0.3 for HI-C) and a higher perceived effort (0.7 ± 0.2 for HI and 0.6 ± 0.2 for HI-C). However, the average EMGs were lower except for higher activation of the flexor carpi ulnaris for HI. This could explain the low tuning of the compensation method as it was based on a subjective evaluation of the haptic interaction. On the other hand, the range of motion were lower for the least important joints for the requested movements (SEP, FP and WA). This could be explained by the subjects' lack of experience with haptic interaction (8 of 9 subjects had never or almost never used a HI).

In conclusion, this study shows that for such tasks and such setups, the subjective feeling of the user using the HI do not correspond to the muscle activity. Such a result is fundamental to classify what can be transferred from virtual to real at a biomechanical level. The development of additional algorithms is needed to achieve a higher biomechanical fidelity, i.e., the user feels and undergoes the same constraints as the interaction with the same real object.

Disclosure statement

No potential conflict of interest was reported by the author(s).