Real-time simulations of human tongue movements with a reduced order model of a non-linear dynamic biomechanical model

Keywords:

• Model order reduction
• biomechanical models
• machine learning
• real-time simulation
• human tongue

1. Introduction

A 3D biomechanical model of the human tongue has been developed to study speech production (Hermant et al. 2017) and to help clinicians to predict the functional consequences of tongue cancer surgery (Buchaillard et al. 2007). To simulate movements in speech and in swallowing, such a Finite Element (FE) model requires a transient analysis in order to generate tongue trajectories over time in response to various patterns of muscles activations. It also requires a large deformation hyper-elastic framework to account for the complex non-linear geometrical and mechanical properties of tongue tissues. For all these reasons, any simulation of a one-second movement requires many hours of computation with a standard computer. This paper proposes to evaluate the Model Order Reduction (MOR) technique developed by ANSYS® called Dynamic ROM Builder (DRB), which is accessible in the ANSYS Twin Builder product. Contrary to other recent MOR techniques that propose real-time computations for quasi-static analysis (Lauzeral et al. 2019), the DRB is adapted to a transient analysis and should simulate, in real-time the non-linear dynamical behavior of the tongue model.

2. Methods

2.1. Machine learning-based MOR

Tongue deformation over time results from the time variations of m muscle commands (inputs variables) described in the model by n time varying outputs sampled at n_t regular time steps. Thus, the mechanical response of the model to muscle commands is described by the “excitation” and “output” matrices ${\mathrm{G}}_{\mathrm{m},{\mathrm{n}}_{\mathrm{t}}}$ and ${\mathrm{X}}_{\mathrm{n},{\mathrm{n}}_{\mathrm{t}}}$ characterizing respectively the command vectors and the output vectors at each of the n_t sampling time steps.

The objective of the MOR is to learn, from a limited number of sets $\{{\widehat{X}}_{\mathrm{n},{\mathrm{n}}_{\mathrm{t}}},{\widehat{G}}_{\mathrm{m},{\mathrm{n}}_{\mathrm{t}}}\}$ (called “scenarios” henceforth) a functional model of the FE model. For this, the dynamical behavior of the tongue is approximated by a Reduced Order Model (ROM) defined by two equations, $$\dot{X}\left(t\right)=f\left(X\left(t\right),G\left(t\right)\right)$$ $$X\left(0\right)=X_0,$$ where f is a non-linear function which parameters must be learned by minimizing the average quadratic error E defined as $$E=\sum _1^N\frac{\left(\frac{1}{n}{\sum }_{i=1}^n\left(\frac{1}{n_t}{\sum }_{j=1}^{n_t}{\left(X_{i,j}-\widehat{X_{i,j}}\right)}^2\right)\right)}{N},$$ with N being the number of scenarios.

The function f is implemented as a neural network similar to the one used in Siegelmann et al. (1997). A standard optimization method such as gradient descent is used in the iterative optimization process of f, which stops when error E goes below a predefined threshold. If this threshold cannot be reached, a “free variable” i(t) is iteratively added to the model in the form of an additional cell in the inputs and in the outputs of the neural network. These free variables are the way to account for high order non-linearities over time without increasing the depth of the neural network, avoiding thus the “vanishing gradient” problem.

2.2. Biomechanical model of the human tongue

The tongue model is described in Hermant et al. (2017). It is based on a FE mesh consisting of 7763 nodes and 8780 hexahedral elements. Mechanical properties of the tongue soft tissue are accounted for with a 2-parameter Mooney-Rivlin constitutive law (${\mathrm{C}}_{10}=192\mathrm{P}\mathrm{a};$ ${\mathrm{C}}_{20}=90\mathrm{P}\mathrm{a}$). Tongue viscosity is approximated with a Rayleigh model ($\mathrm{\alpha }=20;$ $\mathrm{\beta }=0.0$). Tongue quasi-incompressibility is modelled with a Poisson ratio v of 0.4999 and “no-displacement boundaries conditions” are defined on the external nodes in contact with the jaw and on the lowest boundary of the mouth floor.

2.3. Learning and evaluation data

We focused on tongue movements resulting from the activations of two muscles, the Styloglossus (SG) and the Genioglossus Posterior (GG-P), which have been shown to be important for speech production and swallowing. In all the simulations the excitation associated with each muscle is an activation that lasts 0.35 s, with an amplitude that varies from 0 to α ∈ [0.2; 1.0] and with a transition time t_α ∈ [0.05 s; 0.11 s].

For the learning of the ROM, 20 scenarios are used for the SG alone and 100 for the co-activations SG + GG-P, in which α varies independently for each muscle. The evaluation of the ROM is then conducted with 20 and 32 scenarios, respectively.

3. Results and discussion

Table 1 shows the mean RMS Error and standard deviation of the prediction of tongue trajectories made with the ROM computed over all the evaluation scenarios on the whole duration of the trajectories of the 1861 nodes of the tongue surface. It shows that the MOR algorithm can predict very accurately the movement due to single muscle activations but is less precise with two muscles. This seems coherent because tongue movement is known to be more complex when two muscles are activated (high non-linearities). This is confirmed by looking at the sagittal trajectories of some FE nodes located at the tip, the top and the back of the tongue during SG + GG-P activation (Figure 1).

Figure 1. Sagittal displacement of nodes along the horizontal and vertical axes during SG + GG-P activations. (dotted line: ROM, solid line: ground-truth: full model).

Table 1. Mean RMS error and standard deviations of the predictions of the ROM. (cf. Trajectories in Figure 1).

Activations	Mean (mm)	Sd (mm)
SG	5.13E-2	1.32E-2
SG + GG-P	1.03	3.92E-1

Figure 1 superimposes the trajectories generated by the ROM and by the full model. To better analyse where most of the errors of the ROM are located, Figure 2 plots the 3 D error map all around the tongue surface. The highest errors are located around the tip and dorsum. This is not surprising since these regions are the most affected ones by SG and GGP activations. This is consistent with the difference in complexity of the ROM when one or two muscles are activated. For single activation, no free variable is needed whereas one free variable is added for double activation. As could be expected on a theoretical level, more non-linearity induces more complexity in the ROM.

Figure 2. 3 D error map of the ROM for SG + GG-P.(Tongue tip on the left – Back on the right).

4. Conclusions

The ANSYS MOR algorithm has been used to approximate a non-linear FE model of the human tongue, whose modeling of behavior and fast movements requires transient FE analyses.

ROM have been constructed for a “simple” case (SG activation) and for a “more complex” case (SG + GG-P). Model comparison shows the necessity to add a free variable to integrate higher non-linearities. This shows the robustness of the method for various levels of complexity. Next challenge is to be able to predict tongue movement during contact situations.

Additional information

Funding

This work was supported by the ANR (ANR-11-LABX-0004) and MIAI @ Grenoble Alpes, (ANR-19-P3IA-0003).