Unifying Amplitude and Phase Analysis: A Compositional Data Approach to Functional Multivariate Mixed-Effects Modeling of Mandarin Chinese

Figures & data

Figure 1 An example of triplet trajectories from speakers F02 and M02 over natural time. F(emale)02 tonal sequence: 4-5-1, M(ale)02 tonal sequence: 2-1-4; Mandarin Chinese rhyme sequences [oŋ-@-iou] and [ien-in-$$], respectively. See supplementary material for full contextual covariate information.

Table 1 Covariates examined in relation to F₀ production in Taiwanese Mandarin. Tone variables in a 5-point scale representing tonal characterization, 5 indicating a toneless syllable, with 0 representing the fact that no rhyme precedes the current one (such as at the sentence start). Reference tone trajectories are shown in the supplementary material section: Linguistic Covariate Information

Effects	Values	Meaning	Notation mark
Fixed effects
Previous tone	0:5	Tone of previous syllable, 0 no previous tone present	tnprevious
Current tone	1:5	Tone of syllable	tncurrent
Following tone	0:5	Tone of following syllable, 0 no following tone present	tnnext
Previous consonant	0:3	0 is voiceless, 1 is voiced, 2 not present, 3 sil/short pause	cnprevious
Next consonant	0:3	0 is voiceless, 1 is voiced, 2 not present, 3 sil/short pause	cnnext
B2	linear	Position of the B2 index break in sentence	B2
B3	linear	Position of the B3 index break in sentence	B3
B4	linear	Position of the B4 index break in sentence	B4
B5	linear	Position of the B5 index break in sentence	B5
Sex	0:1	1 for male, 0 for female	Sex
Duration	linear	10 sec of msec	Duration
Rhyme type	1:37	Rhyme of syllable	rhymet
Random effects
Speaker	N(0, σ^2speaker)	Speaker effect	SpkrID
Sentence	N(0,σ^2sentence)	Sentence effect	Sentence

Figure 2 Summary of the overall estimation procedure resulting in the estimates of the functional principal components and scores via the covariates in the linear mixed effect model.

Figure 3 The multivariate mixed effects model presented exhibits a crossed (nonbalanced) random structure. The vowel-rhyme curves (V) examined are cross-classified by their linguistic (Sentence—P_i) and their nonlinguistic characterization (Speaker—S_i).

Figure 4 Corresponding amplitude variation functions w (top row) and phase variation functions h (bottom row) functions for the triples shown in Figure 1.

Figure 5 Functional estimates (continuous curves) are shown superimposed of the corresponding original discretized speaker data over the physical time domain t.

Figure 6 W (amplitude) eigenfunctions Φ: mean function ([0.05, 0.95] percentiles shown in gray) and first, second, third, fourth, fifth, and sixth functional principal components (FPCs) of amplitude.

Table 2 Percentage of variances reflected from each respective FPC (first 9 shown). Cumulative variance in parenthesis

	Amplitude/(w)	Phase/(s)
FPC1	88.67 (88.67)	49.40 (49.40)
FPC2	10.16 (98.82)	19.25 (68.65)
FPC3	0.75 (99.57)	9.02 (77.68)
FPC4	0.22 (99.80)	6.53 (84.19)
FPC5	0.10 (99.90)	4.34 (88.53)
FPC6	0.05 (99.94)	2.98 (91.51)
FPC7	0.02 (99.97)	2.32 (93.83)
FPC8	0.01 (99.98)	1.96 (95.79)
FPC9	0.01 (99.99)	1.29 (97.08)

Table 3 Actual deviations in Hz from each respective FPC (first 9 shown). Cumulative deviance in parenthesis. (Human speech auditory sensitivity threshold ≈ 10 Hz)

	Amplitude/(w)
FPC1	121.16(121.16)
FPC2	66.52 (187.68)
FPC3	31.22 (218.90)
FPC4	17.50 (236.40)
FPC5	9.00 (245.39)
FPC6	4.86 (250.26)
FPC7	3.64 (253.90)
FPC8	2.71 (256.61)
FPC9	1.96 (258.56)

Figure 7 (Phase) eigenfunctions Ψ: mean function ([0.05, 0.95] percentiles shown in gray) and first, second, third, fourth, fifth, and sixth functional principal components (FPCs) of phase. Roughness is due to differentiation and finite grid; the corresponding warping functions in their original domain are given in Figure S.1 in the supplementary material.

Table 4 Random effects std. deviations

Estimate	$w{\text{FPC}}_1$	$w{\text{FPC}}_2$	$w{\text{FPC}}_3$	$w{\text{FPC}}_4$	Duration	${\text{sFPC}}_1$	$s{\text{FPC}}_2$	$s{\text{FPC}}_3$	$s{\text{FPC}}_4$
Speaker	89.245	6.326	3.655	1.330	2.806	0.289	0.023	0.022	0.030
Sentence	38.674	4.059	0.045	0.102	0.043	0.049	0.043	0.042	0.043
Residual	114.062	44.386	15.399	10.072	4.481	0.959	0.591	0.431	0.370

Figure 8 Random effects correlation matrices. The estimated correlation between the variables of the original multivariate model (Equation (3.19)$$\begin{array}{cc}\hfill A_{n\times p}& =X_{N\times k}{B}_{k\times p}+Z_{N\times l}{\Gamma }_{l\times p}+E_{N\times p},\end{array}$$) is calculated by rescaling the variance-covariance submatrices ${\Sigma }_{R_1}$ and ${\Sigma }_{R_2}$² of ${\Sigma }_{\Gamma }$ to unit variances. Each cell i, j shows the correlation between the variance of component in row i that of column j; row/columns 1–4: wFPC1-4, row/columns 5–8: sFPC1-4, row/columns 9: Duration.