Neural network prediction of bed material load transport

Abstract

Bed material load, which comprises bed load and suspended load, has been extensively studied in the past few decades and many equations have been developed, but they differ from each other in derivation and form. If a process can be related to various flow conditions on a general basis, a proper understanding of bed material load movement can be ascertained. As the process is extremely complex, obtaining a deterministic or analytical form of it is too difficult. Neural network modelling, which is particularly useful in modelling processes about which knowledge of the physics is limited, is presented here as a complimentary tool for modelling bed material load transport. The developed model demonstrated a superior performance compared to other traditional methods based on different statistical criteria, such as the coefficient of determination, Nash-Sutcliffe coefficient and discrepancy ratio. The significance of the different input parameters has been analysed in the present work to understand the influence of these parameters on the transport process.

Editor D. Koutsoyiannis

Citation Kumar, B., 2012. Neural network prediction of bed material load transport. Hydrological Sciences Journal, 57 (5), 956–966.

Résumé

Le charriage, qui comprend une charge de fond et une charge en suspension, a été abondamment étudié au cours des dernières décennies et de nombreuses équations ont été proposées, qui diffèrent les unes des autres par leur mode d'élaboration et par leur forme. Si, de façon générale, le processus peut être relié aux conditions de l'écoulement, on peut aussi chercher á comprendre correctement le mouvement des matériaux charriés. Comme le processus est extrêmement complexe, il est trop difficile d'en obtenir une forme déterministe ou analytique. La modélisation par réseaux de neurones, qui est particulièrement utile dans la modélisation des processus dont on connaît imparfaitement la physique, se présente ici comme un outil complémentaire pour le modélisation du charriage. Le modèle développé a démontré une performance supérieure á celles des méthodes traditionnelles basées sur différents critères statistiques, tels que le coefficient de détermination, le coefficient de Nash-Sutcliffe et le taux de divergence. Nous avons dans ce travail analysé la signification des différents paramètres d'entrée afin de comprendre leur influence sur le processus de transport.

Key words:

• alluvial channel
• bed material load
• critical shear
• input significance
• neural network

Mots clefs:

• chenaux alluviaux
• charriage
• cisaillement critique
• signification des données d'entrée
• réseau de neurones

INTRODUCTION

Understanding sediment transport in alluvial channels is of vital importance to both sedimentological and engineering studies. Transport of sediment particles is commonly divided into suspended load and bed load. By definition, suspended load is material that is buoyed by the water motion and bed load is the material that moves near to the bed by traction and saltation. The bed material load is the sum of the bed load and the suspended load. The bed material load transport in alluvial rivers is the principal link between river hydraulics and river form (Gomez 2006) and is responsible for building and maintaining the channel geometry (Goodwin 2004). It provides the major process linkage between the hydraulic and material conditions that govern river-channel morphology, and knowledge of transport is required not only to elucidate the causes and consequences of changes in fluvial form, but also to make informed management decisions that affect a river's function. There are two general approaches to the determination of total load. The first is to compute bed load and suspended load separately and then add them. A number of sediment transport models designed to describe bed load and suspended load have been formulated, e.g. Meyer-Peter and Müller (1948), Einstein (1950), Bagnold (1966), Yalin (1963) and Chang et al. (1967). The second is to determine bed material load directly, as done by Graf (1971), Karim and Kennedy (1990), Yang (1996), Yang and Lim (2003) and Yang (2005). Depending on the conditions under which data are collected, the same formulas could have different accuracy ratings, and also may not agree with each other and observed data; hence it is difficult for an engineer to select an appropriate formula for a given river (Vanoni 1975, Yang 1996). As bed material load transport is an immensely complex process, the expression of the transport process through a deterministic mathematical framework may not be possible in the foreseeable future. In parallel with research into sediment transport, there has been an emergence of new modelling paradigms such as data mining (DM). This has opened up new opportunities for modelling processes about which the level of available knowledge is too limited to put the relevant information into a mathematical framework. DM is presently being utilized in almost all branches of science as a complementary alternative to the more traditional physically-based modelling system. Use of artificial neural networks (ANN) remains in the forefront of this complementary modelling practice. ANN modelling has been used in a wide variety of applications. Neural network techniques have been used to study several hydrological and hydraulic phenomena, including water quality, streamflows, rainfall, runoff, sediment transport, and to infill missing data (Dolling and Varas 2002, Rajurkar et al. 2002, Tayfur 2002, Cigizoglu 2003, Hu et al. 2005, Giustolisi and Simeone 2006). ANNs have also been successfully applied to sediment discharge prediction by Nagy et al. (2002) and Cigizoglu (2002). Kisi (2004) predicted suspended sediment concentration by using neural networks. Caamaño et al. (2006) used ANN techniques to derive the bed load sediment transport formula. Doğan et al. (2007) applied ANN modelling to predict total load. Sasal et al. (2009) used ANN modelling to predict bed load transport. The primary application of ANNs involves the development of predictive models to forecast future values of a particular response variable from a given set of independent variables. Recent studies have provided a variety of methods for quantifying and interpreting the contributions of the variables in the neural networks (Olden and Jackson 2002, Gevrey et al. 2003). As mentioned earlier, a proper understanding of the parameters governing the transport process is of paramount importance in the planning and design of hydraulic structures in alluvial settings. Thus the methods developed for input significance testing through ANN have been applied in the present work to find which are the most important parameters governing the bed material transport.

DATA STRUCTURE

Brownlie (1981) produced one of the most comprehensive compilations of existing flume and field data in this area, and the present work is based on those data (1767 in total), which consist of both field and flume type data. Each data set has complete records of the flow discharge, channel width, flow depth, hydraulic slope, median sediment size, sediment gradation, specific gravity of sediment and temperature. Observations with parameter value of –1 were discarded because of incompleteness, as mentioned in Brownlie (1981). Variables used to characterize the bed material load transport are as follows:

–	Channel geometry: b (width of the channel), y (flow depth) and BF (bed form of the channel)
–	Dynamic properties: Q (channel discharge), Sf (friction/energy slope), τ b (bed shear stress) and τ c (critical shear stress or Shields' shear stress)
–	Sediment properties: d (mean size of sediment), σ (gradation coefficient of the sediment particles) and Gs (specific gravity)
–	Fluid properties: ν (viscosity)

Bed material load transport (C) is a function of all the above parameters, i.e.:

(1)

The idea of including bed form in the analysis is due to the fact that bed form is generally classified based on the increasing order of bed shear stress (Southard and Boguchwal 1990). The observed bed form types are indicative of the transport condition. Bed form type and number as defined by Vanoni (1975) was considered in the present analysis; the classification is as follows: plane bed near or before initiation of motion (1), ripples (2), dunes (3), transition (4), plane bed (5), standing waves (6), and antidunes (7). The Brownlie (1981) database gives all the values of the different data sets except for values of bed shear stress and critical shear stress. Calculation of bed shear stress depends upon the aspect ratio (ratio of channel width to flow depth) of the channel (Prasad 1991). Prasad (1991) states that for an aspect ratio b/y ≥ 4, bed shear can be computed by using flow depth. The Williams (1970) method (converted to SI) has been adopted for the smooth-wall correction for the observations having b/y < 4:

(2)

Critical shear is the shear related to the incipient motion condition. Incipient motion of bed material in a channel due to flowing water refers to the beginning of movement of bed particles that previously were at rest. Incipient motion of sediment is an important process, because it represents the difference between bed stability and bed mobility. Many researchers have used the explicit form of Shields (1936) relationship (Hager and Oliveto 2002, Rao and Sreenivasulu 2006). Here, the following relationship given by Rao and Sreenivasulu (2006) has been used to calculate critical shear stress:

(3)

where R _* = u _* d/v and u _* is shear velocity. Bed material load transport occurs when bed particles have enough momentum such that the bed shear stress exceeds the critical shear stress (Owen 1964). Figure 1 shows the calculated bed shear stress and critical shear stress of all the observations, and shows that bed shear stress is distinctly higher than the critical shear stress of the observations considered in the present analysis.

Fig. 1 Estimated value of bed shear stress and critical shear stress.

In the present analysis, equation (1)(1) shows the dependent and independent quantities or, in terms of ANN modelling, the input and output vectors.

ANN MODELLING

Artificial neural networks, coupled with an appropriate learning algorithm, can be used to learn complex relationships from a set of associated input–output vectors. The most versatile learning algorithm for the feed-forward layered network is back-propagation (Irie and Miyanki 1988). Hornik et al. (1989) proved that neural networks have the ability to find any nonlinear relationship between inputs and outputs without having a priori knowledge about the system, provided sufficient hidden nodes and hidden layers are chosen. According to Hornik et al. (1989), one hidden layer is a universal approximator of a physical process. In a neural network, the knowledge lies in the interconnection weights between neurons and the topology of the net (Jones and Hoskins 1987). Therefore, an important aspect of a neural network is the learning process whereby representative examples of the knowledge to be acquired are presented to the network so that it can integrate this knowledge within its structure. Learning implies that the processing element somehow changes its input/output behaviour in response to the environment. The learning process thereby consists in determining the weight matrices that produce the best fit of the predicted outputs over the entire training data set. The basic procedure is to first set the weights between adjacent layers to random values. An input vector is then impressed on the input layer and is propagated through the network to the output layer. The difference between the computed output factor of the network and the target output vector is then adapted with weight matrices using an iterative optimization technique in order to progressively minimize the sum of squares of the errors. The major drawback of the back-propagation algorithm is that it is affected by local minima. Various other modifications to back-propagation have been proposed and the Levenberg-Marquardt modification (Hagan and Menhaj 1994) has been found to be very efficient in comparison to others such as the Conjugate gradient algorithm or Quasi-Newton algorithm. Levenberg-Marquardt works by making the assumption that the underlying function being modelled by the neural network is linear. Based on this assumption, the minimum can be determined exactly in a single step. The calculated minimum is tested, and if the error there is lower, the algorithm moves the weights to the new point. This process is repeated iteratively on each generation. Since the linear assumption is ill-founded, it can easily lead Levenberg-Marquardt to test a point that is inferior (perhaps even wildly inferior) to the current one. In order to avoid this, the Levenberg-Marquardt algorithm determines the new point through a compromise between the step in the direction of steepest descent and the above-mentioned leap. The equations for changing the weights during training in the Levenberg-Marquardt method are given as follows:

(4)

where J is the Jacobian matrix of the derivative of each error to each weight, μ is a scalar and e is an error vector. The Levenberg-Marquardt algorithm performs very well and its efficiency is found to be several orders above the conventional back propagation with learning rate and momentum factor. All the modelling and analysis was done by the use of the neural network toolbox of the MATLAB® software.

Determining the number of neurons in the hidden layer is also an important issue. The common strategy of the constructive methods is to start with a small network, train the network until the performance criterion has been reached, add a new neuron and continue until a “global” performance in terms of error criterion has reached an acceptable level (10^-5). As recommended by Masters (1993), observations are randomly divided into two statistically consistent sets: a training set for model calibration and an independent validation set for model testing. For development of the ANN model here, two-thirds of the total data were used for training and one-third for validation. It should be noted that, like all empirical models, ANNs perform best in interpolation rather than extrapolation (Masters 1993); consequently, the extreme values of the available data are included in the training set. The statistics of the data used for the training and testing sets are given in Table 1. Once the data had been divided into training and testing sets, the input and output variables were preprocessed by scaling them between 0.0 and 1.0 to eliminate their dimension and to ensure that all variables receive equal attention during training. The simple linear mapping of the variables' practical extremes to the neural network's practical extremes was adopted for scaling, as it is the most common method for this purpose (Masters 1993). As part of this method, for each variable x with minimum and maximum values x _min and x _max, respectively, the scaled value of x_n is calculated as follows: x_n = (x – x _min)/(x _max – x _min). The final architecture of the neural net used in the analysis is 11-16-1 and given in Fig. 2.

Fig. 2 Neural network architecture.

Table 1  Descriptive statistics of the observations used in the analysis. Bed forms considered in the present analysis are: plane bed near or before initiation of motion (1), ripples (2), dunes (3), transition (4), plane bed (5), standing waves (6) and antidunes (7)

	Q (m^3/s)	b (m)	y (m)	Sf	d (m)	τ b (N/m^2)	τ c (N/m^2)	σ	Gs	ν (m^2/s)	C ppm by mass
Training data set
Mean	16.12	8.67	0.315	0.0045	0.0012	4.48	0.82	1.40	2.65	1.09E-06	3398.36
Standard deviation	67.227	29.26	0.629	0.0048	0.0024	5.06	1.90	0.39	0.03	1.24E-07	6257.11
Minimum	0.00059	0.131	0.0079	0.000045	0.00019	0.26	0.12	1	2.25	6.47E-07	3.71
Maximum	486.82	162.43	4.29	0.027	0.027	51.09	21.79	3.85	2.68	1.57E-06	52238
Count	1200	1200	1200	1200	1200	1200	1200	1200	1200	1200	1200
Testing data set
Mean	15.80	7.51	0.314	0.0044	0.0014	4.53	1.00	1.40	2.64	1.01E-06	3044.88
Standard deviation	70.04	26.52	0.667	0.0048	0.0028	5.66	2.32	0.42	0.03	1.21E-07	5385.15
Minimum	0.00091	0.134	0.013	0.000058	0.00019	0.29	0.14	1	2.25	6.89E-07	4.2
Maximum	451.39	140.22	4.24	0.0258	0.026	47.54	20.98	3.78	2.68	1.59E-06	36100
Count	567	567	567	567	567	567	567	567	567	567	567
Q: channel discharge; b: width of the channel; y: flow depth; S f : friction/energy slope; d: mean size of sediment; τb: bed shear stress; τc: critical shear stress or Shields' shear stress; σ. gradation coefficient of the sediment particles; G s : specific gravity; ν. viscosity; and C: bed material load transport (equation (1)(1)).

The transfer function used in the hidden layer and the output layer is tan sigmoid. The maximum number of epochs was set to 5000. The idea behind choosing sigmoid functions as transfer functions is that they bear a greater resemblance to the biological neurons. In the case of sigmoid functions, the output of the neurons varies continuously but not linearly with the input.

RESULTS AND DISCUSSION

Results of the neural modelling are shown in Fig. 3, which clearly shows that the linear coefficient of correlation is very high between the observed data and values predicted through neural nets; the values were 0.977 and 0.957 in training and testing. Overall the linear coefficient of correlation is 0.969 (Fig. 3). The weight matrices of the present model and its application are given in the Appendix.

Fig. 3 ANN model results: (a) for training sets, (b) for test sets, and (c) for entire sets.

As mentioned above, there are two general approaches to the determination of bed material load. The first is to compute bed load and suspended load separately and then add them (Einstein 1950). This indirect determination method contradicts what is observed under natural flowing conditions where there is no sharp distinction between the bed load and suspended mode of transport (Yang 2005). The second approach is direct determination (Engelund and Hansen 1967, Ackers and White 1973, Yang 1996). Graf (1971) observed that difficulties involved in estimating the bed load and suspended load equations to a reliable degree are responsible for errors of as much as 100%. Stall et al. (1958) used the approaches of du Boys (1879), Schoklitsch (1934) and Einstein (1950) for field measurements and reported that Schoklitsch (1934) gave fairly good values, within 30% agreement, whereas the other two methods gave values that were completely off (about 750%). Yang and Wan (1991) observed that formulas developed by Einstein (1950), Colby (1964) and Toffaleti (1968) may not be suitable for laboratory flume and small river application. Arifin et al. (2002) carried out predictive analysis of some bed material load formulas. They observed that three equations, namely Engelund and Hansen (1967), Ackers and White (1973) and Yang (1996), which were derived using flume data, gave poor performance when tested against field data. Sinnakaudan et al. (2006) developed a multiple linear regression model to predict bed material load based on Malaysian river data with a linear coefficient of correlation of 0.67. Yang and Wan (1991), based on the discrepancy ratio, found that the Yang (1996) formula works well with a discrepancy ratio of around 0.89 to 1.02 for bed material load ranges between 1000 and 50 000 ppm by weight. Here, a comparative analysis has been made with some traditional direct determination methods and the present developed model. The bed material load formulas selected for comparison are those proposed by Engelund and Hansen (1967), Acaroglu (1968), Graf (1971), Ackers and White (1973), Brownlie (1981), Karim and Kennedy (1981), Yang (1996) and Yang (2005). The comparisons were done on the basis of R ², Nash-Sutcliffe model efficiency and discrepancy ratio with standard deviation. The Nash-Sutcliffe model efficiency coefficient (E) is used to assess the predictive power of models (Nash and Sutcliffe 1970), and is defined as:

(5)

The value of E can range from –∞ to 1.0, with higher values indicating a better overall fit and 1.0 indicating a perfect fit. An efficiency of 0 (E = 0) indicates that the model predictions are as accurate as the mean of the observed data, whereas an efficiency less than zero (E < 0) occurs when the observed mean is a better predictor than the model. The discrepancy ratio and standard deviation have been used to indicate the accuracy of the goodness-of-fit. The discrepancy ratio indicates the goodness-of-fit between the predicted and observed results. One way to measure the goodness-of-fit is by use of the average discrepancy ratio and standard deviation (Yang and Simôes 2005) based on the average value of the logarithm ratio between the computed and measured:

(6a)

and

(6b)

where n is the number of observations. For a perfect fit = 0 and σ _a  = 0.

Performance analyses of these formulas are shown in Fig. 4, and Table 2 compares the traditional methods with the present model against different indicators.

Fig. 4 Performance analyses of bed material load formulas.

Table 2  Statistical performance of bed material load transport formulas

Methods	R ^2	E		σ a
ANN model—training	0.977	0.807	0.103	0.298
ANN model—testing	0.957	0.754	0.129	0.316
ANN model—overall	0.969	0.723	0.134	0.326
Engelund and Hansen (1967)	0.659	0.597	0.268	0.756
Acaroglu (1968)	0.684	0.289	0.584	0.823
Graf (1971)	0.685	0.288	0.568	0.821
Shen and Hung (1972)	0.681	0.094	0.874	0.704
Ackers and White (1973)	0.772	0.669	0.156	0.627
Brownlie (1981)	0.78	0.689	0.442	0.814
Karim and Kennedy (1981)	0.44	0.324	0.831	0.752
Yang (1996)	0.543	0.161	0.341	0.652
Yang (2005)	0.6	0.236	0.543	0.614

Brownlie (1981) suggested that empirical models trained or calibrated purely from flume data do not generally perform well under field conditions, with the implication that scale effects may play an important role. In this context, use of the present model may give better predictive capability because the model has been trained using both field and flume data. Table 2 suggests that the present ANN model performs better than other methods when compared with different statistical measures. Model performance in all three cases: training, testing and overall is almost the same, which indicates the better generalization of the model. Thus the present model can be used in predicting the total bed material load in flumes as well as in the field.

In the neural network, the connection weights between neurons are the linkages between the input and the output of the network, and therefore are the link between the problem and the solution (Olden and Jackson 2002, Gevrey et al. 2003). A variety of methods are available for the estimation of the contribution of predictor variables in relation to the output. The present work uses three methods (Garson algorithm, connection weights and partial derivative) for input significance testing. The connection weights method calculates the product of the raw hidden-input and hidden-output connection weights between each input neuron and output neuron, and sums the products across all hidden neurons (Olden and Jackson 2002). Garson (1991) proposed a method for partitioning the neural network connection weights in order to determine the relative importance of each input variable in the network. The partial derivative method computes the partial derivatives of the ANN output with respect to the input neurons (Dimopoulos et al. 1999) by making use of the Taylor expansion to approximate the output perturbations with respect to either the input or weight perturbations. Calculations of the different approaches are tabulated in Table 3.

Table 3  Input significance test of the variables

Variables	Connection weights	Garson's algorithm	Partial derivative	Ranking
Q	3.188	16.95	0.270	1
y	3.105	16.5	0.166	2
d	2.703	14.4	0.112	3
τ b	2.214	11.8	0.105	4
τ c	2.095	11.1	0.082	5
Sf	1.957	10.4	0.080	6
BF	1.191	6.3	0.058	7
B	0.946	5	0.058	8
V	0.729	3.9	0.033	9
Gs	0.469	2.5	0.025	10
σ	0.210	1.1	0.011	11

As evident from Table 3, and expected, the dynamic variables have much more influence on the bed material load transport than any other group of parameters. Of the dynamic parameters, flow discharge and flow depth have the greater influence on the process. Each bed material load transport equation available in the literature has its own validity and range of applicability of its data set (Sinnakaudan et al. 2006). The calculations shown in Table 3 indicate that any error in determining flow discharge and flow depth may result in under- or over-estimation of bed material load transport. Table 3 also shows that determining the bed particle size is equally important.

CONCLUSIONS

Bed material load is a measure of the rate of transport of sediments in a river. Predictions on the rate of transport of sediments are required as a basis for the design of hydraulic structures, managing scour-related problems and others. Bed material load transport models are often very complex and depend on semi-empirical or empirical equilibrium transport equations that relate sediment fluxes to physical properties such as velocity, depth and characteristic sediment particle sizes. In this work, an ANN model for predicting bed material load transport has been developed using several published flume and field data sets. The performance of the developed ANN model can be said to be satisfactory given the variability of the different data sets and nature of the experimental conditions used. The nonlinear and nonparametric nature of the neural network model is desirable for field applications; it also brings about more opportunities to go wrong in the modelling and application process. Thus, it is worthwhile noting that predictions from ANN models are better when used for ranges of input variables similar to those utilized in model training. Data set sample size is another important issue in neural network modelling. Neural networks typically require larger sample sizes than conventional statistical procedures for model building and validation. In general, the larger the sample is, the better is the chance for a neural network to adequately approximate the underlying complex patterns without suffering from the problems of overfitting/underfitting. In engineering applications, errors in determining governing parameters affect the accuracy of the prediction of bed material load transport. Experimentation has inherent errors which cannot be neglected. The present analysis, based on the input significant test through an ANN model shows that for the accurate prediction of bed material load transport, there should be more accurate determination of hydraulic parameters.

Acknowledgements

The authors gratefully acknowledge the financial support that was received from the Department of Science and Technology, Government of India (SERC-DST: SR/S3/MERC/005/2010) to carry out the research work presented in this paper. The author would like to thank the anonymous reviewers and Dr Michel Lang for their valuable comments and suggestions for improving the quality of the paper.

APPENDIX

Application of the model

where Y [1] is the output value of the model; W ₁ [16 11] is the hidden input-layer weight values; W ₂ [1 16] the hidden output-layer weight values (Table A1); X [11 1] represents the input patterns and f ₁ and f ₂ are the hyperbolic tangent transfer functions. The parentheses [] represent the dimension of the matrix.

Table A1  Weight matrices

W 1											W 2
Q	ν	y	τb	BF	d	σ	Gs	b	Sf	τ c
−0.1558	−0.2773	1.0477	0.0129	0.2011	−0.2158	0.1744	−0.8004	−0.0097	−0.0665	−0.1336	−0.8695
0.0911	0.1230	−0.3697	0.0684	1.8339	0.1861	−0.2698	−0.0685	0.4674	−0.1077	−0.2067	0.2221
−0.2325	0.2858	−0.4104	0.4163	0.1193	−0.2689	−0.8905	−0.0381	−0.7755	0.7752	−0.1155	0.2097
−1.2946	0.4959	0.5904	0.5939	0.3196	−0.6002	−0.0756	−1.7632	−0.2341	−1.3411	0.8841	−0.2477
−1.1393	0.3146	0.0506	0.5158	1.1465	1.1357	0.0151	−0.2895	0.2894	1.3928	−0.2864	0.0997
−0.9666	−0.0274	−1.0328	0.5945	−0.9163	0.7381	−0.2802	−1.4654	0.0735	0.4512	−0.2167	0.0000
−0.3283	0.0803	5.6167	3.0315	1.4699	1.8563	−1.4297	−0.7516	0.8443	0.0298	0.2088	0.0815
−0.8896	0.3008	0.5761	−2.0237	0.0446	−0.3332	0.9142	0.2024	0.9183	0.2331	−1.0951	−0.0829
−0.3328	0.3336	−0.2417	−1.1195	0.1102	−0.0197	−1.7063	0.4604	0.6857	−1.0897	1.3110	−1.1683
0.8940	−0.0132	1.0262	−0.1982	−0.2299	−0.1648	−0.4126	0.1082	0.4695	−0.0200	2.2779	0.3138
0.0437	−0.1040	−4.2744	−0.1417	−0.7450	−0.2747	−1.0096	−0.1128	0.8009	0.0191	−1.7323	−0.3956
−0.1216	−0.7499	−0.7576	−0.3355	−0.3592	0.3934	−0.3788	−0.1462	0.5131	0.2008	−1.3517	−0.0991
0.1190	−0.0819	0.1587	−0.1738	0.3811	−0.9379	−1.1022	0.0056	0.9724	−0.0415	−1.8211	0.9032
0.5689	1.4315	−0.2085	−0.3312	0.4113	0.1756	−1.5672	−0.0903	−0.2907	−0.0169	−1.6850	−0.4707
−3.0459	1.0170	−0.4964	0.1447	0.3097	0.0264	−0.4860	−0.0642	−1.0142	0.8469	0.3909	−0.0211
0.7778	0.5366	0.4451	0.1627	0.1752	1.0257	−0.4383	−0.0475	0.3107	0.0392	1.0492	2.9038