SEM Modeling with Singular Moment Matrices Part II: ML-Estimation of Sampled Stochastic Differential Equations

Abstract

Linear stochastic differential equations are expressed as an exact discrete model (EDM) and estimated with structural equation models (SEMs) and the Kalman filter (KF) algorithm. The oversampling approach is introduced in order to formulate the EDM on a time grid which is finer than the sampling intervals. This leads to a simple computation of the nonlinear parameter functionals of the EDM. For small discretization intervals, the functionals can be linearized, and standard software permitting only linear parameter restrictions can be used. However, in this case the SEM approach must handle large matrices leading to degraded performance and possible numerical problems. The methods are compared using coupled linear random oscillators with time-varying parameters and irregular sampling times.

Keywords:

• coupled stochastic oscillators
• Kalman filtering (KF)
• maximum likelihood (ML) estimation
• nonlinear parameter restrictions
• oversampling
• stochastic differential equations (SDEs)
• structural equation models (SEMs)

Acknowledgments

I would like to thank the referees for detailed comments which helped me to improve the presentation of the article.

Notes

¹The intercepts are obtained from the interpolated exogenous variables . A more elaborated nonlinear treatment of (14–16) is given in Appendix B.

²The model has numerous applications, for example, the statistical analysis of sunspots (Bartlett, 1946; Arato, Kolmogoroff, & Sinai, 1962; H. Singer, 1993), in physics (for an overview see Gitterman, 2005) or in psychology (coupled oscillators; Boker & Poponak, 2004).

³The formal derivative ζ(t) = dW(t)/dt is the white noise process (cf. Arnold, 1974).

⁴The discussion on identification, embedding and aliasing is well known (cf. B. Singer & Spilerman, 1976; Phillips, 1976b; Hansen & Sargent, 1983; Hamerle et al., 1991; Hamerle, Nagl, & Singer, 1993; H. Singer, 1992a).

Note. Top: Exact EDM (Δ _t  = .5). Bottom: Linearized EDM, oversampling (J = 5; δt = .1). SEM = structural equation model; KF = Kalman filter; ML = maximum likelihood; EDM = exact discrete model.

⁵See also http://www.fernuni-hagen.de/imperia/md/content/ls_statistik/sde.zip for nonlinear filtering.

⁶One may interpret the system as interaction of married couples over time.

Note. ML = maximum likelihood; SEM = structural equation model; KF = Kalman filter.

Note. Oversampling times δt = 0.1, 0.05, 0.025. Numbers rounded to four digits. The likelihoods and evaluation times are shown in the last two rows. ML = maximum likelihood; SEM = structural equation model; KF = Kalman filter.

Note. Oversampling times δt = 0.1, 0.05, 0.025. Numbers rounded to four digits. The likelihoods and evaluation times are shown in the last two rows. ML = maximum likelihood; SEM = structural equation model; KF = Kalman filter.

⁷cf. e.g., Langevin (1908), Uhlenbeck and Ornstein (1954), Bartlett (1946, 1955/1978), Itô (1951), Stratonovich (1960), Kalman and Bucy (1961), Schweppe (1965), Nelson (1967), Bergstrom (1976a), Coleman (1968), Jazwinski (1970), Black and Scholes (1973), Jennrich and Bright (1976), Phillips (1976a), Doreian and Hummon (1976, 1979), Jones (1984), Moebus and Nagl (1983), Arminger (1986), H. Singer (1986, 1990, 1992b, 1998b), Zadrozny (1988), Hamerle et al. (1991), Hamerle et al. (1993), Oud and Jansen (2000), Oud and Singer (2008a, 2008b).

⁸See: http://www.fernuni-hagen.de/imperia/md/content/ls_statistik/sde.zip,http://www.fernuni-hagen.de/imperia/md/content/ls_statistik/publikationen/semarchive.exe. The LSDE approach was originally implemented in SAS/IML (H. Singer, 1991a).

⁹See: http://reference.wolfram.com/mathematica/note/SomeNotesOnInternalImplementation.html#14568.