An introductory survey of probability density function control

Abstract

Probability density function (PDF) control strategy investigates the controller design approaches where the random variables for the stochastic processes were adjusted to follow the desirable distributions. In other words, the shape of the system PDF can be regulated by controller design.Different from the existing stochastic optimization and control methods, the most important problem of PDF control is to establish the evolution of the PDF expressions of the system variables. Once the relationship between the control input and the output PDF is formulated, the control objective can be described as obtaining the control input signals which would adjust the system output PDFs to follow the pre-specified target PDFs. Motivated by the development of data-driven control and the state of the art PDF-based applications, this paper summarizes the recent research results of the PDF control while the controller design approaches can be categorized into three groups: (1) system model-based direct evolution PDF control; (2) model-based distribution-transformation PDF control methods and (3) data-based PDF control. In addition, minimum entropy control, PDF-based filter design, fault diagnosis and probabilistic decoupling design are also introduced briefly as extended applications in theory sense.

Keywords:

• Survey
• probability density function
• stochastic systems
• non-Gaussian distribution
• minimum entropy

1. Introduction

Since the random noises widely exist in industrial processes, relevant research has been performed to investigate modelling, control and application of stochastic processes. To simplify the system model, we can assume that all the system variables are Gaussian noises. Based on this assumption, many theoretical results and applications have been presented, for example, self-turning control, minimum variance control, linear quadratic Gaussian control and Markov jumping parameter system stochastic control have been developed by Astrom (1971), Goodwin and Sin (2014), Sain and Liberty (1971), Sworder (1969). The design objectives of all the mentioned methods only focus on the minimizing mean and variance of the system variables. As far as the linear stochastic system with Gaussian random variables is concerned, the shape of the system variable probability density function (PDF) can be fully determined by its mean and variance.

In practice, the system noises in the most of the industrial processes are not necessarily Gaussian; furthermore, the non-linearity of the systems would also result in non-Gaussian properties even if the investigated system subjected to Gaussian noises. In particular, the mean value and the variance of the system variable cannot be used as a sufficient characterization tool for the stochastic processes analysis. In other words, the PDF shape of the system variable can be adopted as the more suitable analysis tool to completely characterize the behaviour of a stochastic process. Therefore, a PDF-based control method provides the accuracy and flexibility for the control strategy design with various design requirements.

In the last 20 years, PDF control for stochastic system has been a significant research topic while a lot of relevant results have been presented. Based on the discussion above, PDF control sometimes is also called as PDF-shaping control for non-Gaussian distribution. Technically speaking, PDF-shaping control strategies can generally be classified into: (1) using the evolution of the system PDF which can be described by the partial differential equation (PDE) or inverse function; (2) transforming the PDF by orthogonal decomposition or weight-based neural network; (3) approximating the PDF or joint PDF by data and kernel density estimation (KDE) and (4) PDF optimization leads to a so-called minimum entropy control and its extended applications.

It has been shown that the PDF control methods are significant to industrial process while the practical applications have been introduced by Wang, Wang, and Guo (2009), Yue and Wang (2003b), e.g. flame shape control, paper-making process, etc. Recently, a number of new technologies inspire the control designs such as data science. Motivated by the development of data-based design, this survey not only recalls the existing methods in detail but also the theoretical extensions on data-based PDF control. In addition, the state of the art PDF-based applications are summarized as well, such as PDF-based filtering, fault diagnosis and probabilistic decoupling, etc.

2. Model-based direct evolution PDF control

PDF control can be restated as minimizing the distance between target PDF and investigated PDF. Therefore, the evolution of the investigated the PDF of the system variable becomes a key task. In other words, we can transform the PDF control to a target tracking description if the PDF can be formulated for the given stochastic dynamic systems. Thus, this section indicates the evolution of the PDF by direct approach and the control strategy is shown in Figure 1.

Figure 1. The block diagram of direct evolution PDF control.

2.1. Partial differential equation approach

Generally, Fokker-Planck (FP) equation governs the evolution of the PDF expression for the stochastic process. The continuous-time stochastic process has been used to present the stochastic system, which is described by following Ito's stochastic differential equation (SDE) (1) $\mathrm{d}{X}_t=b\left(X_t,t;u\right)\mathrm{d}t+\sigma \left(X_t,t\right)\mathrm{d}{W}_t$(1) where $X_t=(x_{1t},\dots ,x_{nt})\subset {\mathbb{R}}^n$ is subjected to the deterministic infinitesimal increments driven by the drift function b, σ denotes diffusion function and random increments which is given by the independent multi-dimensional Wiener process $W_t\in {\mathbb{R}}^m$.

It is noticed that the variable of a stochastic process can often be fully characterized by the distribution shape. Denote $\gamma (x,t)$ as the PDF for the stochastic process $X_t$ with respect to time t. Thus, the purpose here is to design a control input signal $u\in {\mathbb{R}}^l$ so that the process evolves towards the pre-specified target PDF ${\gamma }_d(x,t)$ with any given initial distribution $\gamma (x,t_0)$. In next essential step, the evolution of the PDF of $X_t$ subjected to the dynamics of the stochastic process $X_t$ can be recognized and expressed by the following FP equation (Jazwinski, 2007; Whittle, 2012): (2) $\begin{array}{rl}& \frac{\mathrm{\partial }\gamma \left(x,t\right)}{\mathrm{\partial }t}+\sum _{i=1}^n\frac{\mathrm{\partial }{b}_i\left(x,t,u\right)}{\mathrm{\partial }{x}_i}\\ & -\frac{1}{2}\sum _{i,j=1}^n\frac{{\mathrm{\partial }}^2\left({\left[\sigma \left(x,t\right){\sigma }^T\left(x,t\right)\right]}_{ij}\gamma \left(x,t\right)\right)}{\mathrm{\partial }{x}_i\mathrm{\partial }{x}_j}=0\end{array}$(2) To approximate the target PDF ${\gamma }_d(x,t)$ as closely as possible, the following cost function was adopted by Crespo and Sun (2002) (3) $J\left(\gamma ,u\right)=\int {\left(\gamma \left(x,t\right)-{\gamma }_d\left(x,t\right)\right)}^2\mathrm{d}x$(3) Therefore, a consistent framework of formulating an optimal control strategy for stochastic processes can be obtained using the FP equation and the formulated objectives with PDF.

Using the proposed framework, many results of non-linear systems that have exact steady-state PDF solutions have been obtained by Caughey and Ma (1982), Cai and Lin (1996), Wang and Zhang (1998), Elbeyli and Sun (2002), Socha and Blachuta (2000). In particular, optimal control laws can be obtained based upon a receding-horizon model predictive control framework. These control laws minimize the objectives subjected to the given constraint which is governed by the FP equation. PDF-shaping control problems for single-dimensional and multi-dimensional stochastic processes have been presented by Annunziato and Borzì (2010) and Annunziato and Borzì (2013), respectively. A numerical optimal control has been obtained such as Hamiltonian approach (Palmer & Milutinović, 2011), minimum principle for infinite dimensional systems (Fattorini, 1999), etc. Due to restrictions on stability, Pigeon, Perrier, and Srinivasan (2011) presented a switching linear controller design while an analytical solution of the PDF was given by the Fokker-Planck-Kolmogorov (FPK) equation. The FPK equation-based PDF-shaping control method has also been applied to the non-linear filter design (Challa & Bar-Shalom, 2000) and quantum systems (Annunziato & Borzi, 2012). Derived from Kolmogorov's forward equation, Ohsumi and Ohtsuka (2011) obtained the time evolution of the state PDF with a finite-dimensional control input.

The exact transient solutions of FP equations have been obtained only for very special one-dimensional non-linear stochastic systems. For some special single-dimensional stochastic non-linear systems, the exact transient solutions and exact stationary solutions to FP equations can be obtained. Particularly, the exact stationary solutions for dissipated multi-degree-of-freedom (MDOF) Hamiltonian systems can be divided into five classes and all of these solutions can be used to design feedback controller for tracking a pre-specified stationary PDF. Notice that only the stationary solution has been adopted for single-dimensional stochastic non-linear system by Forbes, Guay, and Forbes (2004b). Furthermore, Zhu and Zhu (2011) proposed an innovative procedure of feedback control design for MDPF stochastic non-linear system using the exact stationary solutions of the dissipated MDOF Hamiltonian systems.

In the above results, the stochastic disturbances are assumed to be Gaussian distribution. However, this assumption is not always the case in practice. Therefore, the generalized FPK (GFPK) equation has been used by Zeng and Zhu (2010) to deal with the multi-dimensional non-linear systems with random excitation which is non-Gaussian wide-band stationary. As shown by Zhu and Zhu (2014), non-linear systems with Poisson-white-noise are targeted a specified stationary PDF based on the GFPK equation. The procedures to design tracking controllers were summarised.

Although the full information of the stochastic distribution can be governed, the analytical solution of FP equation is very difficult to obtain which leads to the following formula approach for discrete-time processes.

2.2. Inverse formula approach

Since the PDE and SDE are difficult to obtain the analytic solutions, the controller design procedure would be simplified if the evolution of the PDF can be derived without solving PDE and SDE. In order to describe the problem, a general discrete-time input–output model has been given as follows: (4) $\begin{array}{r}{y}_k=f\left(y_{k-1},y_{k-2},\dots ,y_{k-n},u_{k-d},u_{k-d-1},\dots ,u_{k-m},{\omega }_k\right)\end{array}$(4) where $u_k$ is the control input, $y_k$ stands for the system output, ${\omega }_k$ is a random noise with known PDF ${\gamma }_{\omega }\left(x\right)$ and the dynamics of the system is characterized by function f. Then, by using the knowledge of the probability theory, the regeneration model between the control input and the output PDF of the stochastic system can be formulated as (5) $\gamma \left(y,u_k\right)={\gamma }_{\omega }\left(f^{-1}\left({\phi }_k,u_k,y\right)\right)\left|\frac{\mathrm{d}{f}^{-1}\left({\phi }_k,u_k,y\right)}{\mathrm{d}y}\right|$(5) where ${\phi }_k=[y_{k-1},y_{k-2},\dots ,u_{k-1},u_{k-2},\dots ]$ denotes all the historical inputs and outputs of the system which is represented by Equation (4(4) $\begin{array}{r}{y}_k=f\left(y_{k-1},y_{k-2},\dots ,y_{k-n},u_{k-d},u_{k-d-1},\dots ,u_{k-m},{\omega }_k\right)\end{array}$(4) ).The numerical solution for the closed-loop control system can be obtained by optimising the following performance index (6(6) $J=\sum _{k=0}^k\int {\left(\gamma \left(y,u_k\right)-g\left(y\right)\right)}^2\mathrm{d}y+u_k^{T}R{u}_k$(6) ). (6) $J=\sum _{k=0}^k\int {\left(\gamma \left(y,u_k\right)-g\left(y\right)\right)}^2\mathrm{d}y+u_k^{T}R{u}_k$(6) where $g\left(y\right)$, $u_k$ and R are target PDF, the control input and the weighting matrix, respectively.

The above control methods can be summarized by Wang (2003), where a recursive formula of the conditional output PDFs evolution was established. However, the stability analysis was not given for the closed-loop system. This resulted in some further developments for the general systems (4(4) $\begin{array}{r}{y}_k=f\left(y_{k-1},y_{k-2},\dots ,y_{k-n},u_{k-d},u_{k-d-1},\dots ,u_{k-m},{\omega }_k\right)\end{array}$(4) ). Guo, Wang, and Wang (2008) and Guo and Wang (2003a) formulated a predictive form for the output PDF. A novel identification method was presented for the stochastic systems with random parameters. In particular, the control input and the system output have been measured to online estimate the unknown PDF of the system parameters while the control input and system output have been used and the output PDF was transformed into a simple algebraic form by the generating functions. This method led to the following development of the scanning least squares algorithm (Wang, Wang, & Wang, 2006). In addition, the approach has been extended to formulate Joint PDF by Yin and Guo (2012). Once the unknown parameters' PDFs were approximated, the control design for output PDF-shaping control could be developed which has been introduced by Wang and Wang (2002a).

Note that these direct evolution approaches need to analytically formulate the PDFs of the stochastic dynamic systems, where the initial PDFs are supposed to be known. Also, the computational complexity cannot be ignored. As a summary, the direct evolution PDF methods analyse the properties of the stochastic distribution with dynamics, however, the shortcomings make these methods difficult for implementation.

3. Model-based transformed evolution PDF control

Without the loss of generality, the PDFs of the dynamic stochastic system can be restated in other formats while the control design should be simplified by the equivalent transformation. Based upon the equivalent performance index, the control strategy can be demonstrated by the block diagram (Figure 2).

Figure 2. The block diagram of transformed evolution PDF control.

3.1. Orthogonal decomposition approach

Jensen and Iwan (1991) and Iwan and Jensen (1993) used the orthogonal decomposition of stochastic functions to deal with some engineering problems with stochastic disturbances, and this approach has been wildly applied to stochastic systems with random parameters (Fang, Leng, & Song, 2003). Using this method, the stochastic control problem for systems with randomness can be equivalent to deterministic case. In order to introduce orthogonal decomposition into PDF-shaping control problem, the following discrete-time stochastic non-linear process model should be considered: (7) $x_{t+1}=f\left(x_t,u_t\right)+{\omega }_t$(7) where $x_t$ and $u_t$ stand for the process state and the manipulated variable, respectively. The general non-linear function f is analytic in terms of the arguments. The independent and identically random sequence is given as the additive disturbance term ${\omega }_t$ while t stands for integer-valued sampling index. Here, the Gram-Charlier parametrization is used to approximate the PDF since it has a close relationship with the moments while the basis functions can be given as follows: (8) ${\phi }_i\left(x\right)={\left(-1\right)}^i\frac{{\sigma }^i}{\sqrt{i!}}+\frac{d^i{N}_{\mu ,\sigma }\left(x\right)}{d{x}^i}$(8) where $N_{\mu ,\sigma }\left(x\right)$ denotes a Gaussian distribution with mathematical expectation and standard deviation. Then we can further define (9) $h_i\left(x\right)=\frac{{\phi }_i\left(x\right)}{N_{\mu ,\sigma }}$(9) Thus, the PDF can be approximated by following finite series of GC functions (10) $\gamma \left(x\right)\approx \sum _{i=1}^n{c}_i{h}_i\left(x\right)N_{\mu ,\sigma }\left(x\right)$(10) where $c_i$ denotes the coefficient which can be used to represent the distribution and the parameters of the controller can be obtained by minimizing the following objective function: (11) $min{J}_{GC}=\sum _{i=1}^n{\left(c_i-c_{i,target}\right)}^2$(11)

Remark 3.1

It is known that due to the linear relation between the moments and the Gram-Charlier coefficients, matching GC coefficients is equivalent to matching the moments (Elbeyli, Hong, & Sun, 2005; Forbes, Forbes, & Guay, 2005).

Based on the above framework, the problem of targeting a desired stationary PDF was investigated by Pigeon et al. (2011), Forbes, Guay, and Forbes (2004a), Forbes, Guay, and Forbes (2002), Forbes, Forbes, and Guay (2003b), Forbes, Forbes, and Guay (2003a), Zhang, Zhang, Pandey, and Zhao (2010), Wang (2012). Based on Gram-Charlier PDFs, Forbes et al. (2004a) and Forbes et al. (2002) investigated the PDF-shaping control for first-order stochastic processes. By Forbes et al. (2003b), the dynamic PDF approximation applications for higher-order processes have been presented using the linear control law parametrizations and Gram-Charlier PDF parametrizations. Based upon the parametrization of the closed-loop stochastic process dynamics and the the corresponding stationary PDF approximation by multivariate Gram-Charlier basis functions, the optimal control design was investigated for discrete-time stochastic processes subjected to a non-quadratic cost function (Forbes et al., 2003a).

In the above proposed approaches, by using the Gram-Charlie expansions as the PDF basis functions, a static feedback control design is presented, then the distribution of the process variable tracks a reference stationary PDF. This technique focused on the stationary state PDF. However, it is not a time-evolving PDF. For the non-linear oscillator with random excitation, Zhang, Zhang, et al. (2010) extended the orthogonal decomposition technique to the stationary response. The PDF of random variable is re-expressed by a standardized multi-variable orthogonal polynomial set. Using the Galerkin scheme, solving the FPK equation was replaced by a first-order linear ODE with unknown time-dependent coefficients. Then, uncertainty responses can be obtained with stationary and non-stationary PDFs.

3.2. Weight-based neural network approach

The main problem of orthogonal decomposition approach is that the time evolution of the state PDF has not been considered sufficiently. It shows that the implementation of the output PDF control for stochastic system subjected to non-Gaussian noise is a challenge topic in the stochastic control filed. By Wang (2012), the system output has been expressed by a PDF and the control input is formulated with respect to this PDF form. Therefore, obtaining the control input to change the shape of the output PDF following its desired PDF become the purpose of the control design. Notice that the control input is only time-related. Motivated by the practical problems of paper-making processes, a practically implementable control strategy was originally developed in 1996 by Wang (2012), which aims at adjusting the output PDF for a class of the stochastic systems with non-Gaussian noises.

Wang (2012) firstly investigated the approximation of the measurable and instantaneous PDFs of the system outputs using B-spline neural network and the neural networks parameters, such as weights and biases, are dynamically linked to the control input. Thus, the PDF control was converted into the control of the neural network parameters.

Suppose that the uniformly bounded random variable $\eta \left(t\right)\in [a,b]$ is subjected to the stochastic dynamic system output; $u_k\in {\mathbb{R}}^m$ denotes the control input, which is designed to adjust the distribution shape of the system output $y\left(t\right)$; $\gamma (y,u_k)$ is the PDF of $y\left(t\right)$. There are three types of B-spline neural networks to approximate $\gamma (y,u_k)$ which are briefly introduced by Wang et al. (2009).

3.2.1. Linear B-spline model (see e.g. Guo & Wang, 2003b; Sun, Yue, & Wang, 2006; Wang, 1998, 1999, 2000; Wang, Zhang, & Yue, 2005b; Yue, Zhang, Wang, & Cao, 2004)

(12) $\gamma \left(y,u_k\right)=\sum _{i=1}^n{B}_i\left(y\right)w_i\left(u_k\right)+e_0,y\in \left[a,b\right]$(12) where $w_i\left(u_k\right)$ stand for the weights of the estimation to the output PDF $\gamma (y,u_k)$, while $B_i\left(y\right)$ denote the pre-specified basis functions and $e_0$ denotes the estimation error. The main problem of B-spline approximation of PDF is that the trained weights can sometimes be partly negative. It is obvious that this is unacceptable since PDF must be positive.

3.2.2. Square root B-spline model (see e.g. Chen & Wang, 2007; Guo & Wang, 2004, 2005a; Wang, 2012; Wang, Baki, & Kabore, 2001; Wang, Kabore, & Baki, 2001; Wang & Zhang, 2001; Wang & Wang, 2002b; Wu, Zhang, Ma, & Guo, 2005; Zhang, Wang, Zhang, & Hou, 2009)

To overcome the disadvantage of this B-spline model, instead of approximating the PDF directly, the square root of the output PDF should be estimated using the following approach. (13) $\sqrt{\gamma \left(y,u_k\right)}=\sum _{i=1}^n{B}_i\left(y\right)w_i\left(u_k\right)+e_0,y\in \left[a,b\right]$(13)

3.2.3. Rational B-spline model

In order to consider the condition $\int \gamma (y,u_k)\mathrm{d}y=1$, the weights of the B-spline NN are constrained. Therefore, the modified model is used as follows: (14) $\gamma \left(y,u_k\right)=\frac{\sum _{i=1}^n{B}_i\left(y\right)w_i\left(u_k\right)}{\sum _{i=1}^n{b}_i{w}_i\left(u_k\right)}+e_0,y\in \left[a,b\right]$(14) where $b_i={\int }_a^b{B}_i\left(y\right)\mathrm{d}y>0$.

Since the integration of $\gamma (y,u_k)$ over its definition domain $[a,b]$ must be 1, it has been shown that only n−1 weights are independent. We can further denote $V_k$ as the vector of independent weights, then the dynamic of the system can be rewritten as (15) $V_{k+1}=f\left(V_k,u_k\right)$(15) where $f(V_k,u_k)$ is a vector function representing the dynamics of the vector-valued NN weights and the control input. Therefore, Equations (12(12) $\gamma \left(y,u_k\right)=\sum _{i=1}^n{B}_i\left(y\right)w_i\left(u_k\right)+e_0,y\in \left[a,b\right]$(12) )–(15(15) $V_{k+1}=f\left(V_k,u_k\right)$(15) ) supply the general structure of modelling the stochastic distribution systems, where the control input is time-varying and the model output is the output PDF. Then, the optimal control algorithm should be obtained by searching the optimum of the performance index (6(6) $J=\sum _{k=0}^k\int {\left(\gamma \left(y,u_k\right)-g\left(y\right)\right)}^2\mathrm{d}y+u_k^{T}R{u}_k$(6) ) minimum.

In the above framework, a lot of controller designs have been obtained to adjust the shape of the output PDF. Wang (2012) established a linear feedback controller where the control law was presented using the measured output PDFs and the input signal with linear dynamics. After that, Wang, Kabore, et al. (2001) extended this method for the stochastic non-linear systems. In addition, Wang (2012), Wang (1998), Wang (1999), Guo and Wang (2003b), Guo and Wang (2005d) analysed the robustness of the closed-loop systems. By Wang (2012), Wang, Zhang, and Yue (2005a), a novel scanning, recursive parameter approximation algorithm was presented to estimate the linear parameters in model (15(15) $V_{k+1}=f\left(V_k,u_k\right)$(15) ) using the unknown $f(V_k,u_k)$. Based upon the system dimensions, the multi-layer perceptions (MLPs) (Wang, Xiong, Wang, & Yue, 2008; Zhang, Guo, Yu, & Zhao, 2007) and radial basis functions (RBFs) (Afshar, Brown, & Wang, 2009; Skaf, Wang, & Guo, 2011; Yi, Zhan-Ming, & Er-Chao, 2012) can also be used to estimate the output PDFs.

Although the PDF-shaping problem can be well solved by using the above neural networks model, there are several problems of this method, such as (1) for the dynamics of the weighting vector, the shape of the PDFs cannot be changed if only linear models are established; (2) the high computational load is the key problem of the numerical solution and the performance is difficult to meet the requirements including the stability and the robustness of the closed-loop realization. For the practical system implementation, the fixed-structure controller has been developed to overcome the mentioned shortcomings. In particular, PID controller has been presented by Guo and Wang (2003b), Guo and Wang (2005d), Guo and Wang (2005b), Yi, Li, Guo, and Wang (2008), Yi, Shen, and Guo (2009c), Guo and Wang (2004), Guo (2006), Guo and Wang (2009), Yi, Guo, and Wang (2009b), Wang and Afshar (2009), Skaf, Al-Bayati, and Wang (2010), where the parameters of the controller can be obtained using linear matrix inequalities (LMIs). Using the fixed-structure controller design, the total dimension of the parameters can be minimized and off-line design would simplify the algorithm with stability analysis which can be considered as the advantages. Notice that a synthesis and integrated analysis framework can be built up with LMI techniques.

Although different types of neural networks can be used to replace B-spline NN as theoretical extensions, the main problem with the described B-spline approach is that the direct physical meaning of the controller design model is not convincing. Meanwhile, the complicated shape of the output PDF leads to a complex neural networks which also results in the high-dimensional dynamics between the control input and the weighting vector (see model (15(15) $V_{k+1}=f\left(V_k,u_k\right)$(15) )).

4. Data-based PDF control

In practice, it is difficult to establish system dynamic models represented practical industrial processes with non-Gaussian noises, therefore, it dose not make sense to use the mentioned model-based approach for these processes which leads to model-free stochastic distribution control strategy. Figure 3 shows the block diagram, where the indirect data-based approach is given if the model identification procedure is included otherwise the PDF can be approximated by data directly using KDE which called direct data-based approach or data-driven approach.

Figure 3. The block diagram of data-based PDF control.

As mentioned in the previous section, neural networks have been established for both modelling and control of the stochastic non-linear non-Gaussian systems by Yi, Guo, and Wang (2009b), Guo, Yi, and Wang (2009), Zhang, Liu, and Guo (2012). Using the square-root B-spline NN estimation to the measured output PDF, the problem has been converted as the dynamic weights tracking. Fix-structure NN with unknown parameters were employed to describe the dynamics between the weighting vector and the control input. Afshar, Wang, and Chai (2009) presented a dynamic neural network for modelling and control the investigated plant where the randomness of the system output has been attenuated by minimum entropy control scheme with an iterative learning control basis. The above control strategy is usually called data-driven SDC or model-free SDC. In fact, we note that these control algorithms were designed on the basis of the hidden or implicit system modelling. In other words, they are not actually model-free.

Based on the minimum error entropy (MEE) criterion, an optimal control method has been obtained for the semiconductor processes subjected to non-Gaussian noise by Zhang, Chu, Munoz, and Chen (2009). Erdogmus (2002) introduced Parzen window technique to estimate the PDF and quadratic Rényi entropy of the tracking error, and the optimized control input was directly designed based on the sample data of the tracking error without approximating the dynamic system. Motivated by information theory, Ren, Zhang, Jiang, Tian, and Hou (2012) investigated the adaptive control problem for non-Gaussian stochastic systems using single neuron. The control algorithm was developed with the neuron weights training under the generalized MEE principle while the cost function can be estimated by the Parzen windowing technique. MEE is also used by Cheng, Yue, Xing, and Ren (2018) for multipath estimation problem. The joint PDF can also be controlled by data-driven approach shown by Yin, Zhang, and Guo (2015). Naturally, data-based control can also be combined by intelligent optimization methods such as PSO, which has been reported by Yin, Zhang, Zhou, and Wang (2014). Moreover, Zhang, Zhang, Ren, Hou, and Fang (2012) and Zhang, Zhang, and Wang (2011) have proposed the neural PID controller for a class of non-linear non-Gaussian system by minimizing the entropy of the system error. Furthermore, MEE criterion can be used for parametric identification for neural interaction characterization (Zhang & Sepulveda, 2017). Similarly, the iterative learning design for non-Gaussian stochastic system has also been given by Zhou, Yue, Zhang, and Wang (2014). To avoid the weights of mean-value and entropy, survival information potential (SIP) is used to replace the MEE criterion which has been introduced by Xu, Zhao, Ren, Cheng, and Gong (2018).

The advantage of this date-based method is that the control algorithms are developed in the data-based framework, while the accuracy of the system model is not essential since the accurate model is difficult to obtain. However, the model is still needed when analysing stochastic systems. Due to the fact that the distribution is estimated by data, the control performance relay on the data set quality and the computational complexity is another shortcoming.

5. Extensions: minimum entropy control, filtering, fault diagnosis and probabilistic decoupling

In the previous sections, the control algorithms have been presented where we suppose to know the target PDF. However, the target PDF sometimes is not available. In this case, the performance criterion can be selected as the minimum entropy, see e.g. Yi, Li, Guo, and Wang (2008), Guo et al. (2008), Guo and Wang (2005c), Guo and Wang (2007), Wang and Wang (2004), Wang (2002), Yi, Guo, and Wang (2009a), Yue, Jiao, Brown, and Wang (2001), Yue and Wang (2003a), Yue and Wang (2003b), Zhang, Yu, and Guo (2006), Wang and Sun (2004), Afshar, Nobakhti, and Wang (2009), Afshar, Nobakhti, Wang, and Chai (2010). In particular, the performance criterion is defined as follows: (16) $J=-\sum _{k=0}^k\int \gamma \left(y,u_k\right)\log \left(\gamma \left(y,u_k\right)\right)\mathrm{d}y+u_k^{T}R{u}_k$(16) or defining e=r−y with r is the set point, we have (17) $J=\int \gamma \left(e,u_k\right)\log \left(\gamma \left(e,u_k\right)\right)\mathrm{d}e+u_k^{T}R{u}_k$(17) while the first term denotes the entropy of the system output in (16(16) $J=-\sum _{k=0}^k\int \gamma \left(y,u_k\right)\log \left(\gamma \left(y,u_k\right)\right)\mathrm{d}y+u_k^{T}R{u}_k$(16) ) and tracking error in (17(17) $J=\int \gamma \left(e,u_k\right)\log \left(\gamma \left(e,u_k\right)\right)\mathrm{d}e+u_k^{T}R{u}_k$(17) ), respectively. R is weighing matrix. Based upon this performance index, the PDF tracking problem is transformed to PDF optimization problem and the block diagram of the control strategy is shown in Figure 4.

Figure 4. The block diagram of PDF optimization control (minimum entropy control).

For measurable output PDFs, the B-spline approach is used to re-express the measured output PDF and the system dynamics is described by a set of differential or difference equations which link the B-spline weighting vector to the control input (see e.g. Wang & Wang, 2004; Wang, 2002; Yi, Li, & Guo, 2008; Yue et al., 2001; Yue, Zhou, & Wang, 2006) and Equation (5(5) $\gamma \left(y,u_k\right)={\gamma }_{\omega }\left(f^{-1}\left({\phi }_k,u_k,y\right)\right)\left|\frac{\mathrm{d}{f}^{-1}\left({\phi }_k,u_k,y\right)}{\mathrm{d}y}\right|$(5) ) should be considered to further formulate the relationship of the system output PDF and the control input even if the PDF is unmeasurable. Yue and Wang (2003a) presented a recursive optimization solution based on MEE control which guaranteed the local stability for the closed-loop system. The result of Yue and Wang (2003a) was extended to the stochastic system model with two inputs and two outputs (Zhang, Ren, & Wang, 2012). As a summary, the MEE principle has been widely used for control and optimization of the stochastic non-Gaussian systems (see e.g. Zhang & Wang, 2008; Zhang, Chu, et al. 2009; Zhang, Du, et al. 2012; Ren, Zhang, Zhang, & Hou, 2012).

Remark 5.1

As an optimization problem, all the mentioned frameworks can be re-organized while the performance index can be obtained by analytical PDF formula, equivalent transformed PDF or data-estimated PDF.

Filtering design and fault diagnosis are two important problems in non-linear and non-Gaussian systems. Recently, Guo and Wang (2005a), Zhang et al. (2007), Yi et al. (2012), Skaf, Ahmad, and Wang (2011), Zhang, Yu, et al. (2006), Guo, Wang, and Chai (2006), Guo, Zhang, Wang, and Fang (2006), Zhang, Du, et al. 2012, Zhang, Liu, et al. (2012), Guo, Yin, Wang, and Chai (2009b),~Li and Guo (2009), Skaf, Ahmad, and Wang (2011), Yao, Qin, Wang, and Jiang (2012), Yao, Cocquempot, and Wang (2010), Yin and Guo (2009), Zhang, Guo, and Wang (2006), Skaf, Ahmad, Wang, and Wang (2011), Ren and Wang (2010) investigate these problems using SDC concept.

Under the assumption that the output PDF was measurable, Lu, Dai, and Xue (2009) was concerned with the problem of filtering for output PDFs of the stochastic singular dynamics which is modelled by B-spline functions. By Guo and Wang (2005c), a new formulation of the residual PDF was made to link the residual PDF to the gain matrix of the filter, and the optimal filtering gain matrix was then solved by minimizing the entropy of the residual. The minimum entropy filter of Guo and Wang (2005c) has presented a better performance in reducing the randomness of the filter residual and is more general and suited for non-Gaussian systems. By Zhang, Cai, and Wang (2010), a minimum entropy filter was presented for estimating states in networked control systems with multiple-packet transmission mechanism and non-Gaussian time-delay and noises. The filter was designed for non-linear NCSs via information theoretic learning approach based on stochastic gradient algorithm. However, minimum entropy criterion may not guarantee that the estimation errors approach to zero. Following the minimum information divergence criterion, a hybrid characteristic function of the conditional estimation error was introduced to construct the performance index of the tracking filter by Zhou, Zhou, Wang, Guo, and Chai (2010). An analytical solution of the filter gain matrix was then obtained so that the PDFs of the filtering error can follow a target distribution shape. Nevertheless, it is a little complicated to calculate the analytical solution. Adaptive approaches were also investigated for solving filtering problem in non-linear non-Gaussian systems in last decades. Since MEE criterion ensures that the estimation error has small uncertainty, it was used for supervised training of non-linear stochastic system by Erdogmus and Principe (2002); Principe (2010). However, entropy does not change with the mean of the distribution and the algorithm may not yield zero-mean error. Therefore, the result may be corrected by properly modifying the bias of the output processing element of the neural networks. Maximum mutual information criterion was proposed for adaptive filtering by Chen, Hu, Li, and Sun (2008), this criterion is robust to measure distortions. Nevertheless, the maximum mutual information criterion leads to non-unique optimum solution. It is necessary to use a priori information about the unknown system in order to obtain unique solution.

Based on the designed filter, fault diagnosis problem was studied for non-linear and non-Gaussian systems. Based on the results of Guo and Wang (2005a) and Li and Guo (2009), non-linear non-Gaussian systems were described by combining an improved square-root B-spline model with a further non-linear dynamic model. Once B-spline expansions have been made for PDFs, further modelling was still needed to reveal the relationship between the input and the weights related to the PDFs, a non-linear filter was then constructed as a residual generator such that the fault can be detected and diagnosed. The proposed filter is suited for non-linear non-Gaussian systems, however, the output PDF should be measurable, moreover, it is not easy to build the state space expression of the weights related to the PDFs in practical systems. How to formulate the entropies of the output stochastic distributions has been investigated for the fault detection and the fault isolation problem of SISO systems, respectively, by Guo, Wang, and Chai (2006) and Guo, Yin, Wang, and Chai (2009b). For multivariate random vectors, the formulation of the joint PDF of the output becomes much more complicated even if the transformation between the input and output is linear (Guo & Wang, 2005c). Following the recent developments reported by Guo and Wang (2005c) and Guo, Yin, Wang, and Chai (2009b), Yin and Guo (2009) established a novel filtering scheme to isolate the target fault from the multiple nuisance faults and disturbances based on the generalized entropy optimization principle for the multivariate stochastic non-Gaussian systems. New fault diagnosis (FD) and fault tolerant control (FTC) algorithms for non-Gaussian singular stochastic distribution control (SDC) systems were presented by Yao et al. (2012), Yao, Guan, and Wang (2015) and Yao, Lei, Guan, and Wang (2016). Different from general SDC systems, in singular SDC systems, the relationship between the weights and the control input was expressed by a singular state space model, which increases the difficulty in the FD and FTC design. In addition, Jin, Guan, and Yao (2017) further presented the minimum optimization for fault diagnosis with mean constraint.

Recently, a novel decoupling control strategy entitled probabilistic decoupling was presented for stochastic dynamic systems by Zhang, Zhou, Wang, and Chai (2015) and Zhang and Yin (2018) while the couplings among the system outputs have been quantized in terms of the output PDFs. In order to minimize the couplings in probability sense, the distance between the joint PDF and the product of output marginal PDFs has been considered as performance index. Furthermore, the mutual information criterion can be used as an extended minimum entropy performance index to achieve the probabilistic decoupling (Zhang & Hu, 2018; Zhang & Wang, 2016). Based on the similar structure, the performance enhancement of the stochastic dynamic system is also investigated by Zhou, Zhang, Wang, Zhou, and Chai (2018) and Zhou, Zhang, and Wang (2016) using PDF optimization.

6. Conclusion and prospect

This survey reports the developments of the PDF control in four aspects: direct evolution, transformed evolution, data-based approximation and PDF control applications. All the methods in this category can be included as stochastic distribution control with the similar structure (Pigeon et al., 2011), basically (i) briefing the non-linearity, parametrization for stochastic processes, (ii) formulating an approximate or parametrized PDF expression and (iii) minimizing an appropriate criterion to characterize the distance between output PDFs and desired PDFs.

All the mentioned methods have their own benefits and shortcomings, in particular, the direct evolution approach needs to derive the complex analytical formula and the stability of the data-driven framework should be further analysed. Meanwhile, the orthogonal decomposition technique dose not take time evolution of PDF into account and neural network modelling does not indicate the physical meaning of the PDF evolution. Since the extended applications have also been discussed in this paper, the PDF control would affect other research fields in the future. In other words, there are still many challenges for this research topic. Particularly, the following issues can be considered as the prospects: (1) AI-inspired data-driven control method for PDF-shaping problem, such as deep learning neural networks application in stochastic control field; (2) Cyber-physical system monitoring and fault diagnosis problem can be taken into account following the existing non-Gaussian fault diagnosis methods and filter designs and (3) complete theoretical analysis of PDF for complex stochastic models with Markov-jump parameters, time-delay, uncertainties, etc.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Qichun Zhang  http://orcid.org/0000-0003-2479-8195

Additional information

Funding

This work was supported in part by the De Montfort University - DMU HEIF'18 project, in part by the Natural Science Foundation of Shanxi Province [grant number 201701D221112], and in part by the National Natural Science Foundation of China [grant numbers 61503271 and 61603136], these are acknowledged.