Sequential Estimation in Stochastic Approximation Problem with Autoregressive Errors in Observations

Abstract

The problem considered is that of approximating the root of a function under the assumption that the observation noise is modelled as a stationary autoregressive process with unknown (nuisance) parameters. An example is given which shows that in this case the estimate provided by the usual Robbins–Monro algorithm can be arbitrarily far away from the root after any fixed number of iterations. The article proposes an adaptive stochastic approximation procedure for estimating the root uniformly in the nuisance parameters in asymptotic and non-asymptotic settings. In the asymptotic setting the procedure ensures convergence in the mean of the first order to the root of a function. In the non-asymptotic statement some additional conditions are imposed on the function, the proposed scheme ensures estimating the root with a prescribed mean absolute error uniformly in the nuisance parameters.

Keywords:

• Sequential procedure
• Autoregressive models
• Stochastic approximation
• Linear systems
• Identification
• Fixed-accuracy estimation
• AMS (1991) Subject Classification: Primary 93E20
• Secondary 93C73

Acknowledgment

We would like to thank the anonymous reviewer for careful reading of the article and many helpful comments. This research was supported by the RFFI-Grant 00-01-00880.

Recommended by A. Irle