A Simple Formula for Mixing Estimators With Different Convergence Rates

Abstract

Suppose that two estimators, ${\widehat{\theta }}_{S,n}$ and ${\widehat{\theta }}_{N,n}$, are available for estimating an unknown parameter θ, and are known to have convergence rates n^1/2 and r_n = o(n^1/2), respectively, based on a sample of size n. Typically, the more efficient estimator ${\widehat{\theta }}_{S,n}$ is less robust than ${\widehat{\theta }}_{N,n}$, and a definitive choice cannot be easily made between them under practical circumstances. We propose a simple mixture estimator, in the form of a linear combination of ${\widehat{\theta }}_{S,n}$ and ${\widehat{\theta }}_{N,n}$, which successfully reaps the benefits of both estimators. We prove that the mixture estimator possesses a kind of oracle property so that it captures the fast n^1/2 convergence rate of ${\widehat{\theta }}_{S,n}$ when conditions are favorable, and is at least r_n-consistent otherwise. Applications of the mixture estimator are illustrated with examples drawn from different problem settings including orthogonal function regression, local polynomial regression, density derivative estimation, and bootstrap inferences for possibly dependent data.

• Bootstrap
• Convergence rate
• Mixture estimator
• Oracle property.

Additional information

Notes on contributors

Stephen S. M. Lee

Stephen M. S. Lee is Professor (E-mail: [email protected]) and Mehdi Soleymani is Research Associate (E-mail: [email protected]), Department of Statistics and Actuarial Science, The University of Hong Kong, Hong Kong.

Mehdi Soleymani

Stephen M. S. Lee is Professor (E-mail: [email protected]) and Mehdi Soleymani is Research Associate (E-mail: [email protected]), Department of Statistics and Actuarial Science, The University of Hong Kong, Hong Kong.