INTERVAL ESTIMATION OF A NORMAL PROCESS STANDARD DEVIATION FROM ROUNDED DATA

ABSTRACT

In standard statistical analyses, data are typically assumed to be essentially exact. It is increasingly evident with the proliferation of digital readouts that this assumption can in practice be a poor one. We consider the analysis of data for which observed variation is potentially small in comparison to the finest unit of recording. In particular, we focus discussion on interval estimation of the parameter σ when a rounded sample comes from the N(μ, σ²) distribution with both parameters unknown. The traditional method and a (modified) rounded-data likelihood-based method are compared. We find that the likelihood-based method provides a reliable means of estimating σ, even in the face of the possibility that it is small.

Keywords:

• Coverage probability
• Gaging

Acknowledgments