Nonparametric Maximum Likelihood Estimation from Samples with Irrelevant Data and Verification Bias

Abstract

Suppose that some measurements come from a distribution F that is of interest and others come from another, irrelevant distribution G. Some measurements from F are verified and known to be from F. The other, unverified measurements may be from F or from G. Not all measurements from F are equally likely to be verified, and no measurement from G is ever verified. This model applies to measurements of low concentrations obtained using gas chromatography/mass spectroscopy, for example, as is shown in this article. How well a feature T(F) of F can be estimated when there are unverified data depends on what can be assumed about F, G, and the conditional probability v(x) of verifying a measurement of x from F. If F, G, and v are unrestricted, then more than one choice of (F, G, v) gives the same distribution p of the observable x, and thus T(F) cannot be uniquely estimated from data. But if the set of values of T(F) that correspond to a distribution p of x is small enough, then it is reasonable to try to estimate that set of T(F) from data. This article shows that the set of possible values of T(F) is a finite interval for some choices of T, such as the mean, and proposes estimators of the interval of possible values. An example using data from a Love Canal study shows that the partially identified set of T(F) can be sufficiently small for estimators of T(F) to be useful.

Key Words:

• Environmental data
• Identifiability
• Nondetect