Kernel smoothed probability mass functions for ordered datatypes

Abstract

We propose a kernel function for ordered categorical data that overcomes limitations present in ordered kernel functions appearing in the literature on the estimation of probability mass functions for multinomial ordered data. Some limitations arise from assumptions made about the support of the underlying random variable. Furthermore, many existing ordered kernel functions lack a particularly appealing property, namely the ability to deliver discrete uniform probability estimates for some value of the smoothing parameter. We propose an asymmetric empirical support kernel function that adapts to the data at hand and possesses certain desirable features. There are no difficulties arising from zero counts caused by gaps in the data while it encompasses both the empirical proportions and the discrete uniform probabilities at the lower and upper boundaries of the smoothing parameter. We propose likelihood and least-squares cross-validation for smoothing parameter selection and study their asymptotic and finite-sample behaviour.

Keywords:

• Discrete
• nonparametric

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 Unordered kernel functions place a binary counting weight on each observation, for example, $1-\lambda$ when $X_i=x$ and $\lambda /(c-1)$ when $X_i\ne x$, $\lambda \in [0,(c-1\left)/c\right]$, but cannot assess distance (see Aitchison and Aitken 1976, p. 419).

2 We are grateful to an anonymous referee who pointed out that Kokonendji and Zocchi (2010) proposed a family of symmetric and asymmetric counting kernels that encompass the properties discussed herein: they write ‘the empirical proportions and the discrete uniform distribution at the extreme of the admissible smoothing parameter’.

3 To see this, note that $d_{zi}=0$ when $z=X_i$, and when $\lambda =0$, $l(X_i,z,0)={\lambda }^{d_{iz}}/{\mathrm{\Lambda }}_i=0^0/(0^0+\sum _{y\in \mathcal{D},y\ne X_i}^c{0}^{|X_i-y|})=1$ when $z=X_i$ and 0 otherwise. So, for any $X_i$ and $z\le x$, $l(X_i,z,0)$ will equal 1 if $X_i$ equals any value $z\le x$ and zero otherwise, hence $\mathcal{L}(X_i,x,\lambda )=\mathbf{1}(X_i\le x)=\sum _{z\in \mathcal{D},z\le x}\mathbf{1}(X_i=z)$ when $\lambda =0$.

4 One can show that the cross-validation function contains two leading terms that are related to λ. The first term is a positive deterministic quantity which is minimised at $\hat{\lambda }=1$, and the second term is a zero-mean $O_p\left(1\right)$ random variable which is minimised at $\hat{\lambda }=1$ with some constant positive probability ${\delta }_1\in (0,1)$. Therefore, the cross-validation function is minimised at $\hat{\lambda }=1$ with a positive probability $\delta >{\delta }_1\in (0,1)$. In general, it is difficult to determine the exact value of δ because the exact value of δ depends on the unknown function $p(\cdot )$. Simulation in Ouyang et al. (2006) shows that $\hat{\lambda }$ takes the upper extreme value one roughly at $60\mathrm{\%}$ chance and takes values between zero and one at $40\mathrm{\%}$ chance. Ouyang et al. (2009) showed similar theoretical and simulation results for the nonparametric regression model with categorical regressors.

5 The Wang and van Ryzin kernel function is $1-\lambda$ when $X_i=x$ and $0.5(1-\lambda ){\lambda }^{|X_i-x|}$ when $X_i\ne x$ where $0\le \lambda <1$. Note, however, that this kernel, like all existing ordered kernel functions of which we are aware, lacks the ability to deliver discrete uniform probabilities for some value of the smoothing parameter. For the normalised version of this kernel, we divide $\hat{p}\left(x\right)$ by $\sum _{x\in \mathcal{D}}\hat{p}\left(x\right)$ to ensure that the probabilities indeed sum to one as per Glad, Hjort, and Ushakov (2003).

6 We are grateful to an anonymous referee who suggested incorporating these methods as comparators and who directed us to the R package Ake (Wansouwé et al. 2015) for their implementation. Note that the authors of this package only consider and implement least-squares cross-validation for smoothing parameter selection. Therefore, we beg the reader's forgiveness for including these comparators alongside those for least-squares cross-validation but not for likelihood cross-validation.

7 Typically, a P-value of 0 is simply recorded as, say, $<2.2\times {10}^{-16}$ when using the R statistical platform. However, a P-value of 1 would be unusual in a two-sided test setting (the two means would have to be identical) but is commonplace in a one-sided setting when the two means differ in the direction of the null, i.e. one is much larger than the other.

8 The median SSE (× 100) is 0.009 versus 0 for the normalised Wang and van Ryzin versus the proposed kernel, respectively, and the median SSE are identical for likelihood and least-squares cross-validation.

9 The negative binomial provided the best fit overall, but its tail is too thin, i.e. as the number of successful patent applications increases, the probability estimates approach zero too quickly yet there is still a substantial amount of empirical probability mass remaining in the tail.

10 Using a Taylor expansion, $f\left({\mathrm{\Delta }}_i\right)\equiv \log (1+{\mathrm{\Delta }}_i)={\mathrm{\Delta }}_i-\left(1/2(1+{\overline{\mathrm{\Delta }}}_i{)}^2\right){\mathrm{\Delta }}_i^2$, where ${\overline{\mathrm{\Delta }}}_i\in (0,{\mathrm{\Delta }}_i)$. Note that $1/(1+{\overline{\mathrm{\Delta }}}_i{)}^2=1-2{\overline{\mathrm{\Delta }}}_i+O\left({\overline{\mathrm{\Delta }}}_i^2\right)=1+o_p\left(1\right)$ because $|{\overline{\mathrm{\Delta }}}_i|\le \underset{1\le j\le n}{max}|{\overline{\mathrm{\Delta }}}_j|=o_p\left(1\right)$.