Investigating Smooth Multiple Regression by the Method of Average Derivatives

Abstract

Let (x ₁, …, x_k, y) be a random vector where y denotes a response on the vector x of predictor variables. In this article we propose a technique [termed average derivative estimation (ADE)] for studying the mean response m(x) = E(y | x) through the estimation of the k vector of average derivatives δ = E(m′). The ADE procedure involves two stages: first estimate δ using an estimator , and then approximate m(x) by ), where ĝ is an estimator of the univariate regression of y on . We argue that the ADE procedure exhibits several attractive characteristics: data summarization through interpretable coefficients, graphical depiction of the possible nonlinearity between y and , and theoretical properties consistent with dimension reduction. We motivate the ADE procedure using examples of models that take the form . In this framework, δ is shown to be proportional to β and [mcirc](x) infers m(x) exactly. The focus of the procedure is on the estimator , which is based on a simple average of kernel smoothers and is shown to be a √N consistent and asymptotically normal estimator of δ. The estimator ĝ(·) is a standard kernel regression estimator and is shown to have the same properties as the kernel regression of y on x ^Tδ. In sum, the estimator δ converges to δ at the rate typically available in parametric estimation problems, and [mcirc](x) converges to E(y | x^Tδ) at the optimal one-dimensional nonparametric rate. We also give a consistent estimator of the asymptotic covariance matrix of δ, to facilitate inference. We discuss the conditions underlying these results, including how √N consistent estimation of δ requires undersmoothing relative to pointwise multivariate estimation. We also indicate the relationship between the ADE method and projection pursuit regression. For illustration, we apply the ADE method to data on automobile collisions.

Key Words:

• ADE regression
• GLIM models
• Kernel estimation
• Nonparametric estimation