Interactive Tree-Structured Regression via Principal Hessian Directions

Abstract

An interactive approach to tree-structured regression is introduced. Unlike other procedures driven by cost optimization, this approach focuses on the exploration of geometric information in the data. The procedure begins with finding a direction along which the regression surface bends the most. This direction is used for splitting the data into two regions. Within each region, another direction is found, and the partition is made in the same manner. The process continues recursively until the entire regressor domain is decomposed into regions wherein the surface no longer bends significantly and linear regression fit becomes appropriate. For implementing the direction search, the method of principal Hessian directions is applied. Several simulation and empirical results are reported. Comparison with three methods—CART, SUPPORT, and MARS—is made. The benefit of using geometric information is highlighted.

Key Words:

• Dimension reduction
• Principal Hessian directions
• Recursive partitioning
• Sliced inverse regression
• Tree-structured regression
• Visualization