A Deep Factor Model for Crop Yield Forecasting and Insurance Ratemaking

Abstract

Effective agricultural insurance and risk management programs rely on accurate crop yield forecasting. In this article, a novel deep factor model for crop yield forecasting and crop insurance ratemaking is proposed. This framework first utilizes a deep autoencoder to extract a latent factor, called the production index, that integrates salient spatial temporal patterns in the original yield data. Then, a concatenated deep learning model is constructed to enhance the modeling of the production index and the reconstruction of crop yields. Convolutional neural networks are employed to capture the high-dimensional and highly nonlinear structure within the crop yield data, as well as its interactions with weather and economic variables. The proposed deep factor framework is applied to the county-level data in the state of Iowa. Empirical results show that the newly proposed deep factor model significantly improves the prediction accuracy, especially in the test set. Based on a retain–cede crop insurance rating game between a private insurer and the government, we show that the proposed deep factor model provides economically and statistically significant improvement over the current Risk Management Agency ratemaking methodology.

Notes

1 Different statistical models have been proposed in the literature for crop yield modeling and agricultural risk management, including parametric distribution models (e.g., Gallagher 1986; Sherrick et al. 2004), nonparametric distribution models (e.g., Ker and Goodwin 2000), and mixture distribution models (e.g., Woodard and Sherrick 2011; Porth, Zhu, and Tan 2014). Recently, big data, machine learning, and artificial intelligence have been successfully applied in agricultural science and agronomy (see van Klompenburg, Kassahun, and Catal [2020], Nyéki and Neményi [2022] and Barriguinha, de Castro Neto, and Gill [2021] for reviews, and the references therein). For example, Piekutowska et al. (2021) used linear regression and nonlinear neural networks to predict potato cultivars before harvest. Ulfa et al. (2022) developed linear mixed models and incorporated remote sensing indexes to improve yield prediction. Nyéki et al. (2021) use different artificial intelligence models to predict maize yields with spatiotemporal data. Researchers have also discussed how to improve prediction by using high-quality big data. For example, Midtiby and Pastucha (2022) utilized images acquired by a unmanned aerial vehicle to improve pumpkin yield prediction. Khor et al. (2021) constructed a Fresh Fruit Bunch Index (FFBI) model with oil palm fresh fruit bunch yield data, which has a higher correlation with the Oceanic Niño Index and higher predictive ability. There is also a burgeoning literature on machine learning in actuarial science (see, e.g., Richman and Wüthrich 2021; Gomes, Jin, and Yang 2021; Perla et al. 2021; Richman 2021). However, applications to the area of agricultural risk management and insurance have been largely limited (Ghahari et al. 2019).

2 Convolution layers arrange inputs into feature maps and pass them through an activation function, typically nonlinear. A pooling function is utilized to downsample feature maps, which reduces the number of parameters and creates translation invariance to small shifts.

3 Backpropagation is a procedure to apply chain rule for calculating derivatives of an objective function. For details on backpropagation, see, for example, Hastie et al. (2005).

4 We also tested other activation functions, such as tanh or sigmoid functions, as well as other pooling functions, such as average pooling, and we found consistent results.

5 For comprehensive comparison, prediction performance results in the test sets from all CNN structures considered in Table 2 are provided in Online Appendix A.

6 To save space, details of the model selection results are provided in Table B.1 of Online Appendix B.

7 To save space, details of the model selection results are provided in Table B.2 of Online Appendix B.

8 For more information about SRA, see the 2023 Standard Reinsurance Agreement (USDA 2022).

9 For details of the RMA methodology, see, for example, Liu and Ker (2021).

10 For brevity, we consider a 90% coverage level, which is the most dominant coverage level and accounts for 90% of the liabilities in U.S. area yield policies. Other coverage levels have similar results.

Additional information

Funding

The author is grateful for the research funding support from the Nanyang Technological University Start-Up Grant (04INS000384C300), Singapore Ministry of Education Academic Research Fund Tier 1 (RG143/19), and the Society of Actuaries Education Institution Grant.