Do We Exploit all Information for Counterfactual Analysis? Benefits of Factor Models and Idiosyncratic Correction

Abstract

Optimal pricing, that is determining the price level that maximizes profit or revenue of a given product, is a vital task for the retail industry. To select such a quantity, one needs first to estimate the price elasticity from the product demand. Regression methods usually fail to recover such elasticities due to confounding effects and price endogeneity. Therefore, randomized experiments are typically required. However, elasticities can be highly heterogeneous depending on the location of stores, for example. As the randomization frequently occurs at the municipal level, standard difference-in-differences methods may also fail. Possible solutions are based on methodologies to measure the effects of treatments on a single (or just a few) treated unit(s) based on counterfactuals constructed from artificial controls. For example, for each city in the treatment group, a counterfactual may be constructed from the untreated locations. In this article, we apply a novel high-dimensional statistical method to measure the effects of price changes on daily sales from a major retailer in Brazil. The proposed methodology combines principal components (factors) and sparse regressions, resulting in a method called Factor-Adjusted Regularized Method for Treatment evaluation (FarmTreat). The data consist of daily sales and prices of five different products over more than 400 municipalities. The products considered belong to the sweet and candies category and experiments have been conducted over the years of 2016 and 2017. Our results confirm the hypothesis of a high degree of heterogeneity yielding very different pricing strategies over distinct municipalities. Supplementary materials for this article are available online.

Keywords:

• Counterfactual
• Demand estimation
• Factor models
• High-dimensional testing
• Optimal pricing

Acknowledgments

The authors thank an associate editor and three anonymous referees for very insightful comments.

Supplementary Materials

In the Supplementary Material, we report additional empirical findings and the proof of the main result. We also compare the proposed methodology to some alternatives available in the literature.

Notes

1 Due to a confidentiality agreement, we are not allowed to disclosure either the name of the products or the name of the retail chain.

2 We consider a scalar variable for each unit for the sake of simplicity, and the results in the paper can be easily extended to the multivariate case.

3 We could also have included lags of the variables and/or exogenous regressors into ${\mathit{Z}}_{-1t}$, but again, to keep the argument simple, we have considered only contemporaneous variables; see Carvalho, Masini, and Medeiros (2018) for more general specifications.

Additional information

Funding

Fan’s research was supported by NSF grants DMS-1712591, DMS-2052926, DMS-2053832 and ONR grant N00014-19-1-2120. Masini’s and Medeiros’ research was partially supported by CNPq and CAPES. We are also in debt with Thiago Milagres for helping us with the dataset and all the team from the D-LAB@PUC-Rio for providing a superb research environment.