When the Data Source Writes the Conclusion: Evaluating Agricultural Policies

Abstract

Statistics describe realities, but they also shape them, since they are used to design or support policies. As such accurate statistics are important. Using the agricultural sector in Rwanda as a case study, we demonstrate that dubious statistics can spread quickly. According to data from the Food and Agricultural Organization (FAO), yields have increased by 60 per cent since the implementation of large scale agricultural reforms, while other datasets point towards more modest gains. Yet, estimates in line with those of the FAO dominate the official discourse. We suggest that the discrepancies between datasets may be explained by the difficulties of collecting accurate agricultural statistics combined with an incentive to overestimate yields to show that the reforms have worked.

Acknowledgements

We would like to thank two anonymous reviewers for their valuable suggestions on previous versions of this paper. All the data used in this paper is publicly available. The FAOSTAT data can be downloaded from their website: http://faostat.fao.org/. The data from the household surveys in Rwanda (EICV 2 and 3) are available from http://microdata.statistics.gov.rw/index.php/catalog. The Stata code used to analyse the data is available upon request from the authors.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. EICV: Enquête Intégrale sur les Conditions de Vie.

2. It is common practice to aggregate total food production by adding up the calorific values of all food crops. For instance, the well known ‘Daily per-capita energy supply’ indicator of FAO uses this approach (Smith, 1998). We then divided total aggregated production in its energy content by the energy content of beans, one of the main staple crops in Rwanda, to get a familiar expression of yields, that is, expressed in kg/ha. This normalisation facilitates interpretation of the results, but does not influence the findings. This approach is similar to the well-known conversion to cereal equivalents (Rask & Rask, 2014).A second, common approach to aggregate food production is to convert total production into monetary value. We did not opt for this approach because we did not have good price data. Moreover, this approach requires tricky assumptions about inflation and regional differences in price levels.

3. As a robustness check, we calculated the evolution of total food production using FAOSTAT statistics. Results showed that total food production has increased even more since 2007 (by 92%) than overall yields (66%). This increase was driven by a sharp increase in cassava and maize production. Detailed results are available upon request.

4. To evaluate which of these two factors (that is an increase in crop-specific yields or a shift towards crops with the largest increase in yields), contributed most to the total increase in yields, we calculated overall yields in 2013 keeping the share of land devoted to each crop constant at 2007 levels. This calculation revealed that yields still increased by 60 per cent from 1253 kg/ha to 2022 kg/ha. Hence, the increase in crop-specific yields is by far the most important factor explaining the increase of overall yields.

5. More detailed results are available upon request.

6. A second approach used by FAO to estimate yields is model-based, meaning that yields are predicted based on observable variables such as weather patterns, fertiliser use or population growth. It seems very unlikely that this was the case for Rwanda, given the large increase in yields since 2007.