Short and Long Recall Errors in Retrospective Household Surveys: Evidence from a Developing Country

ABSTRACT

While recall errors in retrospective data from household surveys may generate estimation biases, the nature and the relative magnitude of the errors are still largely unknown, especially in the context of developing countries. To bridge this gap in the existing studies, we conduct a resurvey of respondents of the Vietnam Household Living Standards Survey (VHLS) 2006. The combined data set allows us to investigate a variety of short-term and long-term errors associated with recall surveys. First, our empirical results suggest that when we ask total expenditure rather than categorised expenditures, long recall errors are no worse than short recall errors. Second, we found mean-reversion only for long recall errors in the sum of categorised expenditures but not necessarily for total expenditure. Finally, the inclusion of household size, asset, income, and geographical dummy variables in regression analyses may mitigate the biases arising from measurement errors.

Acknowledgements

We acknowledge the financial support from the Research Institute for Economy, Trade, and Industry (RIETI). We would like to thank Dang Kim Son, Nguyen Ngoc Que, Truong Thi Thu Trang, and Phung Duc Tung for their invaluable cooperation in the household survey. We would also like to thank Sarah Bales, Bob Baulch, John Gibson, Masahisa Fujita, Paul Glewwe, Takashi Kurosaki, and the participants at the RIETI International Workshop ‘Poverty and Vulnerability in Developing Countries’ held on 15 September 2009 for their useful suggestions and comments. The usual disclaimers apply.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. X in practice would also be recall data; thus, it may suffer from classical measurement errors, too.

2. In practice, survey questions that correspond to y^L and y^S may not be identical. Thus, phrasing issues could potentially introduce confounding factors.

3. Since the RIETI-CAP survey aims at collecting data to help design an insurance scheme that covers avian influenza and other natural disasters (flooding), sub-samples of VHLSS 2006 are chosen from four different provinces that were (1) only hit by avian influenza (Ha Tay province); (2) only hit by natural disasters (Nghe An Province); (3) hit by both avian influenza and natural disasters (Quang Nam Province); and (4) neither hit by avian influenza nor by natural disasters (Lao Cai Province). The selection of these four provinces was made using commune-level data in VHLSS 2004 (Nakata et al., 2010). Appendix Section A shows the locations of these provinces.

4. We ask annual per cent change ranges in total bought/bartered expenditures and in total self-generated or given consumption values between 2006 and 2007. Each range is represented by a particular number; medians for all ranges except for the two extreme ranges, that is, 60 per cent decrease for any decreases larger than 50 per cent, and 60 per cent increase for any increases larger than 50 per cent. Even if we change values for the two extreme ranges to 75 per cent changes, all the qualitative results remain unchanged.

5. Data collection period of VHLSS 2006 was in May, June, and September 2006 while RIETI-CAP survey was done from February until April 2008. Hence, there are some timing issues in data collection. However, both total expenditure and the sum of categorised expenditures are affected in the same way due to this timing issue. So, it does not necessarily favour one over the other.

6. We employ Robinson’s (1988) ‘double residual methodology’ to estimate a semiparametric regression model with nonparametric income or asset effect. Then we use the Hardle and Mammen’s (1993) test by which we compare the nonparametric and parametric regressions. We cannot reject the null hypothesis that the nonparametric regression and the linear regression give the same fitted equation at 1 per cent statistical significance level in all cases (total expenditure and the sum of categorised expenditures; bought/bartered goods and self-produced goods). Hence, we conclude that our linear regression results are valid. The results are available upon request.

7. With respect to self-generated goods, both the OLS and median regression show that households with a larger asset holding tend to report a higher gap (Nakata et al., 2010). According to the median regression results, households with lower incomes report a higher gap, rural households report a lower gap, and there is a reporting bias specific to certain occupations such as unskilled workers in the mining, construction, manufacturing, and transportation industries and other unskilled workers. With respect to the overall results, with F-tests, we strongly reject the null hypothesis of the jointly zero coefficients.

Additional information

Funding

This work was supported by the Research Institute of Economy, Trade and Industry (RIETI) [NA].