Reliable, challenging or misleading? A qualitative account of the most recent national surveys and country statistics in the DRC

Abstract

Abstract This paper presents a comprehensive assessment of the national information architecture in the Democratic Republic of Congo (DRC) between 1970 and 2010. In general, “the numbers” can be qualified as poorly reliable, though an important distinction should be made between aggregate country statistics and microlevel survey data. Whereas the latter inherently contain the purer and less manipulated pieces of information, the former have proven to be the result of an obscure blend of aggregation, estimation, permutation and negotiation, often with a weak informational basis. By contrast, survey data in the DRC are intrinsically of good quality and collected increasingly, although too many concerns remain about the poor accessibility of primary datasets, the fragmented metadata and the problematic sampling base to claim representativeness.

Résumé Cet article présente une évaluation complète de l'architecture du système d'information nationale de la République démocratique du Congo (RDC) entre les années 1970 et 2000. Dans l'ensemble, les données sont peu fiables, bien qu'une distinction importante doive être faite entre les données agrégées au niveau national et les microdonnées d'enquête. Les premières sont le résultat d'un mélange obscur d'agrégation, d'estimation, de permutation et de négociation sur une base informationnelle pauvre. En revanche, les données d'enquête de la RDC sont d'assez bonne qualité et sont collectées de plus en plus fréquemment, mais, pour attester de leur validité, elles souffrent encore de trop nombreuses limites sur le plan de l'accès aux données originales, de la disponibilité des métadonnées et des plans d’échantillonnage.

Keywords:

• development statistics
• national surveys
• Demographic Health Surveys
• sampling design
• Democratic Republic of Congo

Acknowledgements

The authors are grateful to the editors of this journal and two anonymous referees for their insightful and constructive comments. The authors also acknowledge the valuable input of Fidèle Bikangi, Dieudonné Vangu, Ronald van Dijk, Ivana Bjelic, William Prince, Milorad Kovacevic, Chris Rockmore, Guillaume Muhindo, Florence Marchal, Moise Tshimenga Tshibangu, Doris Wiesmann, Harold Vandermeulen and Stefaan Marysse. As usual, all errors remain the authors' sole responsibility.

Biographical notes

Wim Marivoet holds a PhD in applied economics from the University of Antwerp, Belgium. His research interests include welfare measurement, human development and coping strategies. Geographically, he has mainly published on the Democratic Republic of Congo (DRC).

Tom De Herdt is senior lecturer at the Institute of Development Policy and Management (IOB) at the University of Antwerp, Belgium. His research interests include capabilities, local governance and social norms. He has mainly published on poverty, governance and reconstruction in the DRC but has occasionally done work on Nicaragua, Rwanda and Cameroon.

Notes

1. This quotation comes from an international economic expert and forms the title of Gill's (1993) critique on the more conventional methods of (rural) poverty research.

2. Furthermore, to allow international comparisons, a conversion to international dollars is required, which has also proved problematic in case of the DRC (see De Herdt 2004, 25–26).

3. In line with the remarks above, the WDI (World Development Indicators) estimates for the years 2005–2010 downloaded in June 2012 indeed differ (slightly) from those downloaded in June 2013.

4. The precise status of this revision has been very confusing, and actually still is. For some time, the metadata of the WDI database certified GDP data of the DRC with an SNA1993 label. After some intense email traffic between Kinshasa, New York and Washington, a data specialist at the World Bank finally admitted that “the note in WDI is premature”, and it was later modified to reflect current practice (which is SNA1968). However, the World Economic Outlook database of the IMF (versions October 2012 and April 2013) still erroneously state that the DRC already made this transfer to SNA1993 (see metadata of the World Economic Outlook).

5. This was more or less openly admitted by the chief of state: “Tout se vend et tout s'achète dans ce pays. Et dans ce traffic, le moindre accès au pouvoir public constitute un veritable instrument d’échange, convertible dans l'acquisition illicite de monnaie ou d'autres biens, ou dans l’évasion de toutes sortes d'obligations”. Opening speech for the 11th extraordinary conference of the MPR, 25 November 1977.

6. An important exception to this might be Cour (1989).

7. This insight may at least also explain why official estimates increasingly converged over time, as noted in Figure 2.

8. Again, the metadata of the World Economic Outlook show sloppiness, as it states this last census occurred in 1983.

9. Besides, this death toll is also subject to serious debate, as estimates seem to vary between approximately 200,000 (Lambert and Lohlé-Tart 2008) and 4.6 million (Coghlan et al. 2008) casualties. A similar debate on numbers arose after the administrative census carried out in 1970, which further increased the need to organise a national census (see below).

10. This upward revision (based on the 1-2-3 Survey of 2004–2005) would amount to 57 per cent, and this same figure seems to apply to each year between 2006 and 2010 (République démocratique du Congo 2011b, annex 3). This latter aspect again underscores the fact that most series do not provide pure data observations but result from extrapolating a single observation to several successive years.

11. Overall, this paper draws on a research experience of more than seven years, including multiple short-term missions and one longer research stay with an internship at the National Institute of Statistics in Kinshasa.

12. For a more detailed historical overview of the country's demography between 1887 and 1992, see Akoto Mandjale and Iba Ngambong (1992, 13–32).

13. Whereas the second study was assigned to the Department of Demography within the Catholic University of Louvain, the first one was supervised by Joseph Houyoux of the Department of Sociology within the same university and largely financed by SICAI. Given their timing, their methodological comparability and their focus on household budget and assets, these city surveys offer a nice window to analyse the evolution in Congolese livelihoods over the period 1970–2010. In fact, Houyoux and his team repeated the same exercise in some cities in the eastern part of the country during the 1970s, as well as in Kinshasa in 1969 and 1986.

14. To give an idea, almost 1,000 people have been employed at the central headquarters in Kinshasa for an extended period of time, and not less than 45,000 individuals were directly involved in the field for several weeks (République du Zaïre 1982). On the financial side, it has been too complicated to determine the total cost of this census operation, but estimates for the upcoming census (scheduled for 2014) amount to 170 million dollars (Muhindo 2010). Both this estimate and the officially reported costs of many national surveys (as summarised in Table 1) seem high, even for Western standards, which may point to financial corruption. However, transportation of polltakers to the many remote corners of the country is very costly, and basic infrastructure (like electricity) and computer facilities at every cycle also often need updating.

15. For all national household surveys discussed in this paper, a stratified multistage sampling design typically applies. To stratify, each province (except for Kinshasa) is typically subdivided into three sectors, for which the following primary sampling units are used: (1) neighbourhoods in statutory cities; (2) other/smaller cities; and (3) collectivities of villages (“secteurs/chefferies”). For the latter two sectors, a second-stage sampling often procedes the final sampling of households, which may involve different (but often pragmatic) procedures.

16. Complementary to this sample and financed by USAID, another 1,683 families were surveyed to obtain data on immunisation coverage (République démocratique du Congo 2002, 7).

17. This platform is freely accessible after registration at www.childinfo.org. However, the raw data of MICS1 are not directly available on site, although they can be obtained upon request.

18. A more detailed investigation revealed that this discrepancy is probably due to an erroneous coding of some geographical variables for a limited number of observations. Unfortunately, without knowing the precise underlying data protocols, it was impossible to solve this issue.

19. However, a substantial (but isolated) problem exists for the majority of household assets surveyed in MICS2. More precisely, only a few assets were recorded in both the urban and rural sectors, despite their very large presence in both environments and their occurrence on both questionnaires.

20. One important exception in this respect is the National Human Development Report of 2008 (République démocratique du Congo 2008b), which made substantial use of the MICS1 and MICS2 datasets.

21. However, some modules within Phase 3 (like the one on material and financial patrimony [module 25] or on perceptions of living conditions [module 26]) do not seem to have been (correctly) conducted.

22. Given this methodological and thematic comparability, the DHS survey of 2007 can easily be considered as a MICS survey of the third generation, which then also explains the sudden shift from MICS2 to MICS4.

23. Moreover, even by relying on more general technical notes and those used for other sub-Saharan African countries, the reproduction of the wealth index was only possible for 79 per cent of all observations.

24. Unfortunately, this adjustment was not part of the previous MICS survey results either, which also complicates any straightforward comparison in this respect.

25. Similar to the selection bias introduced in many surveys' sample design, voter registration has likely been lower in many conflict-prone areas in the east of the country, which has been under the shifting control of rebels and insurgents of different kinds.

26. It involves a difference of less than 130,000 people, or around 0.2 per cent of the estimated population in 2004–2005.