How to do (and how not to do) fieldwork on Fair Trade and rural poverty

Abstract

Abstract The Fair Trade, Employment and Poverty Reduction (FTEPR) project investigated poverty dynamics in rural Ethiopia and Uganda. When designing fieldwork to capture poor people often missing from standard surveys, several methodological challenges were identified and, in response, four decisions were made. First, FTEPR focused on wage workers rather than farmers and improved on standard questionnaires when collecting labour market information. Second, researchers adopted contrastive venue-based sampling. Third, sampling was based on clearly identifiable “residential units” rather than unreliable official registers of “households”. Fourth, an economic definition of “household” was used rather than the more common definition based on residential criteria.

Résumé Le projet Fair Trade, Employment and Poverty Reduction (FTEPR), qui portait sur la dynamique de la pauvreté dans les régions rurales d’Éthiopie et d'Uganda, a dû relever plusieurs défis pour rejoindre les personnes pauvres échappant aux enquêtes standardisées. Quatre décisions ont été prises à cet égard. Premièrement, le projet a mis l'accent sur les travailleurs salariés plutôt que sur les agriculteurs et il a amélioré les questionnaires habituellement utilisés pour récolter de l'information sur les marchés du travail. Deuxièmement, il a adopté un plan d’échantillonnage raisonné des lieux d'enquête. Troisièmement, l’échantillonnage s'est basé sur des unités résidentielles facilement identifiables plutôt que sur les registres officiels des ménages qui sont peu fiables. Enfin, les ménages ont été définis en termes économiques plutôt qu'en fonction du lieu d'habitation.

Keywords:

• methodology
• poverty
• agriculture
• labour markets
• Africa

Biographical notes

Christopher Cramer is Professor of the Political Economy of Development at SOAS. He has worked in and on sub-Saharan Africa for more than 25 years, teaching and conducting research, on rural labour markets, commodity processing and violent conflict.

Deborah Johnston is a Reader in Development Economics at SOAS, London. She has worked on sub-Saharan Africa for over 20 years, researching rural labour markets, poverty, welfare and land. She is co-editor of the Journal of Agrarian Change.

Bernd Mueller has been research officer in the FTEPR project, for which he spent extended periods in Ethiopia and Uganda on fieldwork missions. His main research interests include rural labour markets and the political economy of rural development. After his departure from FTEPR, he joined the Decent Rural Employment Team at FAO in Rome, as rural employment specialist.

Carlos Oya is Senior Lecturer in Political Economy of Development in the Development Studies Department, SOAS, London. He has done primary research in Mozambique, Senegal, Mauritania, Uganda and Ethiopia, focusing on the political economy of agrarian change, capitalist accumulation, rural wage labour and poverty. He is also co-editor of the Journal of Agrarian Change.

John Sender is Professor Emeritus of Economics at SOAS, London. He has designed surveys and conducted fieldwork in many rural areas of Africa and Asia over a period of almost 40 years, usually focusing on the development of labour markets.

Notes

1. Fair Trade, Employment and Poverty Reduction in Ethiopia and Uganda was a four-year research project (2009–2013) funded by the UK's Department for International Development (DFID); see http://www.ftepr.org/

2. Venue-based sampling is a method developed by the US Centers for Disease Control and Prevention (Muhib et al. 2001; Vermund et al. 2010), among others.

3. See http://www.ftepr.org for information on publications.

4. The World Development Report 2008 emphasised, for example, that “stunningly little policy attention has been given to the structure, conduct and performance of rural labour markets and how they ease successful transitions out of agriculture” (World Bank 2007, 221).

5. On the poor coverage of rural wage employment in commonly cited Ethiopian surveys, see specifically Rizzo (2011). The more general point has been made that “in much of the development literature on pro-poor growth nowadays, little or no attention is paid to the underlying mechanisms that determine the dynamics of income … specifically, the dynamics of employment growth and of how and to what extent productivity growth translates into the growth in labour earnings is left out of the equation” (Wuyts 2011, 10). Similarly, Amsden (2010, 57) points out: “Despite championing the cause of poor people around the world, and dramatising the human condition, the United Nations’ Millennium Development Goals make not the slightest mention of employment generation as a means to battle poverty.”

6. Important exceptions to this neglect of wage workers include research by Valkila and Nygren (2009), Luetchford (2008), Maertens and Swinnen (2012) and Maertens, Colen and Swinnen (2011).

7. FTEPR fieldwork highlighted precisely that employers, who have close ties to local officials, are keen to avoid situations where their workers have the freedom to engage independently and privately with researchers. Local security officials and the police in one fieldwork site detained research assistants for several hours and lectured the senior researchers on “proper” research methods, which included asking “the owner” of a large agro-export (multinational) business to select workers and then interviewing only the selected workers at the workplace.

8. A more recent attempt to assess the impact of Fair Trade on poverty reduction also depended upon the Fair Trade organisations funding the research to select the research sites and the producers organisations to be studied (Klier and Possinger 2012, 4).

9. In Ethiopia, the kebele is the smallest administrative unit. It is broadly comparable to a ward.

10. A similar claim was made concerning the choice of the 36 villages surveyed in an influential study of poverty in rural Uganda: “The selected villages represent quite well the considerable diversity that exists within the two selected regions.” However, the researchers make it clear that the actual choice of villages was heavily influenced by the wishes of district-level bureaucrats; the measures, or the relevance to issues of poverty, of the indicators of “diversity” are not discussed (Krishna et al. 2006). Another study in four districts of rural Uganda (of coffee producers) sampled only those producers appearing in the Uganda National Household Survey (UNHS). Unfortunately, the UNHS was not designed to be representative of coffee producers (or of households in each district), so that the sample cannot be considered representative of robusta producers in the districts concerned, let alone of coffee producers in Uganda as a whole. This fundamental problem did not prevent the World Bank-funded researcher from drawing conclusions about “the Ugandan coffee market” and “the majority of coffee grown in Uganda” from unrepresentative data (Hill 2010, 455, 438). The fact that the UNHS specifically excluded larger scale coffee farmers in Uganda from the survey is another important reason for caution in extrapolating its results to the coffee market as a whole (Ssekiboobo 2008, 7).

11. The broader political relevance of the sample is also questionable. The ERHS is not representative of the ethnic and religious composition of the rural Ethiopian population; for example, Oromos are underrepresented (Kumar and Quisumbing 2012, 2). Further criticisms of World Bank-funded household surveys in other developing economies, focusing on their failure to provide a robust, transparent record of poverty incidence, has been provided by Walters, Marshall and Nixson (2012).

12. For site selection purposes, large-scale farms were defined as enterprises employing at least 75 wage workers; small-scale farms were defined as enterprises that are members of Fair Trade certified smallholder cooperatives, or as enterprises employing fewer than 75 workers. The Holeta site in Ethiopia was selected not only because it included small-scale flower farms, but also because it was the only area of floricultural production known to have experienced labour disputes and trade union interventions. After site selection these definitions were adjusted in line with sector and geographic norms.

13. A more detailed discussion of the selection of the 12 research sites (including maps), showing how contrastive exploration has been operationalised, can be found in Methodological Issues, FTEPR Discussion Paper No.1, “How to do (and how not to do) fieldwork on fair trade and rural poverty” on the project website www.ftepr.org.

14. Ethnographic work in two villages in northeast Ethiopia describes how local officials administering the Productive Safety Net Programme constructed lists of households so as to reserve the benefits of the programme for “the more affluent and economically potent households”, excluding “the poorest and chronically food-insecure households”, many of which depended on casual agricultural wage labour (Bishop and Hilhorst 2010).

15. For example, fieldwork in Kabale District in Uganda compared the official list of households maintained by one LC1 Chairman with a careful FTEPR village census (the LC1 is the lowest level of local council, usually a village or neighbourhood). The chairman's list was found to be grossly inaccurate. There is also evidence, insufficiently discussed in the relevant survey documentation, that the lists of households at the kebele level in Ethiopia, which are regularly used as rural sampling frames, are also unreliable. For example, a choice has to be made between alternative lists of households held by the kebele chairman, local health extension workers or development agents; one or more of these lists may well have been amended by the survey team (IFPRI and EEPRI n.d.). It has been admitted that not all villages sampled in the Ethiopian Rural Household Surveys had good lists of registered households (Dercon and Hoddinott 2009, 7). A quantitative survey in the northeastern highlands of Ethiopia, backed up by careful qualitative work, indicated that official kebele lists usually excluded households that did not pay tax, as well as some single-person households and people belonging to “socially marginalised groups” (Sharp, Devereux, and Amare 2003, 36).

16. Epidemiologists have pioneered the use of these technologies for surveys in rural Africa; see for example Vanden Eng et al. (2007). FTEPR benefited from advice and training in the use of handheld computers (or personal digital assistants, PDAs) with GPS provided by Anja Terlouw and James Smedley of the Liverpool School of Tropical Medicine.

17. The census of RUs in rural Ethiopia and Uganda was much less problematic than similar exercises listing unregistered urban populations in China with the aid of PDAs (Treiman et al. 2005, 13). Depending on settlement density and topography, the FTEPR research teams might enumerate between 80 and 150 RUs a day in rural settings, while in the more urban settlements of Ziway the number could rise to more than 300 a day.

18. The software used for GPS navigation and the collection of GPS census data was CDC­GPS2, developed by a team of researchers at the Centre for Disease Control (freely available at http://ftp.cdc.gov/pub/gpscs/). The digital questionnaires were designed and programmed using Syware Visual CE (http://www.syware.com/products/visual_ce.php).

19. In some smallholder coffee research subsites in Uganda, all of the RUs identified in the census, as opposed to a random selection of RUs, were revisited to obtain the expanded sampling frame of individuals. This strategy was adopted in research sites where export crop wage workers lived in scattered RUs interspersed with many other RUs containing no such workers. The aim was to ensure that the population list of individuals contained a sufficient number of the scattered wage workers so that a random sample drawn from the list of individuals was likely to capture respondents with and without relevant labour market experience.

20. The list of possible classifications of respondents varied across research sites. The electronic questionnaire included additional questions for some research sites, reflecting the type of variation that FTEPR hoped to achieve in the context of different crops and production conditions.

21. The site selection and sampling methods and GPS technology allow for a follow-up survey of a subsample of the original respondents. FTEPR research also involves qualitative research methods, including life histories of a small sample of those included in the initial survey. The advantages of “nesting” life histories within larger quantitative surveys are described in Schatz (2012) and in Sender, Oya and Cramer (2006).

22. At the other end of the scale, the largest and richest farmers in a rural area may also be excluded from lists of households or farm households because their farms are not defined as being operated by “households” (Choudhry 2008, 11) or simply because surveys of households usually exclude the top end of the wealth/income distribution (Székely and Hilgert 1999; Deaton 2001; Banerjee and Piketty 2003, 4). The domestic and farm servants living with and working for the rural rich are, therefore, also missing from rural household surveys.

23. Some implications of the failure to collect information on young, mobile rural people who are defined as ‘non-residents’ in conventional household surveys have been quantified using data from Burkina Faso. Their exclusion has a major influence on assessments of rural living standards (Akresh and Edmonds 2010). In Vietnam, assessments of rural and urban living standards have been shown to be unreliable for the same reasons (Pfau and Giang 2008).

24. The definition was designed to include the following four categories of linked individuals: (1) those who live permanently with the principal respondent and who share income and expenditure; (2) those who, even if not sharing residence on a regular basis, make significant economic contributions (in cash or in kind) to the expenses of the household/respondent; (3) those who, even if not sharing residence, regularly depend on economic contributions in cash or in kind from the respondent or others in the RU; and (4) those who, even if not resident at all in the same place as the respondent, either can be relied upon by the respondent, or receive contributions from the respondent.

25. Evidence from different disciplines “shows that the household as defined by survey statisticians may bear little resemblance to the social unit in which people live” (Randall, Coast, and Leone 2011, 217).

26. The innovative FTEPR questionnaire is available at www.ftepr.org.