A Dream Experiment in Development Economics

Abstract

In this article, the authors discuss a unique project carried out by 13 teams of four students each in the undergraduate Development Economics class during the 2012 spring semester at a private liberal arts college. The goal of the “Dream Experiment” was to think of an idea that promotes development, employs concepts from development economics, uses a real-world situation from a developing country, and has implications for policy if the experiment goes ahead. The authors present details of the project's modus operandi, provide a case study as an example, and highlight lessons for economics pedagogy.

Keywords:

• experiment
• policy
• teaching development economics

Keywords:

• JEL codes A13
• A22
• O12

The purpose of undergraduate economics, as many economists have argued, is to teach students how to think like economists (Becker 2000). Despite this general consensus, the typical undergraduate economics class consists of one-way lectures, coined “Chalk and Talk” style teaching, that rarely encourage students to actively utilize concepts in economics to think critically.

Economists in the field of teaching and learning of economics point to two main issues with the current teaching style: the lack of relevance to the students’ lives and students’ passivity. As Becker (2000, 2004) explained, economics pedagogy can lack application to a student's academic life as well as life after college. For example, in an introductory-level microeconomics class, assumptions, such as perfect competition or perfect information, can be so simplistic that students may find these rudimentary concepts meaningless. In large lectures, students hardly interact with the professor or actively participate in learning. Buckles and Hoyt (2006) advocated a more active role in understanding economics, wherein students read and write about economics, discuss economic ideas, and apply economic concepts. One branch of economics where active learning and relevance can be intertwined is development economics.

In this article, we discuss a unique project carried out by 13 teams of four students each in the undergraduate lower-level elective Development Economics class during the 2012 spring semester at a private liberal arts college in the United States. The goal of the experiment was to come up with an idea that promotes active learning through application of concepts from class and also uses a real-world situation from a developing country that has implications for policy if the experiment goes ahead.

The evidence for active learning as a more effective learning method is mixed and largely anecdotal; however, interactive techniques appear to increase student satisfaction with the class, to increase interest in economics in general, and to lead to more independent thinking (Zanca 2011). At the undergraduate level in the United States, development economics textbooks summarize the macroeconomic- and microeconomic-level research including growth models and explaining patterns using survey data from developing countries. They focus on theory as well as empirics but stress less on the paradigm of randomized experiments that have now become popular in development. Randomized experiments have become the gold standard for understanding causal effects on education and health of interventions such as conditional cash transfers to families in Mexico (Gertler and Boyce 2001), deworming in Kenya (Miguel and Kremer 2004), incentivizing teachers to come to school (Duflo, Hanna, and Ryan 2012), and performance pay to teachers in India based on students’ grades (Muralidharan and Sundararaman 2011). These experiments have led to considerations of how “nudges” in policies can make a difference, specifically, how small institutional changes to information and incentives in a local environment lead to changes in behavior and outcomes. However, these interventions are usually carried out at large research organizations such as The Abdul Latif Jameel Poverty Action Lab (J-PAL) or Innovations for Poverty Action (IPA), and students rarely get a hands-on flavor of such experimentation. To address this lacuna in development economics teaching, the goal of this project was to make students aware of the design and construction of a real-world experiment.

SETTING UP THE DREAM EXPERIMENT

The step-wise set up of the dream experiment is illustrated in table 1. During the first week of classes, we asked the 52 students who were registered for the development economics class to write down names of developing countries where they had any prior experience either through travel or an internship. Thirty-four students in the class had been to at least one developing country. Based on their self-reported experience, 13 regions were selected for the experiment, and four students were assigned to each region. Students were assigned such that at least two students in each team had some experience in the region, and remaining students were distributed randomly among the regions. Within each group, the range of students’ previous background in economics varied, from freshmen who had only taken introductory economics to senior economics majors.

TABLE 1 Timeline of the Dream Experiment

Week 2	The Dream Experiment begins in Week 2 after assignment of teams.
Weeks 2–4	Choose the theme (e.g., growth, education, health, corruption, gender, aid) and a narrower area within your assigned region.
	Brainstorm to put forth two ideas that are important to development in your region within the first two weeks.
Weeks 4–8	Obtain feedback from professor and narrow down focus to one idea.
	Try to get in touch with local NGOs or through Center for Community Engagement to make the project more feasible.
	Decide how and why the experiment will be undertaken (motivation, methodology, and timeline).
	Write related literature on the topic.
	Make a budget for the project. It should be less than $10,000 (exclude air fare and accommodation).
Week 9	Prepare first draft of the experiment to receive feedback from professor by end of Week 9.
Weeks 10–13	After receiving feedback on the first draft of your paper, work to polish and refine it.
Week 13	Present your Dream Experiment and submit paper by end of that week.

The final goal of the “Dream Experiment” was for students to address a market failure in their region. To accomplish this task, students created their own randomized controlled experiments that had the potential to be carried out in reality with the help of a Non-Governmental Organization (NGO). Project proposals for the experiments were to be between 3,000 and 5,000 words inclusive of footnotes, budget, and references. There were several skills that were evaluated in the dream experiment, each being worth 10 points: They included originality, review of related literature, feasibility, addressing concerns, policy implications, using concepts from class, regional knowledge, presentation clarity, answering questions during presentation, and overall quality of the paper. The project accounted for 20 percent of the overall course grade. A feasible experiment scoring the highest grade (after addressing any concerns) could then potentially be funded internally by a center for community engagement at the college up to a limit of $10,000. ¹

In the dream experiment, students combined their previous knowledge of their team's country with newly-learned development economics concepts. Following the introduction to neoclassical economics, the course addressed research related to a new theme in development (e.g., health, education, corruption, gender, psychology). Furthermore, the lectures presented several examples of randomized experiments within each theme—the reasons for carrying out an experiment as opposed to an observational study and how these were set up to answer a research question. ²

The structure of the project began with students choosing two theme-based questions relevant to their region. Students started creatively developing their own ideas for carrying out experiments within these umbrella topics, often combining different ideas like microfinance, health, and education into one proposal (for example, see the Indian case study). Utilizing knowledge of the team's country as well as drawing inspiration from experiments discussed in class, the groups then brainstormed possible modes of intervention. Afterward, the teams discussed their experiment ideas with the professor and honed down the possibilities to one unique and feasible experiment. To assure feasibility, the experiment had to include a number of components, such as a reasonable time frame for execution, a budget within the $10,000 limit, and an NGO that would provide the infrastructure for the experiment.

At this stage, the teams, through personal or college connections, began to contact local NGOs that would be interested in providing facilitative support for their experiments. This aspect of the project was important for the feasibility of an experiment. During weeks 4 through 8, students also mapped out the details of their experiment, clarifying the experiment's goals, structures, and economic mechanisms. By the end of week 9, students wrote a first draft of their project proposal. The proposal was to have the following sections: introduction, methodology, concerns, budget proposal, and bibliography.

CONTENT FOR THE EXPERIMENT PROPOSAL

Introduction

The introduction presented the motivation for the experiment and described the market failure that the experiment was designed to correct. Students were guided that it was better to cite statistics rather than making unsupported claims (e.g., “Literacy rates of women are half that of men” as opposed to, “We believe gender inequality in literacy is a huge problem”). In addition, the introduction included literature reviewing development economics relevant to the theme and how the group's proposed experiment would enhance previous experiments and incorporated concepts discussed in class. The introduction also explained how the group's proposed experiment would contribute to relevant previous experiments, including concepts discussed in class related to the group's experiment. The latter section of the introduction regarding class concepts was worth 10 points on its own. The concepts may be theoretical (e.g., poverty traps), or they may span more than one theme (e.g., health and education).

Methodology

The methodology section outlined the goals and structure of the experiment, as well as the role of partnering local institutions and the modes of quantifying the results. Students had to decide the unit of randomization and the necessity of having a large enough sample size for the experiment. The students were expected to address spillover effects, the outcome variables and their measurement, the hypothesis, and the underlying mechanisms that may be driving the hypothesized result. At a minimum, students developed material which addressed the following questions: What is the treatment? Where exactly would you carry out the study? Do you have a map of the area? When do you propose to conduct the project? Have you made any attempts to contact the local coordinators or stakeholders?

The teams were encouraged to contact NGOs before an idea was fully developed. This would be beneficial if organizations were slow to respond, but more importantly, the group could work with the NGO to create an experiment that worked well with the organization's structure, and there was more time to search for other NGOs if the first one did not work out. The disadvantage was that the NGO may not take the students seriously without a detailed proposal. Ideally, both practice and theory would go hand-in-hand, and students within a team would specialize in their comparative advantage, some focusing on the practical arrangement with the NGO while others created the conceptual framework for the experiment.

Concerns

This section delineated potential ethical or statistical concerns and how the group intended to address these issues. Statistical concerns included possibility of attrition, selection bias, and lack of explanatory power. Possible negative consequences on subjects were to be incorporated. Addressing these revealed the importance of securing necessary permissions. It taught the students about ensuring honesty and taking special care when designing experiments for vulnerable groups (e.g., young children). Ethical issues which arose included allowing infants of HIV-positive mothers to switch to donkey milk from breast milk in Lesotho, football sessions to deter youth crime in Jamaica, and sex education for migrant men in Mexico. Project teams were also encouraged to think about the external applicability of their experiment, in particular to other developing countries and even to the United States.

Budget

Finally, a detailed account of the budget for the experiment, including finding out local wages for enumerators, data-entry operators, as well as operational and monitoring costs was described. Students were also encouraged to think about the cost of the “treatment” and costs related to printing, designing, and translating of questionnaires.

FEEDBACK

Once the teams received comments on their first drafts, they incorporated the feedback into a final draft. Each team then presented their project not only to the class and professor, but also to a panel of experts, who ranged from economics professors to community engagement coordinators. Teams were required to communicate in a 10–12-minute presentation the importance of their proposed project and the general structure of their experiment.

In their final proposal, the Kenya team planned to decrease the dropout rate of immunizations through the issuance of regular message reminders on mobile phones. Team Caribbean wanted to provide tutoring and an extracurricular program with incentives to disadvantaged students in Kingston, Jamaica. Team Nicaragua proposed fighting malaria through locally grown cocoa, which has flavonol and has been found to be effective in earlier research. A couple of other projects included educating religious leaders in Nigeria about gender equality and targeting anemia in Nicaragua. We detail below the experiment constructed by Team India.

CASE STUDY: INDIA

The India team began the project by discussing their connections in South Asia, whether through summer internships or community service, fellowship affiliations, or personal contacts. After initial discussions, they decided to think more about the project before identifying an NGO that might be able to help them with the project. They discussed which topic on which they would like to direct their focus and decided to try and combine different areas, especially health, microcredit, and gender equality. The team searched online sources to identify experiments that combined some of these areas and used these studies and concepts from class (incentives, demand/supply side, technology, etc.) to try and come up with a unique experiment. The team talked about their various ideas in a group meeting and brought a narrowed list to the professor to discuss feasibility. After these discussions, they had a general idea for their project, but one that could have headed in a couple of directions. They shared their ideas with the NGO that they had decided would be most compatible with their project. For the first draft of the proposal, the team divided the work between the four members and then met numerous times to flesh out the proposal and formulate the first draft. The team presentation was based on a slightly modified version of this first draft.

The team subsequently submitted a final draft incorporating comments received by the professor and the audience as well as the NGO's responses to its queries. The group was unable to carry out their proposed experiment because the set-up of their project proved to be incompatible with the structure of the NGO that they had contacted. By the time the team members realized this issue, there was not enough time in the semester to modify their experiment or search for another NGO.

Introduction

The main objective of the India team's dream experiment was to analyze the effect that child nutritional information provided in conjunction with microcredit group meetings had on meeting attendance and child welfare within rural areas surrounding Udaipur, Rajasthan. The experiment was designed to look at a range of outcomes across medical, financial, and social constructs. The team predicted that hour-long sessions conducted by a nutritionist and occurring directly after the microcredit meeting would yield a positive impact on child health. These nutrition-focused sessions were predicted to act as an incentive to attend the microcredit meeting, which, as explained below, should increase social capital and lead to financial benefits. Furthermore, increased attendance within all-female groups could lead to improved female empowerment.

Having traditionally been a patriarchal society, India seriously underperforms in gender equality. Only 12 percent of the individual bank loan accounts belonged to women in 2006, and in the same year, for every 100 bank deposits in the name of men, there were only 35 bank deposits for women (Chavan 2008). Self-Help Groups (SHGs) are one of the ways this discrimination against women, especially in the financial sector, has been addressed. As grassroot-level financial intermediaries, SHGs consist of groups of generally low-income women who come together to save and to lend to each other. The financial benefits of microcredit groups are well-documented (Banerjee et al. 2009), while other more qualitative data from the Udaipur region also show a social empowerment increase among women in SHGs (Morris 2010). The India group's experiment aimed to study the effects of increasing female attendance to SHG meetings on women's empowerment and financial gains.

The India team's experiment was also designed to analyze the impact of educating mothers by providing information on child health to the mothers within the SHGs. This is especially relevant in India where child health is a major issue. Health surveys show that an Indian child is more than 20 times as likely to be malnourished on average as compared to an American child (IIPS and Macro International Inc. 2007).

The proposed study drew on several strands of literature within microcredit, gender, and health studies. The most relevant microcredit studies focus on the importance of social capital in the effectiveness of microcredit. Most directly related to this current study is that of Feigenberg, Field, and Pande (2010), who found that increasing the frequency of group meetings from monthly to weekly lowers defaults, improves informal risk-sharing capabilities, and increases economic cooperation among group members. The authors argued that it is not just the pre-existing social capital among group members but also social capital created through group meetings that leads to the success of a microcredit group. The India team's dream experiment was designed to increase the attendance of women to microcredit group meetings, which in turn promotes greater social capital among these women. Increased social capital should lead to greater success for the microcredit groups both financially and socially as well.

Another aspect of the India dream experiment involved the issue of malnourished children and how supplying mothers with nutritional information affects child nutrition. Webb and Block (2004) examined how Indonesian mothers’ formal education and specific knowledge of nutrition affect child nutrition indicators, and found that mothers’ education level benefits long-term child nutritional status while nutritional knowledge influences short-term nutritional status. The India team intended to add to this literature by analyzing the effect of regular nutrition informational meetings on child malnutrition when bundled with financial services.

The last body of literature relevant for this study is the bundling of financial and nonfinancial resources. Smith (2002) showed that microcredit institutions in Honduras that bundle credit with health services reduce diarrhea and increase income. Several studies compare the effect of health information sessions between microcredit and non-microcredit members, although self-selection biases are not well-addressed (Leatherman et al. 2012). Little research has been done to analyze the effect of health information to act as an incentive for microcredit attendance, a key part of this proposal.

Methodology

In order to conduct the experiment, the team planned to use three randomized groups. Group 1 was the control group, in which no nutritional information would be given. Group 2 included individuals which obtained nutritional information via cooking classes after the meetings. Following the SHG meetings, a government nutritionist would meet with the parents to give health information and offer cooking lessons to aid in making the children healthier. At each meeting, the nutritionist would teach the mothers how to cook one or two inexpensive, healthy recipes. The nutritionist would be used as both an informational tool and a dynamic incentive to raise the opportunity cost of missing a meeting. Group 3 participants received nutritional information via recipe booklets distributed after the meetings in addition to health information provided by the nutritionist. This experiment was scheduled to begin August 2012 and to last about five months, based on the 16-week cycles used by Smith (2002) in which each cycle was a different health promotion, to obtain a larger amount of information about the effects of the treatments. The India team's study would last the equivalent of one of these cycles.

Each SHG consisted of about 15 people, and each test group was to comprise 40 SHGs. Test groups would be randomly selected from pre-existing SHGs set up by a large NGO operating in Rajasthan, India. Because of the information and health aspects of the testing, the experiment was expected to have a large risk of spillover effects. However, these were to be mitigated by choosing SHGs from separate villages so that communication and interaction among the three groups would be reduced.

As explained above, the team's hypothesis was that offering nutritional information would improve the health of children, the financial position of participating women, and these women's level of social empowerment. They predicted that the hour-long meeting with the nutritionist after the meeting would yield a positive impact on child health.

The India team project design would enable them to measure impacts using child health tests and analyze women's financial situations through surveys. Attendance at meetings would be measured by simple sign-in sheets. Child health would be analyzed using the weight-for-age and height-for-age of the child, which could be recorded at day care centers. Financial outcomes would be measured by changes in assets, including total income, loan repayment rate, and savings. Questionnaires administered before and after would allow the team to measure forms of female empowerment, such as household bargaining power, community participation and leadership, public mobility, and development of self-confidence.

Concerns

One methodological concern with this experiment was selective attrition: Those in control groups may be more likely to stop attending meetings and drop out of the group, changing the group size and hampering its effectiveness. Another concern was the lack of uniformity of treatment across different villages as nutritionists may vary in their experience, local knowledge, and motivation. Finally, the six-month time frame may not be enough time to observe an increase in social capital.

Budget

The budget was as follows:

1.	Survey/questionnaire cost: $2,000
2.	Government nutritionists (100*6months*2 hours/month): $60–70/month, $400
3.	Nutrition booklets (100*$0.20 each): $20
4.	Researcher fees: a. Field manager (1*6months*40 hours/month): $80/month, $480 b. Survey assistants (2*6months*40 hours/month): $100/month, $600
5.	Total: $3,500

LESSONS IN ECONOMICS

While students gained an insight into a specific market failure within their assigned region, there also were general gains in economic knowledge. Students were required to think about trade-offs: Statistical power would be impacted by trading off the number of units within a treatment by the number of observations within a unit. Similarly, budgetary constraints meant that the students had to trim an expensive treatment or the number of units in the experiment and yet maintain enough ex-ante power to reject a false null hypothesis. There were also choices between explanatory power and internal validity due to possibility of spillovers into the control group.

Second, students applied marginal analyses when comparing across treatments. For example, in the Ecuador dream experiment to address oral hygiene, there were three treatment groups: dental information to all students, dental information and a toothbrush, and dental information, a toothbrush, and toothpaste if the students had an attendance rate greater than 80 percent.

To address concerns in their proposals, the students had to think about the unintended consequences of their treatments. For instance, the Poland team wanted to pay students to take photos of their peers drinking alcohol. This was supposed to reduce alcoholism among teenagers in Poland. However, the team realized that this would result in collusion and then amended their proposal to answer a new question that tested Gary Becker's (1968) thesis: Does increasing the probability of punishment work better than increasing the extent of the punishment for drinking alcohol (conditional on collusion)?

Understanding effects was also encouraged. Treatments may have heterogeneous effects along a dimension, and this could help uncover causal channels post the experiment.

Finally, incentives may be more effective if the experiment design took account of behavioral biases: The Kenyan team's proposal provided for customized message reminders about time and place of immunizations to reduce dropout rates due to a lack of information or procrastination.

CONCLUSION

The dream experiment was well-received by the class, and it not only motivated students to learn more about the randomized experiments in development, but also gave students the chance to apply the concepts in a real-world scenario. The possibility of receiving funding from the center for carrying out the experiment as well as the contribution toward the final grade incentivized the students. Another advantage of the project was to allow students to learn from each other. This was possible because of the diversity of the class in terms of their own experiences in developing countries and even within-group diversity. There was competition to win the “tournament,” and this made students particularly attentive to limitations in their projects and also to that of others.

Group formation in the experiment had its own trade-offs. Allocation of students in teams could be based on other approaches as opposed to a region with prior experience. Alternative options are making the selection totally random or allowing the students to self-select their own groups. These methods of splitting the groups may be necessary if people do not have any prior experience of developing countries. If students are able to decide who is in their group, this may lead to a type-clustering (such as a group of all seniors), which may help reduce the free-rider problem. It will, however, come at the cost of equal opportunity as having a group of all seniors or all freshmen may create a large disparity in the quality or level of economic knowledge across groups. It also may hamper learning within groups which can occur if groups are more heterogeneous. One may want to take these factors into account when deciding how to allocate students to teams. The choice also will have dynamic effects in terms of selection of students who take the class in the future. For instance, prospective students with the intrinsic motivation for doing developmental work would be more likely to take the class when it includes the dream experiment. Basing groups on a combination of prior experience and randomization of inexperienced students seemed to promote intra-group learning and to give equal opportunity to those who had no experience of visiting a developing country.

Although this project aptly combines real-world relevance and a discussion-based model of modern development economics, there are some shortcomings that must be addressed in future projects. First, due to formation of teams, there is a free-rider problem both during and after the project development. For example, one team member complained that no one else was working hard on designing the experiment. One way to limit this problem is to explicitly delineate each team member's role and responsibilities in developing the first draft. Another solution could be for all students in a team to write down what they think is their contribution relative to their team members. Depending on evaluations by team members, there could be a re-adjustment of grades. This will provide an incentive to increase effort but can also lead to collusion among members who are shirking. An alternative approach to reducing the free-rider problem is giving an option to team members to vote a nonperforming member off the group, who then must write another proposal individually.

Also, as project funding did not allow for all the winning team's members to go to the field, there was hesitation in getting one's hands dirty in fieldwork. ³ There were bureaucratic difficulties such as delays in NGOs’ responses or government approvals, but these also made the team members appreciate the ground work that went into carrying out an experiment.

There are a few other points to consider for replicating this project in other colleges. First of all, it may not be possible to fund large-scale randomized experiments for undergraduate students. However, there are numerous low-hanging fruit that can be obtained even within a small budget. Giving voice to the students, equipping them with the necessary tools, and allowing them to participate in constructing their own field experiment seems to be an exciting opportunity to combine theory and practice in development economics.

Acknowledgments

The authors are grateful to Dan Barbezat, Sarah Barr, Amrita Basu, Rick Griffiths, Hannah Holleman, Adam Honig, Chris Kingston, Dunstan McNutt, Molly Mead, Jessica Reyes, Dwaipayan Sen, Kate Sims, and Geof Woglom for their encouragement and advice during the Dream Experiment. They also thank two anonymous reviewers and the seminar participants at the 2012 Annual Economics Teaching Conference in Orlando for their constructive suggestions. Finally, they thank the students in the Development Economics class who were always full of ideas, in particular, David Beron, Andrew Edelman, Aashish Karki, Darius Onul, and Crystal Yan.

Notes

1. In other colleges, the professor could assist in finding funding for such projects internally or externally through research grants from appropriate organizations.

2. For example, while discussing corruption, students were told about the study by Olken (2007) where he dug up samples of Indonesian roads to find out if there had been corruption in building roads. He compared a top-down approach to reducing corruption through increased official monitoring of funds with a bottom-up approach where villagers were encouraged to keep watch at the grass-root level. Half the villages were randomly assigned the top-down monitoring treatment and the remaining ones were assigned the bottom-up treatment.

3. Other than the budget constraint, there were also time constraints, with many students having committed to internships during the summer.