The Role of Mixed Methods in Improved Cookstove Research

Abstract

The challenge of promoting access to clean and efficient household energy for cooking and heating is a critical issue facing low- and middle-income countries today. Along with clean fuels, improved cookstoves (ICSs) continue to play an important part in efforts to reduce the 4 million annual premature deaths attributed to household air pollution. Although a range of ICSs are available, there is little empirical evidence on appropriate behavior change approaches to inform adoption and sustained used at scale. Specifically, evaluations using either quantitative or qualitative methods provide an incomplete picture of the challenges in facilitating ICS adoption. This article examines how studies that use the strengths of both these approaches can offer important insights into behavior change in relation to ICS uptake and scale-up. Epistemological approaches, study design frameworks, methods of data collection, analytical approaches, and issues of validity and reliability in the context of mixed methods ICS research are examined, and the article presents an example study design from an evaluation study in Kenya incorporating a nested approach and a convergent case oriented design. The authors discuss the benefits and methodological challenges of mixed-methods approaches in the context of researching behavior change and ICS use recognizing that such methods represent relatively uncharted territory. The authors propose that more published examples are needed to provide frameworks for other researchers seeking to apply mixed methods in this context and suggest a comprehensive research agenda is required that incorporates integrated mixed-methods approaches, to provide best evidence for future scale-up.

More than a third of the world's population relies on biomass fuels for cooking and home heating (Bonjour et al., 2013; Rehfuess, 2006). In general, these fuels are burned using traditional stoves or three-stone fires often indoors, resulting in very high levels of household air pollution (HAP) considerably higher than World Health Organization (WHO)–recommended levels for particulate matter <2.5 micrometers (PM2.5; WHO, 2006).

Cooking with biomass fuels on traditional stoves has important consequences for health, the environment, and socioeconomic development. HAP from burning solid fuels is a major risk factor for a range of health issues including childhood pneumonia (Dherani et al., 2008) lung cancer (Hosgood et al., 2011), and adverse pregnancy outcomes (Pope et al., 2010). In some settings, biomass is used non-renewably, and together with emissions of short-lived climate pollutants caused by incomplete combustion, these practices also contribute to climate warming (World Bank, 2011).

The Global Burden of Disease 2010 study (Lim et al., 2013) found that exposure to HAP was associated with around 3.5 million annual deaths and 4.3% of disability-adjusted life years in 2010 worldwide, with the most recent estimate from WHO putting this toll at more than 4 million deaths for 2012 (WHO, 2014). In addition, household cook-fuel combustion contributed to approximately 0.4 million ambient (outdoor) air pollution deaths. Women and children are particularly vulnerable to the associated health risks (Smith et al., 2014), since they spend the longest periods of time close to the stove. Additional aspects that contribute to poor health and social circumstances include risks associated with collecting fuel or, where fuel is not freely available, cost of fuel. Furthermore, time women and children spend collecting fuel can also impact negatively on access to education and development (Karlsson, 2007). Lack of access to more efficient, clean-burning cooking technology contributes to the process of trapping poor households and communities in a cycle of ill-health and poverty (Rehfuess, 2006). Thus, the challenge of promoting access to clean and efficient household energy for cooking and heating is a critical issue facing low- and middle-income countries today.

Improved cookstoves (ICSs) typically require less fuel and can lead to a reduction in HAP and a potential reduction in health risks (Lewis & Pattanayak, 2012; Ruiz-Mercado, Masera, Zamora, & Smith, 2011; Smith et al., 2011). Although a number of ICSs have been shown to markedly reduce emissions in laboratory testing (Albalak, Bruce, McCracken, Smith, & De Gallardo, 2001; Jetter & Kariher, 2009; MacCarty, Still, & Ogle, 2010) evidence from in-home evaluations suggests that few ICSs come close to achieving HAP levels set by WHO in the field.

Dissemination of ICSs and sustained use at scale is complex; stove users are an extremely important part of this process as exclusive use of cleaner household energy technologies and fuels for cooking and heating is a crucial aspect of successful scale-up. One approach to promoting adoption and sustained use of ICS popular in the field is the behavior change approach (Aboud & Singla, 2012; Burwen, 2011; Goodwin et al., 2014).

ICSs and Behavior Change

Understanding whether and how women alter their manner of preparing food, commonly termed behavior change, is critical to realizing the advantages of new technologies (Burwen, 2011). Behaviors required to take advantage of new cooking technologies are influenced by many different personal, financial, social, cultural, and structural factors and therefore tend to be difficult to change (Aboud & Singla, 2012). The evolution of behavior change has seen a move toward multilevel models that focus on individual, social, and environmental factors (Goodwin et al., 2014). However, measuring the motivations and determinants of behavior change is not straightforward. Goodwin and colleagues’ (2014) review of behavior change techniques in relation to clean cooking interventions concluded that there was a lack of consistent and credible evidence in the field and that further work was needed to define and measure the effectiveness of behavior change techniques. Mixed methods can offer an important tool in this respect. This article outlines the role of mixed methods in measuring user behavior change in relation to ICS adoption and sustained use.

Using Mixed Methods for Investigating Behavior Change and ICS Adoption

To date, the literature on HAP and ICS has demonstrated substantially greater PM2.5 emission reductions achieved in a laboratory setting than in the field (Jetter et al., 2012; MacCarty et al., 2010), largely because, in addition to the model of stove used, cooking behavior has a significant influence on levels of HAP. Understanding user cooking behavior and how this relates to HAP is therefore crucial, if ICS users are to achieve the large reductions in emissions required to meet WHO air quality guideline levels. However, measuring HAP and user behavior in the field is complex. Traditional epidemiological approaches to ICS evaluation tell us little about user perspectives. Conversely, most qualitative studies do not measure HAP alongside user views, so little is known about how HAP reduction affects user perspectives or how user views affect ICS use. Given that quantitative approaches have an important role in measuring effectiveness of ICS in reducing HAP and that qualitative research is particularly valuable in exploring stakeholder perspectives (Stanistreet, Puzzolo, Bruce, Pope, & Rehfuess, 2014), designing studies that use the strength of both these approaches, will likely offer important insights into behavior change in relation to ICS uptake and scale up.

Despite the potential of combining quantitative and qualitative methods in studies evaluating behavior change and ICS, the application of this approach, has to date, been limited. In their systematic review of barriers and facilitators to scale up of ICS, Rehfuess, Puzzolo, Stanistreet, Pope, and Bruce (2014) identified only three mixed methods research studies (Jagoe, Bromley, Chengappa, & Bruce, 2006; Jagoe, Bromley, Dutta, & Bruce, 2007; Pandey, 1989). Furthermore, where the use of mixed methods is used, qualitative and quantitative findings are commonly presented separately. This reflects a trend in the mixed methods literature that relates to the degree of integration of quantitative and qualitative findings, particularly with trial designs. For example, Lewin, Glenton, and Oxman (2009) examined how qualitative methods are used alongside randomized controlled trials of complex health care interventions and found that 30% of trials sampled included a qualitative component but there was almost no integration at the level of analysis or interpretation. As O'Cathain, Murphy, and Nicholl (2010) pointed out:

Without integration, the knowledge yield is equivalent to that from a qualitative study and a quantitative study undertaken independently, rather than achieving a whole greater than the sum of the parts.

There are likely to be a number of reasons for the lack of integrative analysis, some conceptual and some practical. Traditionally, qualitative and quantitative research methods have developed from opposing paradigms and some researchers believe that the integration of such opposing epistemological stances is not possible (Denzin, 2010). Furthermore, it can be argued, that the status of the trial design has given it precedence over qualitative approaches, which, in turn, mitigates against integrating findings in a balanced way. Lastly, mixed-methods research requires a team of researchers from different disciplines to collaborate closely on projects. In practice, however, teams often work separately and thus the epistemological divide persists. To develop methods that integrate approaches in the ICS field, researchers first need to overcome these conceptual and practical issues.

In the following section, we outline the steps that need to be taken when planning a mixed methods study to examine behavior change and ICS and we then provide an example of a mixed methods approach used in an ICS study we carried out in Nyanza, Kenya, to illustrate how such an approach can be implemented in practice. Although we focus on ICS, the methods discussed are also relevant to studies focusing on scaling up of cleaner fuel use and behavior change.

Mixed Methods Approaches to Investigating Behavior Change in the ICS Field

Epistemology

In terms of epistemology, pragmatism is a very popular world view in mixed-methods research (Tashakkori & Teddlie, 2010). The emphasis on user experience in ICS research and focus on application in the field, suggest that mixed methods ICS research would be well suited to a pragmatic approach. Pragmatic approaches focus on matching research methods to the research question at hand and treat issues related to the research itself as the principal line of action (Morgan, 2007) while still recognizing the different world views of quantitative and qualitative paradigms. Knowledge is viewed as being relative as opposed to absolute and there may be singular or multiple realities (Creswell & Clark, 2011). Thus, it is able to accommodate different perspectives.

Study Design

There are many different ways of classifying mixed-methods study designs, a detailed discussion of which is outside of the scope of this article. However, more detail on different approaches to classifying mixed methods can be found in Tashakkori and Teddlie (2010) and Creswell and Clark (2011). For the purpose of this article, we have drawn predominantly on the work of Creswell and Clark (2011), and identified the following classifications as being of particular relevance for studying behavior change and ICS use, not least, because they describe and integrate different aspects of study designs currently used in behavior change research.

The Exploratory Model (Also Known as Instrument Design)

This model aims to develop a data collection instrument grounded in the views of the participants, and thus starts with a qualitative phase and then moves on to quantitative instrument design and testing. An example of such an approach would include focus groups and interviews to explore views on stove satisfaction, which would subsequently be analyzed and used to inform the development of a survey tool to assess views on the relative merits of different stove characteristics.

The Convergent Design Model

This model collects qualitative and quantitative data concurrently and merges the findings in relation to specific research questions. Qualitative and quantitative data on stove use and stove satisfaction, that are subsequently integrated at the analysis or the interpretation stage would be an example of this.

Transformative Model

This model is used in research that challenges social injustices and uses a transformative theoretical lens through which to view the research problem. It may use a quantitative or qualitative priority or can give equal priority to the two strands. A mixed-methods study that explores the process and outcome of household decision making on stove acquisition in a rural community using a feminist lens, would be an example of a transformative design.

The Explanatory Model

This model has an initial quantitative phase, followed by a second qualitative phase. For example, researchers may assess stove use and HAP levels through stove use monitoring (known as SUMS) and air quality measurement; and subsequently carry out interviews with stove users to explore their reasons for multiple stove use (also known as stove stacking).

The Nested Model (Also Known as an Embedded Approach)

This approach is generally used with an intervention trial design in ICS research, where a qualitative component is nested within the study (although it could also include a qualitative study design with a quantitative component embedded within the study). A common example is an intervention study, with a qualitative study embedded within the intervention procedure, to understand how participants experience the intervention.

Choice of Model

Any of these models could be used for a behavior change study, the choice being dependent on the focus of the study and the specific research questions posed. The chosen study design will also influence the approach taken to the analysis as discussed later.

Methods of Data Collection

The most common methods of data collection include the following.

Survey

Baseline quantitative surveys can provide sociodemographic data, information on current stove use, cooking practices, fuel collection, and consumption; and follow-up surveys can collect comparable information in relation to ICS use.

HAP and Stove Use Measurements

A range of quantitative household pollution measurements can be taken, initially with the traditional stove (baseline measurements) and subsequently with the ICS, including kitchen (PM2.5) and personal (mother, child) exposure and also stove use monitors (SUMs). These can provide an estimate of household as well as personal exposure to pollutants.

Time Activity Diary

A time activity diary generally includes structured (quantitative) observations although it is also possible to collect more open-ended data at the same time. Households can record cooking episodes, start and end time for cooking, stoves used, type of meal cooked, and number of people cooked for. Such data are useful to compare user-cooking behaviors between the traditional stove and ICS use and can also be used to triangulate with HAP monitoring data. Women can record these data for themselves, or fieldworkers can record the information on their behalf.

Qualitative Interviews

Unstructured, or more commonly semi-structured interviews conducted by trained field staff are useful for exploring user views on a range of topics including cooking behaviors, stove attributes, and views of the ICS.

Focus Groups

Focus group discussions allow participants to share their views on stove functionality, design and acceptability, as well as reasons for multiple stove use and views on stove promotion. Compared to individual interviews, they capitalize on group interaction and people can explore and clarify their views in a way that is not possible in individual interviews (Kitzinger, 2006).

Summary of Methods

Table 1 provides a list of the common factors investigated in relation to ICS uptake and outlines which methods might be used to explore each factor. Some factors can be explored by multiple methods whereas others are more suited to one specific method. This table can be used to identify which methods would be most pertinent to the research questions being posed. It contains three categories, namely: contextual factors, fuel and technology characteristics and user behavior. Although the focus of a study may be on user behavior, it can clearly be influenced by context and by the stove in question, so it is important to measure all three categories of factors at the same time. Further, multiple methods may be used to measure many of these factors, and different aspects of a factor can be measured depending on the method chosen. For example, HAP monitoring can measure one key aspect of stove performance, but qualitative interviews are useful in eliciting user views on stove performance, both of which are likely to affect uptake and sustained use.

Table 1. Methods used to investigate common factors in relation to ICS uptake

	Household air pollution and stove use monitoring	Time activity diary	Survey	Qualitative interviews	Focus groups	Observation
Contextual factors
Household composition		✓	✓			✓
Gender roles		✓	✓	✓	✓	✓
Socioeconomic status			✓	✓		✓
Culture and tradition		✓		✓	✓	✓
Fuel and technology characteristics
Household PM and CO	✓
Personal CO/PM2.5	✓
Stove use	✓	✓	✓	✓	✓	✓
Stove maintenance		✓	✓	✓	✓	✓
Type and quality of biomass	✓	✓	✓	✓	✓
Fuel savings		✓	✓	✓	✓	✓
Stove characteristics			✓	✓	✓
Stove performance	✓		✓	✓	✓
Presence of smoke		✓	✓	✓	✓	✓
User behavior
Stove stacking	✓	✓	✓	✓	✓	✓
Saving time		✓	✓	✓	✓	✓
Gender roles		✓	✓	✓	✓	✓
Willingness to buy			✓	✓	✓
Satisfaction with stove			✓	✓	✓
Health outcomes
Symptoms			✓	✓	✓	✓
Risk of disease outcomes^a	✓
Disease outcomes^b			✓			✓
Burns			✓			✓
Health services use			✓	✓	✓

^aBased on published exposure response functions that describe relation between PM2.5 exposure and relative risk of five disease conditions (Burnett et al., 2014).

^bFor example, reported diagnosed cases.

Approaches to Analysis

There are a complex array of approaches to data analysis and a number of ways in which they may be described and categorized. The following three processes are examples that can be applied to the study designs discussed previously (Creswell & Clark, 2011);

1. Merging involves analyzing quantitative and qualitative findings separately and then combining them in relation to specific themes identified, such as reasons for not using ICSs, or behaviors related to fuel use and procurement (for an example, see Jagoe et al., 2006).

2. Connecting is the chosen approach when mixed methods are carried out sequentially and one method is used to inform the other, for example carrying out qualitative interviews to identify how and why households stove stack, which can then be used to inform a survey to identify how frequent practices are within a community; or conversely, carrying out a survey to identify stove stacking practices and then using the findings to develop a semi-structured interview schedule to explore reasons for stove-stacking practices in more depth.

3. The embedding analytical approach is used when a secondary dataset is embedded within a larger primary design. It would be appropriate in an ICS study where stove users’ views could be explored in relation to stove use practices and HAP reductions. Thus, the qualitative findings would provide an enhanced understanding of the results.

The study design model generally influences whether one method takes precedence over the other; whether mixed methods will be used concurrently or sequentially, and whether integration occurs at the analysis or the results stage of the study. Table 2 outlines how these different elements of mixed-methods design relate to the study design model used.

Table 2. Study design considerations in a mixed-methods study

Study design model	Priority	Implementation	Analysis
Exploratory (also known as instrument design)	Priority qualitative	Sequential with qualitative first	Connecting
Convergent	Equal	Concurrent	Embedding or merging
Transformative	Equal, priority quantitative, or qualitative	Concurrent or sequential	Merging, connecting, or embedding
Explanatory	Priority quantitative	Sequential with quantitative first	Connecting
Nested (also known as embedded)	Priority quantitative or qualitative	Concurrent or sequential	Embedding

Note. Adapted from Creswell and colleagues (2004) and Creswell and Clark (2011).

Validity and Reliability in Mixed-Methods Research

Although validity in mixed-methods studies is recognized as being an important issue, Onwuegbuzie and Johnson (2006) believe discussions are still in their infancy. Tashakkori and Teddlie (2006) suggested that validity in mixed-methods studies should relate to the different research phases; design, data collection, analysis and data interpretation. Thus, validity can be addressed within the quantitative and qualitative strands of the research, as well as during the integration phase. Researchers may therefore choose to use conventional qualitative and qualitative approaches to validation during data design and collection but considerable thought also needs to be given to addressing validity during the data analysis and interpretation phases.

In relation to data analysis and interpretation, there are a number of tools that can be used to strengthen validity, including triangulation of data sources (in this context we are using the term triangulation to describe corroboration between two or more sets of findings), and also member checks (returning to the data source, in this case, the participants, for verification). For example, triangulation of data sources (interview findings, focus groups, and HAP measurements) can be useful to corroborate findings in relation to user behavior and multiple stove use (stove stacking). In addition, when using qualitative methods alongside a quantitative study, it is often not feasible (or necessary) to include all study subjects in the qualitative sample. It is generally feasible to subsequently return to the full dataset and to compare and contrast findings within the wider study population. A more detailed discussion of different strategies that may be used to validate findings is not within the scope of this article, but for further discussion and guidance, see Onwuegbuzie and Johnson (2006), Creswell and Clark (2011), and Tashakkori and Teddlie (2010).

In terms of generalizability, quantitative approaches are concerned with assessing how generalizable the study findings are to the wider population. Conversely, with qualitative approaches, the emphasis tends to be on credibility (confidence in the truth of the findings) and confirmability (the extent to which findings are shaped by the participants) (Lincoln & Guba, 1985). However, taking a pragmatic approach, Morgan (2007) suggested that the distinction between knowledge that is context dependent (qualitative approaches) and knowledge that is generalizable (quantitative approaches) is not a valid one. This is because knowledge is rarely specific to a particular context (for example only relevant to one household) nor is it universally generalizable (representative of all households). Instead, Morgan argued for the importance of transferability. Thus, the more useful question for the field researcher to ask is “Do these results have implications for similar programs in different contexts?” That is, the issue is whether similar results are also found among different populations or if user perspectives vary in different contexts and if so, how. Thus, a pragmatic perspective is practical and useful in ICS field research and mixed-methods approaches allow us to use the strengths of both qualitative and quantitative approaches in relation to the validity and reliability of our findings.

Convergent Designs: A Case-Oriented Approach

As with other aspects of mixed methods, there are different approaches to converging findings. In this article, we examine the case-oriented approach (or case study approach) in the following example. Because much ICS research collects data at the level of the household, individual households can act as cases. This kind of analysis draws on the case study approach and allows a detailed investigation of phenomena within context (Hartley, 2004). The number of cases chosen depends on the type of data, the research question, and the richness of the different data sources. However, emphasis is on exploration and description, not on generalizability, so in some circumstances even just one case can be sufficient, whereas in others, a group of cases may be more appropriate (for a more detailed explanation, see Flyvbjerg, 2006). In an ICS study, a case unit can be defined in a number of ways depending on the research question. Table 3 provides some possible examples.

Table 3. Examples of case units in an improved cookstove mixed-methods study

Case unit	Focus of study aim
Household	To identify trends in individual households over time and with different stoves
Maximum/minimum PM reductions	To identify contextual factors associated with maximum and minimum reduction in PM2.5
Stoves	To identify contextual factors that differ between stoves
Specific stove: maximum and minimum reductions	To identify maximum/minimum reductions associated with individual stove use
Focus on an identified theme	To explore specific themes identified in the analysis

Integrated Methods in Practice: An Example From a Mixed-Methods ICS Study in Kenya

The following example is from a study we carried out in Nyanza Kenya (Christensen et al., 2013) and briefly outlines two ways in which mixed methods can be used in ICS research.

Summary of Study Design

We carried out a study in Nyanza Kenya to evaluate a number of ICSs with the potential for delivering low(er) levels of household air pollution exposure in everyday use, and to determine their acceptability for uptake and exclusive use. Six ICS were field-tested in 42 households, each for a 2-week period. Women were trained to use each new stove. Their traditional stove remained in the home but they were encouraged to use the new stove for cooking. Data collection included a baseline survey on current stove use and cooking practices, time activity diaries, structured interviews, focus groups, kitchen and personal exposure monitoring, and kitchen performance tests. In total, there were 264 two-week trial periods among the 42 households.

We do not present the main findings of the study here. Rather we use the study design as an example of how a mixed-methods approach and analysis can be carried out in an integrated way and we present some example case study findings to demonstrate the insights that can be gained when quantitative and qualitative data are integrated.

Methodological Approach

Our study group comprised a team of researchers from a range of disciplines incorporating quantitative, qualitative and mixed-methods expertise and included experienced fieldworkers and a project manager based on site in Kenya. We wanted to determine effectiveness of the stoves in terms of reducing HAP but we recognized that if the stoves were not acceptable to users, then stove effectiveness would make no difference in practice. We took a pragmatic approach to our inquiry and our specific research questions were as follows:

1. How effective are the stoves in reducing household air pollution in the field?

2. How acceptable are the stoves to the user?

3. What is the relationship between user acceptability and user behavior?

First Stage: Quantitative and Qualitative Analysis

We initially analyzed the quantitative and qualitative data separately, to measure HAP reduction by stove and stove acceptability. For HAP reduction, we reported stove use, cooking practices, fuel consumption by stove, and kitchen and personal exposure by stove type. We carried out a univariable and multivariable analysis of factors associated with 48-hour mean gravimetric PM2.5 (mg/m³) concentration. In relation to stove acceptability, we analyzed the qualitative data using a thematic approach (Ziebland & McPherson, 2006) and views on each ICS in comparison with the traditional three stone were reported, as well as views and rankings of the individual stoves in comparison with each other (to be published separately).

Second Stage: Mixed Methods—Nested Approach

We took a nested approach, merging the quantitative and qualitative findings by stove, reporting HAP measurements, time activity diary, multiple stove use, and characteristics of the stove that women liked and did not like. This provided a summary of the performance and acceptability of each stove. We also examined the relationship between stove ranking and stove use, preliminary analysis suggesting that there was no clear relationship between stove ranking and number of times the stove was used. This approach was useful for drawing generalizable conclusions with regards to the research questions (in this case, how effective are stoves at reducing HAP and how acceptable are they to the user?) as we were able to compare characteristics and performance by stove type, with some incorporation of users’ perspectives. However, it did not allow an understanding of stove use in context or an exploration of the relationship between user acceptability and user behavior.

Third Stage: Convergent Design—A Case-Oriented Approach

As a third stage, we analyzed user behavior within a household context to understand in more depth the relationship between women's perceptions of the stoves and their subsequent use. We used a convergent case study design giving quantitative and qualitative data equal priority.

Because we were interested in contextual factors associated with maximum and minimum reduction in PM2.5, we decided to explore twelve case studies in two groups, six households with the highest absolute reduction in 48-hour mean kitchen PM_2.5 between baseline and ICS use during the monitoring period, compared with the six households with the lowest reduction (all of which in practice had an increase in 48-hour mean kitchen PM2.5 between baseline and ICS). This allowed us to compare and contrast specific aspects of stove use in context between the two groups. We chose to use the convergence coding matrix (the overall process is termed a triangulation protocol), developed by Farmer, Robinson, Elliott, and Eyles (2006) because it provides a clear step-by-step framework to converging data for researchers who are not experienced in mixed-methods analysis, and in addition, it provides a useful framework when a team of researchers are contributing to the analysis and wish to ensure that the approach taken to analysis is uniform. This was originally designed for qualitative studies but can equally be applied to a mixed-methods design as suggested by O'Cathain and colleagues (2010) and is outlined in Table 4.

Table 4. Triangulation protocol using a converging coding matrix

1. Identify cases (households with greatest and least reduction in PM2.5).

2. Data extraction: for each household, list together qualitative and quantitative findings that address the research question(s) of interest.

3. Convergence coding: develop a convergence coding matrix based on the specific research questions. The themes form the rows of the matrix, and the different levels of convergence, the columns.

4. Convergence assessment: for each case, complete the matrix, which sorts findings from each data source into relevant segments that address the research question. Identify whether findings suggest convergence, complementarity, discrepancy, or dissonance.

5. Researcher comparison: multiple researchers review each case and clarify findings. Explicitly looking for evidence of dissonance between findings is a key part of the process. This could be dissonance between cases, or between different data sources.

6. Group comparison: compare findings between the two groups of cases (high PM2.5 reduction versus low PM2.5 reduction).

7. Feedback: feedback of results to research team and stakeholders for review and clarification.

8. Identify further research questions: consider further triangulation by returning to larger data set and/or examining other case groups within the study; or additional data sources (in this example, focus group findings).

Note. Adapted from Farmer and colleagues (2006).

Example Findings From Stage 3: A Case-Oriented Approach

Households with the highest HAP reductions did not differ in terms of households assets compared with the households with the lowest reductions. Our findings indicate that most women in both groups worked outside of the home during the day; this had a major effect on their stove use and cooking practices and thus needed to be taken into careful consideration.

Women reported a number of negative aspects to the stoves, even those that they ranked highly but not all characteristics were considered to be a barrier to regular use. However, difficulty in regulating heat, the need for firewood preparation, difficulty accommodating big pots and the stove being unsuitable for a large family were major barriers to regular use.

Women in the group with the greatest PM2.5 reductions reported continuing to use the new stoves during the monitoring period, even in cases where the stove did not meet all their cooking needs, suggesting that for some women, complying with study requirements during the observation period was a motivating factor, regardless of their stove views.

Ranking a stove highly did not necessarily lead to exclusive use. The majority of households in both groups reported multiple stove use (stove stacking). Their reasons for stove stacking included the need to; cook faster, cook a number of dishes at once, and cook ugali (a local dish similar to porridge which is made of maize), which requires an intense flame. Another factor associated with multiple stove use was children taking responsibility for cooking while women were at work and, as they were not familiar with how to use the ICS, they would cook on the three-stone. We subsequently carried out focus groups with all study participants to explore multiple stove use further and found that the views held by the women within the case studies, were supported by the wider study population.

In one case study, although a woman was using her first choice of stove, there was still an observed increase in PM2.5 levels compared with baseline (three-stone fire use). On further analysis of the qualitative interview data and the time activity diary, she reported that she continued to cook all rice and beans on the three-stone believing it to be more appropriate for traditional dishes, and regularly had two fires lit within the household. On returning to the interview data and carrying out subsequent focus groups, we found the view that traditional foods are better cooked on a three-stone fire, to be widely held by study participants and that for many women, even those who achieved reductions, the view of the ICS was as a technology to add to the existing stove, rather than a replacement for the three-stone fire.

Self-reporting of ICS use in the time activity diary was over-reported, in comparison with ICS use recorded using SUMs data. When this was explored further using focus groups after the fieldwork had concluded, women were subsequently more candid during the focus groups about their continued use of the three stone.

Strengths of a Case-Oriented Approach

A case-oriented approach enables us to explore user behavior in context, integrating both qualitative and quantitative findings. We found Farmer's convergence coding matrix (Farmer et al., 2006) a useful tool in relation to ensuring that all researchers followed the same analytical approach, particularly since expertise in mixed methods varied greatly throughout the team. We recommend, however, that as teams become more experienced that they also explore other approaches to convergence depending on the nature of their data and the specific research questions they are seeking to answer (for further details, see Tashakkori & Teddlie [2010] and Creswell and Clark [2011]).

An example of an integrated finding from our case-oriented analysis was that all but one of the women worked outside the home. On analyzing the qualitative interview data with the findings from the time activity diary, we found that children often took responsibility for cooking while their mother was away. However, the children were not appropriately trained in using the ICS. Therefore, they continued to use the three-stone fire when cooking, and it became apparent that this was a common reason for multiple stove use. Thus, in terms of promoting more exclusive use this would suggest a need for training in the use of the ICS to be extended to all family members responsible for cooking.

Validity and Generalizability

In relation to validity, we found triangulation of data sources (interview findings, focus groups, and HAP measurements) was useful to corroborate findings and allowed us to gain a better understanding of the validity of different methodological approaches in terms of examining specific issues. For example, dissonance between the time activity diary and SUMs data suggested a tendency for participants to overreport ICS use in the time activity diary, perhaps to be seen to conform to the requirements of the study. In addition, we were able to return to the larger dataset and compare and contrast findings within the wider study population. Such an approach was used with follow-on focus groups, which were conducted to confirm the view that the ICS was seen as a technology to add to an existing stove rather than a replacement for the three stone.

Measuring ICS Use Over Time

Measuring sustained ICS use and HAP reduction presents a complex problem to all researchers, not least because many factors are unlikely to remain constant over time, even within households, such as time spent cooking, stove stacking, number of people cooked for, use of stoves outside, and wood drying for example. Furthermore, in the case study example, where households achieved the greatest reductions, this may not necessarily be reflected in their stove use over a longer period of time when not being observed, as findings suggest a desire to conform with the requirements of the study over the monitoring period. Thus, measuring these different factors over a sustained period is very challenging. Further development of mixed methods, and in particular, observational methods using an ethnographic approach, would be useful here because researchers would be able to examine and observe stove use and practices over time, and also providing a better understanding of behavioral practices that affect HAP, such as overstuffing or stove stacking.

Prioritizing Methods

In terms of the role of quantitative and qualitative methods, we would advocate for both methods to be treated as being of equal value during design, execution, analysis, and reporting wherever possible in ICS research. In practice, the key to ensuring this happens is good planning at the design stage. However, we have also provided an example of how different approaches to data weighting can occur within one study. Thus, we would argue that it is possible for the findings of a study to be reported using different mixed-methods design approaches to answer different research questions, or different aspects of the same research question.

Potential Scope of Mixed Methods to Contribute to Behavior Change Approaches

As mentioned previously, facilitating behavior change in the context of ICS promotion is a complex issue. The use of quantitative or qualitative methods alone is not sufficient to gain a full understanding of the relationship between user perspectives and ICS use in the field and further development of methods that integrate the two approaches is an important next step. The methods discussed in this article and the factors represented in Table 2, do not cover more distal factors likely to impact on stove uptake, such as safety legislation, or supply chain factors. Nonetheless, there is no reason why mixed methods could not be extended to explore these. In tandem with this, the behavior change approach should extend beyond stove users to incorporate all stakeholders (e.g., stove suppliers) involved in the upscaling of ICS, but should view stove users perspectives as being key to all aspects of public and commercial activity that contribute to upscaling. For example, stakeholder perspectives are key in identifying stove improvements that can be addressed by manufacturers through design changes. Ultimately, if user needs are addressed through design requirements then users are more likely to move toward exclusive ICS use.

Conclusion

Measuring user behavior in the field, so that stove users can be encouraged to develop cleaner cooking practices is a very challenging and complex goal. A key difficulty in relation to using experimental designs as a method of understanding behavior change in the field, is the linear understanding of causality and the lack of attention to context (Brady & O'Regan, 2009). Methods for integrating qualitative and quantitative findings are available to overcome this problem, but currently, such methods represent relatively uncharted territory. There is much room for development and more published examples are required to provide frameworks for other researchers wishing to apply mixed methods to these issues. This article has made the case for some of the benefits of using mixed methods in ICS research and identified some of the challenges in promoting greater integration of such methods. In summary, qualitative studies can provide a voice for the stove user, particularly in relation to their reasons for choosing to use or not use the ICS, and provide context for the interpretation of stove use behaviors, Quantitative studies can provide a measure of the effectiveness of stoves in reducing HAP, enhance the interpretation of qualitative data and provide information on the generalizability of the findings when specific stove use behaviors are identified.

From a broader perspective, there is a need for a comprehensive research agenda to accompany initiatives aimed at promoting ICSs and cleaner fuels. This agenda should promote quantitative studies of effectiveness, integrated with qualitative studies that explore stakeholder perspectives to provide better evidence with which to promote adoption, and exclusive and sustained use. Within such an agenda there is also a need for mixed-methods behavior change approaches to be broadened to include not only users, but all major stakeholders including researchers, civil society, the government, and the private sector.