Delivering on the promises of wellbeing? Traidcraft Exchange’s experiment with measuring wellbeing

ABSTRACT

Improvements in “wellbeing” are frequently cited as the ultimate objective of “development”. Efforts to operationalise wellbeing quickly turn to discussions of definition, assessment, and measurement. This article describes Traidcraft Exchange’s ten-year work to assess changes in wellbeing resulting from their interventions. We outline the conceptual model Traidcraft adopted, how it was adapted, and how it has delivered on core promises of wellbeing. We explore how differing priorities influenced implementation and results. An explicit focus on wellbeing promises more complete information and positive, empowering, person-centred “development”. Achieving both ambitions requires clear objectives and has implications for the methods and processes used.

KEYWORDS:

• Wellbeing
• monitoring evaluation and learning (MEL)
• INGOs
• person-centred development
• decolonising aid

Introduction

Traidcraft Exchange (TX), a UK-based charity, works in Africa and South Asia, where it has regional offices working with partners and communities. In 2010, Traidcraft published a new strategic plan, “From Fair to Flourishing”. The plan committed both Traidcraft Exchange and its sister company Traidcraft plc (a fair-trade business) to contribute to “a world freed from the scandal of poverty, where trade is just and people and communities can flourish”. (Traidcraft Foundation 2010, 1). The Board of the organisations asked TX to consider how this commitment to “flourishing” could be promoted in practice – and how success could be measured.

At the time, there was widespread interest in “wellbeing” as a goal of public policy and international development. The Sarkozy Commission had reported in 2009 (Stiglitz, Sen, and Fitoussi 2009); the UK Government began gathering data on wellbeing (ONS 2012); the Department for International Development (DFID) and the Economic and Social Research Council (ESCRC) sponsored research at the University of Bath into wellbeing and development (WED) (http://www.bath.ac.uk/soc-pol/welldev/). The OECD was investigating how to measure and report on subjective wellbeing at a national level (OECD 2013). The first “World Happiness Report” was launched in 2012 (Helliwell, Layard, and Sachs 2012). The same year, the Kingdom of Bhutan hosted a “High-Level Meeting on Wellbeing and Happiness” at the UN headquarters (Royal Government of Bhutan 2012).

In the NGO sector, there was similar interest. In 2004, the New Economics Foundation published “A well-being manifesto for a flourishing society” (Shah and Marks 2004). Of more direct relevance to Traidcraft was “Wholly Living: a new perspective on international development”, by Theos, CAFOD, and Tearfund (2010). This report, explicitly drawing on Christian theology, argued “development” as conceived and experienced in both “developing” and “developed” countries was flawed and ultimately harmful. It argued:

we desperately need to regain a fuller, more realistic vision of human flourishing – of humans as creative, productive, responsible, generous beings – if we are ever to address the problems of poverty, inequality and environmental degradation that threaten the world (Theos 2010, 11)

For Traidcraft, “flourishing” or “wellbeing” was not just about an individual's emotional or mental state – “happiness” or “subjective wellbeing” (SWB). Traidcraft understood individuals as fundamentally social, flourishing within relationships rather than only as individuals, and wanted a framework that acknowledged this, and could be used at the project-level.

During 2011, Traidcraft researched possible models for adoption and identified as most suitable a framework being developed by a University of Bath team as part of an ESRC/DFID-sponsored research programme, “Wellbeing and Poverty Pathways” (WPP) (https://researchportal.bath.ac.uk/en/projects/wellbeing-and-poverty-pathways).

Named “Inner Wellbeing”, the framework embodied a holistic, person-centred approach, drawing on both objective and subjective information, and rooted in an understanding of the person as a social being. It was positive in outlook, focused on what people have and what they can be, rather than what they lack (White 2009). It aimed to provide an alternative to the “subjective wellbeing” or “happiness” approach to wellbeing measurement, which tended to a focus on the individual, was quantitative in approach and had roots in a particular culture and perception of wellbeing. This approach promised not only more complete information but also “a more democratic process” – in that variables of assessment were locally generated, and survey questions carefully piloted to ensure relevance and intelligibility (White and Jha 2014; personal communication 2020).

As well as a providing a conceptual model of wellbeing, WPP offered a methodology for generating data across seven wellbeing domains. Traidcraft was attracted by the concept’s holism and ability to quantify wellbeing with statistically robust methods, which the organisation hoped would provide credibility with key supporters.

Traidcraft expected this would generate more complete information about how participants’ wellbeing changed during interventions, and prompt changes in organisational behaviour and project design, leading to better outcomes for participants. For at least one person behind the “From Fair to Flourishing” strategy, a key measure of success was whether wellbeing assessments influenced TX’s behaviour and project design.

This article explores the extent to which TX succeeded in realising these two promises and suggests that while wellbeing assessment has provided more complete information, there has been limited success in achieving the second promise of increasing participants’ influence on project design. We explore why this has been the case.

Methodology

This article is based on an internal report produced for Traidcraft Exchange (TX) as part of a PhD in International Development at the University of Edinburgh. The research used a case study approach and draws on document analysis and in-depth interviews with TX staff, consultants, and ex-staff in the UK and overseas; academics associated with the WPP team; and TX partner staff. This was supported by analysis of survey reports, evaluations, reviews, annual reports, and accounts. It also draws on “Traidcraft: assessing human flourishing” (Spencer, Williams, and Stevens 2014). Findings were discussed in a group verification session with senior Traidcraft programme staff. The research gathered only limited information from staff of partner organisations, and no information from project participants. This is acknowledged as a limitation.

The “inner wellbeing” conceptual framework

The WPP programme aimed to explore the extent to which “subjective” perspectives add value to more “objective” factors in explaining poverty dynamics (White and Jha 2014, 56). In their model, wellbeing is conceived as having three dimensions: material, relational, and subjective, reflecting an individual’s objective conditions and their interpretation of them, affected by their relationships with others, and the cultural norms and values that influence their interpretation of what is good and bad. The model deliberately takes a psychosocial approach, breaking with the individualism of much of the wellbeing literature, assessing the individual’s experience in the context of external factors that influence it. It recognises that people’s assessments of their wellbeing are dynamic. Given the complexities of assessing and interpreting subjective reports of wellbeing and the domains, the Bath team recommended these reports be assessed alongside “objective” data to triangulate and contextualise the information (White and Jha 2014).

In developing a practical tool, WPP drew on the “social justice approach to wellbeing” developed by the Sri-Lankan organisation Psychosocial Assessment of Development and Humanitarian Intervention (PADHI 2009, cited in White and Jha 2014, 56). PADHI’s approach had five domains: access to resources, experiencing competence and self-worth, exercising participation, building social connections, and enhancing physical and mental health (Abeyasekera 2014.) WPP took and re-worked these and added two more: close relationships and values and meaning.

The model was tested in India and Zambia using surveys. Respondents were asked five questions for each of the seven domains, plus three life review questions. The reasons behind their answers were explored in conversation between interviewer and respondent. The questions were scored, and the scores aggregated into a single numeric score per domain. Factor analysis was used to validate the items within each domain and confirmed that the domains together captured “different facets of a single underlying concept” – Inner Wellbeing (White and Jha 2014, 59). The seven domains were:

1. Social connections, reflecting the importance of social and political connections to access resources.

2. Close relationships, reflecting the importance of family and friends to wellbeing.

3. Competence and self-worth, reflecting an individual’s own sense of worth and ability.

4. Physical and mental health, reflecting the importance of health.

5. Values and meaning, reflecting the importance of morals, norms, religion, and spirituality.

6. Economic resources, reflecting the importance of material resources.

7. Agency and participation, reflecting the importance of being able to participate in and influence issues that affect an individual.

To compare the results with more established measures of subjective wellbeing, three general questions were asked, including a widely used “global happiness question” and two questions on participants’ views of how their economic circumstances had changed over time.

Adapting the model in Traidcraft

In 2011, prior to first use, the University of Bath and Traidcraft Exchange (TX) teams modified the IWB framework to improve its fit with TX’s resources, methods, and project MEL. We have called this adapted framework the TX wellbeing framework to distinguish it from the IWB model. The principal modifications were:

• the survey was reduced from 38 questions to 19 in order to make the process quicker, cheaper, and more feasible for project teams. Following the first full pilot, WPP statisticians assessed the reliability and correlation of the reduced TX wellbeing framework and found good levels of both.

• A group-based survey method replaced the individual survey interview. Men and women were briefed in groups about the survey and completed it at the same time, as individuals. This allowed TX to reach the minimum number of respondents needed for credible quantitative analysis (100), while reducing the time and costs involved. The group-based approach also supported people who had low levels of literacy.¹

• A process to aid participants who were not literate was introduced, whereby facilitators explained the questions and used symbols to aid understanding of response options.

The framework was piloted in Rajasthan and Delhi in early 2012, and revised and then tested with baseline surveys in projects in Bangladesh (2012), Tanzania (2013/4), and India (2014). Endline surveys were carried out in 2015 and 2017, respectively. No endline was prepared for the India project. Subsequently, wellbeing data was generated for projects in Kenya, India, Bangladesh, and Tanzania (see Table 1 below).

Table 1. Wellbeing assessments.

Survey	Year of baseline	Year of Endline	Type
Smallholder farmers, Bangladesh	2012	2015	Adapted IWB methodology (Group based survey). Complete cycle.
Small scale tea growers, Bangladesh	2016	Not done	Adapted IWB methodology (Group based survey). Incomplete cycle.
Beekeepers, Tanzania (two locations)	2013 & 14	2015	Adapted IWB methodology (Group based survey). Complete cycle.
Textile Artisans, India	2014	Not done	Adapted IWB methodology (Group based survey). Incomplete cycle.
Export horticulture pack-house workers and farmers Kenya	2014	2016	Adapted IWB methodology (Group based survey). Complete cycle.
Smallholder farmers, Kenya	2015	2017	Adapted IWB methodology: Group survey for baseline, Household (HH) survey for endline. Complete cycle.
Small-scale cotton growers, India	2015	2019	Adapted IWB methodology (Group based survey). Complete cycle.
Informal women workers, Bangladesh	2016	Not done	Incomplete.
Smallholder farmers, Kenya	2019	Not yet due	Baseline only (endline not due), 3 domains, not 6. HH Survey used.
Smallholder farmers, Tanzania	2019	Not yet due	Baseline only (endline not due), 3 domains, not 6. HH Survey used.

By 2014, further changes were made to simplify the model. The principal changes were:

• Removal of one domain: Close relationships. Pilots suggested that responses to questions in this domain were heavily influenced by social expectations, and it was proving difficult to identify questions that overcame this (White, Gaines, and Jha 2014). Although TX acknowledged it compromised the integrity of the IWB model, it was decided to drop this domain.

• Introduction of an additional group discussion after survey questionnaires were completed. Originally, one participant was selected from each group of informants for an individual interview to contextualise and further explore question responses. However, these interviews failed to generate valuable information, and there were concerns about bias and creating division within groups. Individual interviews were replaced with a group discussion. This provided an opportunity for people to informally discuss the questions together and explore broader issues affecting wellbeing. In these, TX also explored how far people attributed changes in their own wellbeing to project activities.

Results from the first complete survey cycle with smallholder farmers in Bangladesh were encouraging. The assessment reported increases in all domains between baseline and endline. Statistical analysis confirmed the differences were statistically significant. The overall wellbeing score increased from 3.21–3.96 on a five-point Likert scale from one to five, indicating an overall increase in reported wellbeing amongst the respondents (TX 2015a).

It should be noted that by 2014, the WPP team had concluded that a single wellbeing score had limited utility, instead recommending the use of domain scores or individual item responses. For TX, a single score remained important for tracking overall outcome-level changes in wellbeing, and reporting to project donors. At the same time, a more detailed discussion of domain results was carried out between head office and project teams.

Results from the next project, with beekeepers in Tanzania, were more mixed. In one project location, all domain scores increased; in the other location, four domain scores increased and two decreased. There were interesting differences between men and women. Overall, the TX team felt the results were complex and hard to interpret or summarise in a concise, easily communicable way.

Other factors raised questions about the results. Many are common to data collection exercises, including limited training for enumerators, and staff turnover reducing knowledge of the approach. In addition, the relatively short period between baseline and endline, and the holding of surveys at different times of the year, complicated interpretation (TX 2015b). Finally, some factors specific to the collection of subjective views may have played a part, including tendencies to return to a “set point” of wellbeing, and adapting to circumstances or new expectations.

The next full cycle of results in 2016 with a project in Kenya’s export horticulture sector were again ambiguous, this time with no significant differences between baseline and endline. However, project implementation had been seriously disrupted by a change in ownership at the partner enterprise (Consumer Trends Ltd 2016).

Results in 2017 from a smallholder farmers project in Kenya were positive, reporting improved wellbeing for men and women, young and old. The mean for both females and males increased from 3.21–3.55, with greater improvements for women. Scores against the general review questions provided similarly positive results. The results again presented a complex picture with women’s improvements concentrated in “Social Connections” and “Competence and Self-Worth”, and men’s in “Economic Resources” and “Values and Meanings” (TX 2017).

The most recent complete cycle reported in 2019 and found overall positive increases in wellbeing for women and men participating in a cotton project in India – and noticeable improvements in the domains for economic resources, physical and mental health, agency and participation, social connections, and competence and self-worth (TX 2019a, 2019b).

While the TX wellbeing assessment was being used in some projects, some project teams chose not to use it. Some felt that while the idea was sound, a different method more in line with available resources and better integrated into the project MEL system was needed. TX teams in East Africa unpicked the framework, taking items that most clearly linked to their project’s “theories of change” and integrated them into the project logframes and the standard survey of project participants used in project MEL.

Both change processes reflected several key concerns: the costs involved, the “stand alone” nature of the original model, and its limited integration with project MEL. There was an uncertainty about a wellbeing “score” – what did it mean? How valid, how reliable was it? And how did it relate to project interventions? The complex nature of the results, exhibiting gains and falls across different domains, and thus a lack of clear conclusions about the direction of change, may also have contributed to a willingness to adapt or deconstruct the framework.

For both approaches, TX continues to calculate an overall wellbeing score at a project-level to assess change over time in the project at an outcome level, as well as assess changes at the domain level.

While both approaches have diverged significantly from the original IWB model, they maintain important, fundamental links with it. These include the concept of wellbeing as a holistic condition, the focus on gathering participants’ own assessment of aspects of their wellbeing, triangulating subjective and objective data, and retaining the three dimensions of wellbeing that underpinned the original concept (material, relational, subjective). In their last two interventions, the East Africa team chose questions that reflect the three dimensions: economic resources, social connections, agency and participation, life evaluation, and competency and self-worth.

At the same time, the changes have sought to address concerns and constraints – principally, the “separateness” of the assessment from project design and review, and cost in time and money. The original adapted wellbeing assessment framework required a separate data collection process in addition to the household survey used for the majority of TX’s project indicators. Facilitators had to be trained on the group process and standardised wellbeing questions, with data collection sometimes overseen by an external consultant. Often, Traidcraft and partner staff would be involved as facilitators, and head office staff would clean, analyse, and report on the wellbeing results. Such resources are always in short supply and have to be prioritised against project delivery and other MEL requirements. A greater integration into the standard MEL system arguably also makes the amended approach more likely to influence future practice at the project level.

Shifting strategic focus

While the approach has been sustained at an operational level, at the strategic level an explicit interest in wellbeing declined within TX. Around this time, the trading context for Traidcraft plc (the business) became increasingly challenging, and the two organisations worked increasingly independently of one another. The wellbeing framework was, at a strategic level, put to one side.

After the initial impetus, the changes of leadership and shifting internal and external priorities meant momentum dissipated. The wellbeing framework has not been consistently included in new projects and there has been no organisational requirement to do so. Project design could include the original TX version, use an amended version or no version.

Analysis of TX annual reports confirms the waning interest in wellbeing at the corporate level. Use of the term “wellbeing” or “well-being” – as a proxy for official interest in the concept – reduced over time. Text analysis finds 16 references in the 2014 Annual Report to “wellbeing”. There were only two references in the 2015 report, and four in the 2016 report. There were no references in annual reports 2017–2020 (TX 2014, 2015, 2016, 2017, 2018, 2019, 2020).

Wellbeing’s resilience

However, as is apparent, an explicit pursuit of wellbeing did not disappear from TX’s work. It continued both within the MEL team in headquarters, and at the project level. By 2020, TX had carried out full or partial wellbeing assessments for ten projects in four countries in two continents. Four used the original TX methodology, the others a modified version. Around half (47%) of a sample of projects between 2014 and 2019 had a high-level objective of improving the wellbeing of project participants.

Findings

Assessing wellbeing is a “good thing to do”

Within TX, there is strong support for the idea that increasing the wellbeing of project participants should be the ultimate objective of programme work, and for assessing changes in wellbeing during projects and finding ways to attribute change to projects. Largely without exception, across the organisation, in head and country offices, people like the idea, feel it is a “good thing”, and that it adds to TX’s understanding and knowledge of the reality of project participants’ lives, experiences, and priorities.

The process is believed to have been useful for participants and staff, an opportunity to investigate issues which are not usually asked about, to validate the subjective experiences of participants as a part of development, and to learn together. It can help to set the “tone” of a respectful, inquisitive way of working, based on concern for, and validation of, participants’ experience of “development”. It can build skills for facilitation and research. It can create space for different kinds of conversations, ones that rarely take place within the confines of more standard MEL and project management practices.

But concerns remain

While it may be a “good thing”, operationalising this focus remains challenging – both conceptually and practically. Is wellbeing primarily of interest to the better off and those in the Global North? Is it measurable in a reliable and meaningful way? If it is, is it feasible for a small NGO like TX to measure it? And what is the best way to do so given its operational focus, capacity, and resources? Will doing so generate information that will be useful for reporting, and improve practice and outcomes? And does it change practice or influence design? Can any observed change be attributed to the intervention? Does it deliver on the promises of more complete information and more democratic approach to development?

A matter for the “better off”?

Is wellbeing relevant to the very poor women and men that TX works with, or is it rooted in middle-class levels of affluence, culture, and preoccupations, insufficiently focused on basic needs? After all, much of the popular discussion of “wellbeing” in high-income, industrialised societies takes an individualistic approach, and seems disconnected from the concerns of those living in poverty.

The academic literature on wellbeing bears out the importance of basic needs in shaping self-assessments of wellbeing – particularly for the poorest. Income and wealth are significant determinants of self-reported subjective wellbeing (SWB) (Helliwell, Layard, and Sachs 2020). Poorer people tend to report lower levels of SWB than wealthier individuals. However, the association between wealth and wellbeing diminishes as people get richer. After a relatively low threshold wealth becomes less important as a determinant of wellbeing (Myers 2000).

For the groups that TX works with, increasing income and assets has an important function in increasing wellbeing. The WPP research and TX wellbeing assessments came to similar conclusions. The first two TX wellbeing assessments were carried out alongside more standard household socio-economic surveys. There was a strong correlation between the two. Both reported improvements over time in both wellbeing and income, suggesting that IWB results are correlated with income levels (White, Gaines, and Jha 2014).

If wellbeing, subjective wellbeing, and income are so closely linked for those living in income poverty, should we just focus on raising incomes, and gathering income data as a proxy for wellbeing? Why take on the extra expense in time and resources, for TX, its partners, and communities and participants?

While income is a strong determinant of wellbeing, it is not the only factor. Wellbeing is not – for any income bracket – only a matter of income. Other factors – mental and physical health, relationships, self-worth, agency, participation, culture, values, faith – all play an important role in the level of wellbeing an individual reports (Myers 2000; Smith et al. 2005; Diener and Biswas-Diener 2001; Diener, Tay, and Morrison 2011).

Diener and Biswas-Diener (2001) found in India that the poorest groups (slum dwellers, sex workers, pavement dwellers) report lower levels of life satisfaction than the average for the community, but not as low as might be expected based on a simple index of income. Being able to socialise, to enjoy family life, to do and be what we desire, to exercise agency and influence over our lives, these are important elements in constituting wellbeing even in conditions of extreme material poverty (Helliwell, Layard, and Sachs 2020). Morality is an important element of this. For most people, personal wellbeing is influenced by their sense of whether they live their lives consistent with their personal morality (White 2018).

There is thus a clear case for looking beyond income to these other factors. Using a multi-dimensional approach to explore with project participants whether and how their lives are changing, and in what aspects, provides a framework for identifying and exploring these issues. It can help inform assessment of the intervention and how it can be optimised for all, moving beyond factors defined at the project design stage and necessarily reduced and simplified in the process. These factors also impact on an individual’s ability and willingness to engage in “development” processes.

Some in TX believe partner organisations in Africa and South Asia are not convinced of the value of the wellbeing experiment. Efforts to test this view have been limited as it was possible to identify only one suitable respondent from TX’s partner organisations who could talk knowledgeably on the wellbeing pilot. While accepting the anecdotal nature of this evidence, this respondent was very positive, reporting that TX’s introduction of the approach has led to wider use of it by their own organisation.

Is it feasible for Traidcraft Exchange?

While there is both a value-driven and an evidence-based case for assessing wellbeing as part of project monitoring and evaluation, given the complexity of assessing wellbeing, its dynamic nature, and the short timeframes of most projects, what is feasible?

Over time, TX has arrived at two approaches to assessing wellbeing – the full TX wellbeing assessment, and an amended version that is tailored to each project, with a reduced number of domains and questions chosen to reflect each project’s specific theory of change. Both seek participants’ views on objective and subjective domains of their wellbeing, and both are used to calculate an overall single wellbeing score. The latter is, in TX’s context, more feasible, but less comprehensive.

The original model has proved too complex and resource intensive for TX to use consistently. The methodology requires a high level of skills both in terms of qualitative and quantitative research. It requires strong facilitation and analytical skills for the group discussion work, and a good level of quantitative, statistical analysis for the processing of survey results. These skills are often in limited supply within a small NGO, and time and structures for robust, reviewed research are lacking.

In the pilot phase, TX staff in headquarters and country offices and partner staff were trained in the approach. This training is regarded as valuable, strengthening skills in research and facilitation, and cultivating respectful listening to communities. However, over time, staff turnover has meant the loss of trained individuals. New staff have not been through the same process.

TX has an internal MEL function and a small group of consultants trained in the approach. TX hoped to use the consultants as a cadre of trained researchers and trainers who understood the approach and could coordinate and support assessments but who were not on the organisation’s payroll, avoiding the need to maintain a core technical team.

However, consultants vary in quality and availability, and the results have been patchy. Given the importance of the facilitation on the quality of results, this has been an important constraint, though one relevant to all attempts to measure change.

It has been difficult to find sufficient time to train programme teams properly on the approach in advance of assessments. Often, training has been cut short, meaning team members may have only a limited understanding of the approach and methods.

Perhaps most importantly, the wellbeing assessments stood apart from project MEL systems. This had several consequences: costs for assessments were over and above normal monitoring costs, implementing the assessments required staff time and the use of project resources which were already overstretched, and findings were not fully integrated into the project MEL. For programme teams struggling to deliver projects on very tight budgets, these factors both generated concerns about the approach, and limited the influence findings had on project design and practice.

The changes reduced the scope of the assessment. This lowered costs and increased the likelihood of information being used. However, the holistic overview has been reduced, along with the group discussions. Information is more tightly limited to questions selected from the overall framework related to the pre-defined project design. The group-based survey had been introduced to reduce costs and time, while maximising respondent numbers and open dialogue. Respondents reported the original process provided an impetus and a space to generate and analyse information that would not normally be available, touching on aspects of people’s lives that are influenced by and influence projects, but which do not normally receive much attention. This appears diminished in the amended approach.

One core dilemma expressed by staff in TX about adopting a wellbeing approach is that it is holistic, but TX’s responsibilities, capacities, and interventions are not. Collecting data is expensive for organisations and respondents (especially those that are income, resource, and time-poor). Wellbeing assessments can throw up a range of issues, many of which are outside the scope of the organisation’s capabilities, raising expectations TX cannot meet. Should organisations with a narrow focus only assess elements of wellbeing rather than the whole? Organisations face these questions with any data collection exercise, but the comprehensiveness of a wellbeing approach makes this a particular challenge.

One of the attractions of a wellbeing approach is that by focusing on the person and not the project, it helps us look outside the confines of an (externally) defined set of objectives and explore the extent to which an intervention may be positively or negatively affecting people, including in unexpected or unintended ways. However, it is not always easy to manage such information within the confines of a project – it may be more convenient for both implementers and funders that the available information focuses on the specific objectives and indicators of the project. After all, these have been defined as the measures of success. Additional complex or contradictory information may confuse judgement and narrative.

Given the complexity of assessment, the practical and resource constraints, and the ethical concerns, reducing the assessment to objective and subjective assessments of aspects of life related to an intervention is a pragmatic response, but does have costs.

“Democratising development”?

The second promise of the wellbeing approach is a kind of “democratisation” – that putting people at the centre will be empowering, that the consultations will be listened to and acted upon, that there will be some shift towards more distributed power.

There are several possible indicators of such changes in behaviour, including deliberate changes in the design process, design changes to existing and future projects, and changes in implementation. Many of these, especially the latter, will be informal and thus hard to capture.

Within TX, views vary on the extent to which the information has influenced project design or implementation, and resultant outcomes. Research for this paper had little opportunity to explore this issue with project participants and partner organisations. The principal identified example of influence on formal project design was the widespread inclusion of increased wellbeing as a high-level objective and indicator in TX projects. In addition, both TX and partner staff report there have been changes in the detailed operational planning and implementation of projects. Some feel the person-centred focus helped create opportunities to gather feedback from communities, informally influencing how some projects were delivered.

However, specific identified examples were limited, and the assessments have been weakly linked to project design processes. Moreover, it is enumerators not respondents who document survey responses, and in both, programme staff who analyse and decide. The weak linkage with project design and implementation means participants have only very indirect influence over project design and actions at best.

Nor are the cited examples of listening to project participants unique to wellbeing approaches – they will be familiar to anyone conversant with participatory development. As one type of person-centred development, a focus on wellbeing does encourage such practices. However, separating out the influence of the wellbeing approach from the wider use of participatory methods within TX is one of the challenges of identifying what difference wellbeing assessments have made. Some respondents credited wellbeing with complementing TX’s community-based, participatory ways of working; others suggested participatory approaches were already part of the organisation’s approach.

Importantly, key staff responsible for the wellbeing work were unaware of this objective of changing projects and behaviour. It was not an explicit strategic objective and was not deliberately promoted or assessed. In practice, the principle focus has been on assessing changes in participants’ wellbeing.

The concept of “democratisation” itself needs unpacking to arrive at realistic objectives, in much the same way that “participation” and “empowerment” have had to be unpacked in development practice. Interviews highlighted that the original TX wellbeing framework’s process is an enjoyable one, one that creates the space of new discussions for women and men in communities, one in which participants feel recognised and listened to, and which could spark changes to project design and practice. Some women participants were reported to have emphasised that this was the first time that they had been asked for their views on these kinds of questions.

While these are undeniably positive, these experiences of participating in the wellbeing assessments are limited to a relatively small number of project participants and occur occasionally. It is unlikely these consultations themselves have had any significant impact on their wellbeing, and they do not involve a transfer of power. The shift to a survey-based, amended approach arguably reduces the prospects of “democratisation”, removing the group-based process and discussions and reducing the potential for facilitating discussions that could be opportunities for open learning – and influencing decisions. At the same time, integrating wellbeing questions into project MEL systems may increase debate in response to findings and feed these into changes in implementation and design.

In practice, the wellbeing experiment in TX has focused on measuring rather than empowering, on proving rather than improving, and has not prioritised the element of promoting agency and voice. There may be an inherent tension between the two, or perhaps there is nothing inherently empowering about a focus on wellbeing. Like participatory approaches, the key may be “how” things are done, and with what objective, rather than “what” is done. For it to “empower” this may need to be an explicit objective – with a clear understanding of what transfers of power are both possible and desirable.

Conclusions

Traidcraft made an ambitious attempt to bring “flourishing” into its day-to-day work. The Inner Wellbeing approach was an attractive starting point for Traidcraft as a way to assess and measure wellbeing for several reasons: it is person-centred and relational, situating individual wellbeing within relationships; it is positive in its outlook; it recognises the importance of agency and participation, values, and moral frameworks in influencing wellbeing. It embodies an aspiration that the approach would encourage greater agency and participation, perhaps even empowerment of project participants. It makes use of both quantitative and qualitative research methods.

TX’s ten-year journey provides some interesting insights into operationalising a wellbeing approach. Within TX there remains a consensus that promoting wellbeing and assessing change in it is a “good thing” although there are conflicting views on wellbeing assessments’ relevance to the very poorest, how to assess it, and the right balance of costs and benefits. Organisational realities are important. Traidcraft is a relatively small NGO in terms of budget, with limited resources and scope. Staff change; strategic and operational priorities are contested and updated; funding sources, project timeframes and practical realities influence what is possible.

People – and especially those in positions of organisational power – have significantly influenced the introduction and sustaining of the wellbeing pilot in TX. Despite a declining commitment to an explicit wellbeing experiment at the corporate level, an interest in promoting wellbeing as the aim of development, and of assessing whether, how, and to what extent interventions are doing so, has remained resilient in the organisation. Organisations – and their different elements – make, disrupt, and sustain changes in policy and practice in an on-going process of creation and negotiation, responding to internal and external changes of power and priorities. TX continues to debate how to ethically and pragmatically balance understanding how wellbeing is changing for project participants with a recognition of the organisation’s limited capabilities and resource.

TX has adapted and integrated the original approach – to the point where it has two related approaches. The outcome is a continuing, partially-owned commitment to assessing how interventions contribute to changes in the wellbeing of participants and beneficiaries, and two methods for assessing wellbeing: an adapted “inner wellbeing” method and an evolving approach that has stripped away much of the framework but retains its basic conception of wellbeing as influenced by material conditions, relationships with others, and subjective interpretations of these and other contextual and cultural factors that influence our sense of wellbeing. The two methods do different things, but they remain rooted in a shared conceptualisation of wellbeing. In doing so, TX has arrived at ways of assessing wellbeing that meet its purposes and its means, in part fulfilling the first promise of wellbeing – more comprehensive information on the lived-reality of project participants, and one which encourages an appreciation of and support for change for women and men, young and old.

Achievements against the second “democratising” promise of wellbeing are more mixed. Individual’s perceptions of their own lived reality have been sought out and measured. These have influenced the decisions made by individuals in the implementation of projects. However, specific examples of direct, formal changes made to project design or implementation made in response to wellbeing assessments are limited.

Why is less clear. We believe this was due to a combination of factors: principally that this aim was not clearly articulated at the start, changes in leadership, the priorities of funders, and resource constraints. As a result, “proving” tended to be prioritised over “improving”. A focus on a single quantitative measure of wellbeing provides a summary of wellbeing which can be tracked and reported. At the same time, the reduction to a single number loses much of the meaning and specificity needed to influence project design and implementation, or communicate results. Distributing power is the harder promise to achieve. Wellbeing assessments have a potential role in supporting that promise, but do not inherently achieve it. Doing so needs to be backed by clear objectives and choices about priorities, resources, and ways of working.

Acknowledgements

We thank all those who spared their time to respond to questions and provide information, and Barbara Bompani and Neill Thin from Edinburgh University and Sarah C White and Shreya Jha, Relational Wellbeing Collaborative, and two anonymous reviewers who provided valuable comments.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Mark Adams

Mark Adams is currently a PhD student (part-time) at the University of Edinburgh and MEAL Manager at SCIAF / Caritas Scotland. He previously worked for 15 years in Sudan and Uganda for Oxfam GB, GOAL Ireland, and ACORD in various roles including Country Director. He holds an MSc in Global Development Management from the Open University, and an MPhil (research) and BA (Hons) Modern History and Politics from the University of Liverpool.

George Williams

George Williams has an MSc in Wellbeing and Human Development from the University of Bath, and a BA (Hons) in Philosophy and Literature from the University of Warwick. From 2010 he worked at Traidcraft Exchange, most recently as the Impact and Learning Manager. In this role he worked closely with Traidcraft Exchange's programme teams in South Asia and Africa, supporting the monitoring and evaluation of programmes working with small-scale producers and workers.

Notes

1 The piloting process sought to address potential biases introduced by the group-based survey method – related to, for example, interactions between respondents and interjections from others during the process, and respondent selection. Learning from the piloting process concluded skilled facilitation would mitigate these risks (Stevens 2012).