Pathways to co-impact: action research and community organising

Abstract

This article introduces the concept of ‘co-impact’ to characterise the complex and dynamic process of social and economic change generated by participatory action research (PAR). It argues that dominant models of research impact tend to see it as a linear process, based on a donor-recipient model, occurring at the end of a project following the take-up and use of findings. PAR challenges this approach, as impact is embedded in cycles of the action research process; the distinction between researchers, research informants and research users is blurred; and micro process-based impacts, including changes in the thinking and practices of co-researchers, are as significant as findings-based changes in policy and practice. A conceptual framework is developed, based on a three-fold distinction between ‘participatory’, ‘collaborative’ and ‘collective’ impact. This is applied to a case study action research project, Debt on Teesside, working with low-income households in North-east England. The project is analysed in terms of participatory impact (e.g. developing skills of participating households, mentor-researchers, and university staff); collaborative impact (e.g. findings-based changes in thinking, policies and practices of advice, community finance and housing agencies, and local authorities resulting from collaborative research); and ‘collective impact’, adapted from the field of social interventions, which involves organisations collectively targeting specific actions based on research (e.g. changing policy and practices of lenders and government relating to high-cost loans).

Keywords:

• Co-impact
• participatory impact
• collaborative impact
• collective impact
• household debt
• community organising
• participatory action research

1. Introduction

Action-oriented and participatory research is increasingly popular in academic, policy and practice environments. Reasons include increasing demands that academic research contributes to changes in policy and practice, and that policy and practice are evidence-based. Paradoxically, the features making participatory and action research relatively successful in bringing about social and economic change, also mean it is hard to identify and attribute precisely the causes and nature of the change. For processes of research and action are inextricably intertwined, distinctions between researchers and researched are sometimes unclear, and frequently many parties are involved, using emergent and unpredictable approaches.

In this type of research, an often diverse range of actors works together in a process sometimes characterised as ‘co-production’ or ‘co-creation’. Hence, as an integral part of the research process, impact is also co-produced. We call this process ‘co-impact’, an umbrella term referring to the generation of change as a result of individuals, groups and organisations working together. In this article, we are interested in the process of creating and evidencing social and economic impact through action research operating in a participatory paradigm. However, we draw on concepts and practices of impact more widely, including the hot topic of ‘collective impact’ rapidly taking off in the non-profit sector in North America. We distinguish ‘participatory’, ‘collaborative’ and ‘collective’ impact, discussing what these mean with reference to a case study action research project exploring household debt in the Teesside area of North-east England. We use data collected during the project and afterwards as part of an evaluative process to identify and describe different types of impact and assess whether the three categories are helpful.

2. Types of impact in participatory and action research

A concern with ensuring that research has a social and economic impact is high on the agenda of funders and governments across the world (Penfield et al. 2014). In the UK, a prevalent definition of research impact is that used by the research funding councils as part of the 2014 Research Excellence Framework (REF) for assessing the quality of universities’ research:

an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia. (www.hefce.ac.uk/rsrch/REFimpact/)

This concept of ‘impact’ is based on a fairly traditional model of the research process. This entails moving from identifying the topic/research questions; through research design, data collection, analysis, interpretation, presentation and dissemination of findings; to the generation of impact. This assumes a linear process, with impact created at the end as a result of the findings (Pain et al. 2015). Further, as Pain (2014) comments, one of the meanings of ‘impact’ is ‘striking a blow’. This implies an identifiable event, which can be observed, measured and evaluated: ‘a concrete visible phenomenon that one party does to another party’ (Pain et al. 2015, 4). In the context of the UK REF, the assumption is that universities do the research, which is then taken up and used by other parties.

Action research challenges this linear model, as individual, group, organisational, social and economic changes are expected to occur throughout the research process, which goes through recursive cycles as preliminary findings are fed into the system being researched, changes are made and further research undertaken (see, for example, Burns 2008; Coghlan and Brydon-Miller 2014; Kemmis, McTaggart, and Nixon 2014; Kindon, Pain, and Kesby 2007; McIntyre 2007; Reason and Bradbury 2008). Participatory research, which entails people with a stake in the issue under study being involved in carrying out aspects of the research, adds an additional dimension. Here, change may occur in individuals and organisations as a result of doing the research, regardless of the findings. We might characterise this as ‘participatory impact’. Often this is part of the rationale for participatory research – the stated aims of which frequently include empowerment and building the capacity of community-based co-researchers as part of the knowledge-making process (see, for example, Bell et al. 2012; Coghlan and Shani 2008; Fals-Borda 1988; Jones and Joseph 2012). As Fals-Borda (1987, 338–345) suggests, this occurs through techniques of collective research, critical recovery of history, valuing and applying folk knowledge, and production and diffusion of new knowledge. However, it is ‘findings-based’ impact that tends to be prioritised by funders and by the Research Excellence Framework (REF) in the UK. And in this context, participatory impact is less well-understood and valued, as it is embedded in the research process and the changes are experienced at a micro-level by individuals.

So far, following the brief for this special issue, we have referred to ‘action research in a participatory paradigm’ rather than ‘participatory action research’ (PAR). This is in recognition that not all action research is participatory in design and process, and not all participatory research is action-oriented. Nevertheless, there is usually some element of participation in action research (particularly in implementing findings), and some element of action in participatory research (as co-researchers learn and change during the process). The research project featuring in this article was specifically designed as action research, with elements of participation, as will be described in the next section. Hence, it could be categorised as PAR, and we will use this term as shorthand, while noting that action research in a participatory paradigm is a more accurate description, given the project’s action focus.

In analysing our case study project, and drawing on our other experiences of PAR (e.g. Banks et al. 2013, 2014) we felt it was useful to make a conceptual distinction between different types of impact that may be generated through PAR. However, while this conceptual distinction is helpful, in practice the different types of impact described below do overlap.

(1)	Participatory impact refers to changes in the thinking, emotions and practice of researchers and core partner organisations, which happen as a result of their involvement in conducting PAR. This may entail learning research skills, developing new insights and understandings that can be used in daily life or in community action, developing confidence, feeling empowered, or passionate about a cause, for example. In non-participatory research, researchers may learn something as part of the process of doing research, but in participatory research this is often a process of learning together. Hence, we call this kind of impact, ‘participatory impact’ and note that it is process-based.
(2)	Collaborative impact is based on the take-up and use of the findings of collaborative research by individuals and organisations to change practice and policy, and influence attitudes and culture. This may include impact on the individuals and organisations involved in the project who use findings, as well as outside individuals and organisations. In participatory research, the impact is generated by individuals and organisations working together. Hence, we call this ‘collaborative impact’, and note that it is more findings-based than ‘participatory impact’, which emphasises process.
(3)	Collective impact involves a deliberate strategy on the part of the research partners (and sometimes others) to achieve a specific, targeted change in practice and/or policy based on issues highlighted via the research. The concept of ‘collective impact’ is currently a hot topic, but is used less in relation to research and more in the context of multiple organisations working together strategically to achieve social change, where interventions are co-designed to tackle ‘wicked’ (intractable) issues, such as poverty or persistently low educational outcomes for children in a neighbourhood (see Kania and Kramer 2011). Research may be involved, but it is not the central focus. Here, however, we are using the concept of collective impact in a research context, with a slightly different meaning, as will be discussed later.

The conceptual distinction between ‘collaborative’ and ‘collective’ impact may sometimes be difficult to see in practice. However, the concept of ‘collective impact’ is useful in relation to focused PAR projects that either aim at the outset to create and collect evidence to inform specific changes in practice or policy or to identify a specific issue as a result of research linked with a change agenda shared by all parties.

We now use a case study of an action research project operating in a participatory paradigm to illustrate these different types of impact – how they can be identified, described and analysed.

3. The Debt on Teesside project

The Debt on Teesside research project was a partnership between a local community organisation working on poverty (Thrive Teesside), a national campaigning organisation (Church Action on Poverty, CAP) and Durham University’s Centre for Social Justice and Community Action. The project was co-designed as an action research project by the three partner organisations, with the aims of researching, alleviating and campaigning on the high levels of indebtedness caused by the use of high-cost credit in low-income households in the Teesside area of North-east England (Banks et al. 2013b). Thrive and Durham University had been collaborating for several years prior to putting in the funding bid (Beacon 2011). A worker from Thrive had approached Banks at the Centre for Social Justice and Community Action for assistance in organising and analysing data that Thrive had been collecting on high levels of household debt. The problem of unmanageable debt, and the predatory tactics of some credit companies, had emerged as a significant issue from earlier research by Thrive on sustainable livelihoods (Orr et al. 2006).

A joint bid for funding from the Northern Rock Foundation for a two-year action research project (2011–2013) was successful, with the University as the fund-holder and Thrive and CAP as the key partners. The University employed a part-time researcher and secretary based at Thrive’s offices, while CAP employed two part-time community organisers who worked for Thrive. The project had three elements:

(1)	detailed research on the levels and dynamics of household debt with a small number of households over time;
(2)	A mentoring scheme offered to households participating in the research;
(3)	Campaigning on key issues emerging from the research.

A total of 24 households were recruited to take part, of which 16 engaged in follow-up mentoring sessions. The reasons for not taking up mentoring varied from people feeling they did not have the time or energy, to moving house and becoming uncontactable. Volunteer mentor-researchers (some of whom had experience of problematic debt themselves) were trained to offer financial mentoring and collect relevant data on the changing financial situations of household participants. The mentors included volunteers at Thrive and employees of local advice agencies seconded by their organisations. The University researcher and Thrive community organisers undertook initial interviews with the 24 participating households to collect detailed information on their finances, attitudes towards money, and their family and life circumstances. An advisory group evaluated progress quarterly and the core team, including the ‘Principal Investigator’ from the University (Banks) and Senior Officer from CAP, met regularly. Key issues were identified for campaigns, and the idea was that some participating households might become involved in campaigns and the work of Thrive.

Thrive and CAP worked within a community organising framework. Community organising involves identifying key issues on which social and economic change is needed, and mobilising a range of diverse organisations to challenge power-holders to implement changes in policies and practices (see Alinsky 1969, 1989; Beck and Purcell 2013; Pyles 2009; Walls 2015). While all community organising entails some research to identify and tackle key issues, this project was explicitly an action research project, characterised by a community organising approach to taking action – what might be called ‘community organising-based action research’ (Banks 2015).

4. Methodology and methods of this article

In June 2016, the co-authors met to identify our individual and organisational learning from the project and developments in our respective organisations and the outside world attributable to the Debt on Teesside project. In this article, we draw on records of that meeting (Evaluation meeting, 2016). We also utilise data collected by the project about the changing nature of the participating households’ finances; evaluation of the mentoring scheme (including interviews with mentors and mentees); and records of Advisory Group meetings, workshops with households, public assemblies, dialogue and dissemination events (for more details, see Banks 2015; Banks et al. 2013a). Finally, we draw on two sets of semi-structured interviews conducted by Durham University researchers who were not part of the project. The purpose of those interviews was explicitly to collect information on the impact of the research as part of Durham University impact monitoring for the REF. The first interviews were conducted at the end of the first year of the project (July 2012) and focused largely on the benefits of community–university partnership working. We only draw on one interview here – with the CAP worker, as he was not interviewed in the later round. The second interviews were conducted in July 2014, one year after the project ended. Eight interviewees were asked what they took from the research individually, organisationally and for their clients/communities, and then asked to identify ideas/findings from the project that were useful in their work (including changes to their own and others’ thinking and practice. Interviewees comprised two Thrive project workers, two members of the advisory group (who were also members of local advice and housing agencies), two mentors seconded from local organisations and two local authority officers who had used the research.

In this article, we focus on identifying the impact of the action research using a ‘forward-tracking’ approach – i.e. starting with the research and tracing forward to investigate impact (Morton 2015, 407). In participatory research with diffuse impacts, a ‘backward-tracking’ approach (starting with a change in practice/policy and assessing what contribution the research made) is less easy. We also confine discussion to the positive impacts of the research (perceived benefits), while acknowledging that a complete impact assessment would also include negative effects. Further, our aim here is to identify the (positive) impact rather than investigate in detail how it was achieved, which would require a full contribution analysis or similar impact evaluation (see Mayne 2008; Morton 2015).

Regarding contributions to this article, Herrington, a Thrive worker, was part of the action research team and evaluated the mentoring scheme in 2013; she provided additional information on the continuing impact of the Debt on Teesside project on Thrive. Carter, a long-standing volunteer activist with Thrive, contributed to the project Advisory Group and provided information on campaigns related to the research and the ongoing use of the research data by Thrive. Banks coordinated the action research project with CAP; she also coordinated this impact study, collating and analysing the data, developing the three-fold impact framework and drafting the article. The authors have worked together over several years and wrote the final project report and other materials together (Banks et al. 2013a). Indeed, Carter and other Thrive volunteers produced a toolkit for community organisations working with universities based on challenging experiences in previous research projects (Beacon 2012) and contributed to an article about using the process of co-inquiry to study co-inquiry (Banks et al. 2014). Hence, we understood and valued our diverse backgrounds and experiences, and differences in interpretation of the significance of the research had already been worked through. In this article, therefore, rather than reflecting on the nature of our relationships with each other, we will focus on identifying and describing the agreed impacts of the research. This includes the participatory impact on ourselves as researchers, activists and co-writers and inevitably involved an ongoing process of collaborative reflexivity, as we placed ourselves in the picture that we were trying to evaluate.

Based on the data described above, we now analyse the Debt on Teesside project in terms of the different types of impact: participatory, collaborative and collective.

5. Participatory impact

As noted earlier, an explicit aim of participatory research is to develop skills, confidence and solidarity among community-based co-researchers through doing research together. Literature on participatory research refers to concepts such as ‘consciousness-raising’ or ‘conscientization’, inspired by radical and liberatory approaches (e.g. Freire 1972, 2001) aiming to empower oppressed and marginalised people to question dominant ideas and interests and mobilise indigenous knowledge for their own benefit (Fals-Borda and Rahman 1991; Kindon, Pain, and Kesby 2007; Reason 1994, 47–48). Other reasons to encourage people with direct experience of an issue to research it, include improving relevance in research design and gaining broader and deeper reach in collecting data from individuals and groups who may be suspicious of outside researchers (Durham Community Research Team 2011; Minkler and Wallerstein 2008). Hence, considerable learning can also be achieved by ‘professional’ or academic co-researchers involved in participatory research projects alongside community-based co-researchers.

Given participatory impact is about the effect on people and organisations of taking part in research, we structure our discussion of the nature and scope of participatory impact in terms of different categories of participants in the Debt on Teesside project: (1) households; (2) core research partners; (3) community mentors and researchers; and (4) advisory group members and organisations.

5.1. Participatory impact: households

The 24 households participating in the project played a variety of roles, which changed over time. Here, we will describe three main roles and the impact associated with each role: (a) research informants; (b) participants in the mentoring scheme; (c) co-researchers and activists.

5.1.1. Impact on household members as research informants

As in much qualitative research, the process of responding to questions from an interviewer may trigger a process of reevaluation or a desire to make changes in interviewees’ lives. In this research, households not only provided data on financial and other aspects of their lives, including attitudes towards money and debt, but were also offered mentoring. Hence, some households were already thinking about possible actions to take by the end of the initial interview. As one participant commented (Interview, 2013):

It’s the first time I have ever spent any time thinking about my money and what I spend it on.

5.1.2. Impact on household members as participants in the mentoring scheme

Sixteen households participated in the mentoring scheme, undertaking between one and 11 follow-up sessions with a mentor. This involved key contacts from the households working with a mentor to assess the state of their finances and plan about how to reduce indebtedness. The mentor’s role was to support participants to make changes in their lives for themselves. Interviews with six households conducted by the university researcher at the end of the mentoring scheme indicated that for some people the mentoring had a positive impact (Banks et al. 2013a, 42–51). Participants reported:

•	greater understanding of how to manage their money;
•	changes in attitudes and behaviour towards money management;
•	improved self-confidence overall.

As two participants commented (Interviews 2013):

Before I came on this project, I owed quite a lot of money - three mobile phone contracts, a gym membership, Littlewoods and loads of other stuff. My mentor helped me cancel my gym membership and we worked out how to pay the other stuff … I’m not rich now, but I do manage a bit better.

It is the first time I have ever thought about money and its value. I used to spend and just buy for the sake of buying. I don’t do that anymore … in fact I am dead tight now. I won’t buy anything unless I really need it. I never waste money like I used to.

5.1.3. Impact on household members as co-researchers and activists

For most participating households, simply working on their own finances was hard enough work. However, five people became more involved in the project, including:

•	co-writing case studies for a final celebratory learning event.
•	Speaking at public assemblies and events. For example, several participants gave testimony at a Thrive public assembly designed to raise awareness of the effects of predatory lending and at a celebratory learning event.
•	Undertaking community organising training – one participant became a Thrive volunteer, undertaking community organising training with CAP in Manchester.
•	Involvement in campaigns, for example, distributing ‘No uninvited traders’ notices to stop cold calling by ‘doorstep lenders’ and making a film for a campaign on affordability and data-sharing.

One of the participants, who acted in the campaign film (shot in his house), commented (Interview, 2013):

I am coming out of my shell a bit more … I knew I needed to open myself up a bit … I got into Thrive and it was like ‘hang on I am doing something now’.

5.2. Participatory impact: core research partners

We now describe the impact on the core research partners: (a) Thrive; (b) CAP; and (c) Durham University.

5.2.1. Impact on Thrive

One of Thrive’s staff (Herrington, co-author of this article) also acted as mentor, and the other was involved in supporting some of the households to get involved in campaigns and the public assembly. The ‘learning by doing’ experienced by the Thrive workers is hard to separate out from the ‘learning from findings’ as both workers were actively involved in ongoing evaluation of the project. The biggest lessons came from the struggle to engage and retain households in the project and support them to change borrowing patterns. Working on the project changed their thinking about what was possible. Herrington (Interview, 2014) realised it was impossible to do financial mentoring on its own, as finances are so entangled with other aspects of people’s lives:

I now look at people in a more holistic way – I can see you can’t do it in a short period of time.

She spoke about realising the need to ‘push’ as well as ‘empower’ people. This learning was used in further work undertaken by Thrive. Reflecting two years later, she remarked about the Debt on Teesside project (Evaluation meeting, 2016) that it ‘adds to our confidence’ and was:

an experience you went through. You don’t forget the lessons you learnt. It helps you move forward.

5.2.2. Impact on CAP

CAP was less involved in the day-to-day action research process, as the staff member was based in Manchester, visiting the project every few months. However, insights from the project fed into CAP’s work on financial exclusion, providing CAP with the opportunity to reflect and take stock of this area of work. As the CAP worker (Interview, 2012) commented:

I think one of the problems for people in positions like our’s is that because we are often so caught up in the delivery and there’s usually not enough resources really to do that well … The research element and the documenting stuff, the creation of time for learning, to evaluate and analyse and learn and then to write all that up, I mean to have that done professionally like that in a project is obviously – it’s huge, a huge benefit.

5.2.3. Impact on Durham University

The Research Associate left before the end of the project, and no follow-up was undertaken with her. The Principal Investigator, Banks (co-author of this article) reflected on the impact on her and the University of collaborating on the research project (Evaluation meeting, 2016):

I learnt a lot about the complexities and pace of collaborative research, and the fact that there is never enough time or money to do the work as well as you’d like. I learnt a lot more about the ethics of partnership working as we encountered numerous day-to-day dilemmas and conflicts during the research.

A significant learning experience for the Principal Investigator and other University staff and students was participating in training in community organising offered by Thrive. This helped develop skills in communication and strategic campaigning. As Banks commented (Evaluation meeting, 2016):

I became involved in the campaigns and the public assemblies that were an integral part of the community organising work. Learning how the process worked was invaluable and has been useful in the teaching I do at the University about community work.

5.3. Participatory impact: community mentors/researchers

The mentors comprised existing and new volunteers with Thrive, one member of Thrive staff and two staff seconded from organisations on the project advisory group: Stockton and District Advice and Information Service (SDAIS) and Fabrick (a housing provider).

The volunteers learned new skills in offering mentoring support and this role helped consolidate their links with Thrive and involvement with other work. The seconded staff reported particularly helpful learning, as the longer-term mentoring role was very different from their customary one-off advice giving. The SDAIS worker commented (Interview, 2014):

Seeing people in their homes has a real impact … Being involved in the project made me see the full picture … it changed the way I think …

He also remarked that the debt work normally offered by his organisation does not provide an ‘holistic service’ (working on debt issues in the context of many other issues in people’s lives), which is clearly needed.

The Fabrick Housing worker (Interview, 2014) reported similar experiences, as he was used to giving ‘one-hit advice’ in his work, which tended to be reactive, when people were in crisis. He saw the Debt on Teesside model as another way of delivering advice, demonstrating the importance of regular contact and keeping people interested and engaged.

5.4. Participatory impact: advisory group members and organisations

The project had a quarterly advisory group, comprising local advice and housing agencies, a credit union, a community development finance institution, academics with relevant research interests from Durham and two other universities and latterly two mentors. Consideration of progress reports, including difficulties in recruitment and retention of household participants and mentors, led to discussion about the nature of the problems households were experiencing, how mentoring support could best be offered and what issues were important for campaigns. This fed into the daily work of the advisory group members in their own organisations – with the most striking insights arising from the discussion of the multiple issues faced by the participating households and working out how mentoring support could be effectively delivered. As the chief officer of SDAIS commented (Interview, 2014):

We were challenged and it made me question what we do. It made me think about doing things differently. I came away from the meetings enthused. [We] talked about what they were finding, and getting access to the results data as it was going on was really helpful. Getting feedback and criticism was like a cold shower – it helped to reflect on practice.

6. Collaborative impact

Collaborative impact refers to the effects of the findings of research that has been conducted collaboratively on both internal and external people and organisations. It fits most easily with traditional conceptions of impact, which are usually findings-based, but acknowledges that in PAR the impact is co-generated. In the case of Debt on Teesside, this involved key participating actors listed in the previous section, as well as outside organisations that took up and used the research findings – including local authorities and third sector organisations.

Morton and Flemming (2013), in a case study exploring the impact of a participatory research project, describe three elements of the research impact process:

Research uptake: people are interested in research, read it, talk about it, come to a presentation, etc.

Research use: people do something with the research, change their view, pass it on to someone else, apply it to practice or policy.

Research impact: a contribution to change as a result of research use.

In practice, these elements are hard to distinguish. However, the difference between ‘uptake’ and ‘use’ is important to bear in mind, as there is a tendency to conflate ‘uptake’ (which will usually accompany a dissemination event) with more active and deliberate use of research findings. The distinction between ‘use’ and ‘impact’ is more challenging. With the exception of passing the research on to someone else, arguably the other descriptions of research use would be considered as research impact according to many current definitions. The distinction between use and impact may be more a question of the degree or significance of the change that occurs, but ‘changing views’ or ‘applying’ research surely should count as impact, if impact is defined as ‘a contribution to change?’ Some definitions of impact are less demanding than the one offered by Morton and Flemming, including that currently in use in the UK REF. It is common in the field of evaluation to distinguish ‘outcomes’ from ‘impact’. ‘Impact’ is sometimes used to refer to longer term, sustained change. This, of course, is harder to track, measure and attribute than shorter-term outcomes.

While it is easy to become mired in definitional niceties, the point of this discussion is to encourage us to think about what exactly we are claiming when we say our research has had an ‘impact’. Arguably many claims for impact are for research use in Morton’s terms, and may be more about outcomes than longer-term change. We are using this broader, inclusive conception of impact here (which includes outcomes), and will consider two types: changes in people’s thinking and understanding; and changes in policy and practice. Arguably changes in policy and practice are based on changes in people’s thinking, but sometimes it is impossible to evidence changed thinking per se (a changed policy may be an indicator of changed thinking), and sometimes changed thinking may not translate obviously into changed practice/policy (Upton, Vallance, and Goddard 2014, 362)

6.1. Collaborative impact: changes in thinking and attitudes

Arguably the research findings about factors influencing the use of high cost credit by low-income households had a significant impact on the thinking and attitudes of key stakeholders in the project. One of the most articulate was the Chief Executive of SDAIS, an active member of the Advisory Group (Interview, 2014):

The report demonstrated how difficult it is for people to change. Change has to come through preventative, intensive work … mentoring may not be viable as is, but bits can be taken from the approach, for example, maintaining long term relationships with people to make the change - workers have an ongoing caseload – not just closing the case when they walk out of the building. We need to help people to change and not to fall back into bad habits. Debt is a long-term issue, so they know that it is going to be a long-term commitment, they should think of keeping the contact open.

The research findings also influenced the thinking of Thrive workers about what kinds of mentoring schemes would be viable. Herrington commented (Interview, 2014):

With hindsight it is impossible to do financial mentoring on its own – it’s not a priority. This informed future work, which we developed with the Volunteer Bureau to give more time to work across the whole.

Another Thrive worker remarked (Interview, 2014):

There were good outcomes from the mentoring, but big attrition … The lessons learnt from this are that it is hard to work with some households.

I wanted to go in and fix things. I understand more about the choices people make now, and why. I am asking more questions now.

For another local organisation, a housing provider, the representative on the advisory group commented that the project helped crystallise the way she thinks (Interview, 2014): ‘[it]made it more real – real case studies about real households’. Arguably this was one of the key features of the project: that it offered detailed insights into the realities of people’s lives, including where debt fitted into the bigger and complex picture. This was useful to external agencies that were not directly involved in the project, but were aware of, and used, the findings.

During the project, and after the end, several events were held to raise awareness of emerging issues and disseminate findings and recommendations. Middlesbrough and Stockton Councils invited the project team to give presentations, and Council representatives attended project events. The Head of Policy, Improvement and Engagement at Stockton Borough Council asked for anonymised data on household circumstances as the Council wanted to use real data to piece together issues facing families, feeding into a workshop on the Social Fund. This Council Officer commented (Interview, 2014):

We wanted to look at the trigger points of crisis. The data was really interesting, showing how lots of people do not have resilience to debt. There is a need to do something quickly, need to get money into the families to prevent debt problems. It was also very apparent from the data how overwhelming the experience of debt is. Suicide needs to be prevented, people are very emotionally involved in their debt.

The Council’s Head of Housing and Community Protection also remarked (Interview, 2014) that the research:

Showed the power of having time to spend with individuals and that the reality of achieving this is very difficult.

It is impossible to track impact on the many others who attended events or downloaded materials. People completing evaluation forms following events sometimes predict how research findings or materials might be used. For example, after an event at Durham University on credit unions in June 2015, some delegates stated they would be prepared to: ‘be involved through sharing ideas and learning with colleagues from across the UK’; ‘facilitate a link up with churches to develop volunteer networks’; ‘promote financial and digital inclusion and support’ within their organisation and partner organisations; and work to ‘advocate further development of understandings between Credit Unions and CDFIs [Community Development Finance Institutions]’ (Banks, Tischer, and Hall 2015). These predictions may or may not have materialised. Equally, evaluation forms filled in immediately after an event cannot necessarily capture the changes in thinking that need time to percolate over the following weeks or months.

6.2. Collaborative impact: changes in policy and practice

We now describe the changes in policy and practice by (a) the core research partners (Thrive, CAP and the University); (b) Stockton and Middlesbrough Councils; (c) SDAIS; and (d) other organisations.

6.1.1. Changing policy and practice: Thrive, CAP and the University

Based on the research findings, Thrive applied for grants and made bids for contracts to undertake new community-based interventions in the Teesside area relating to social and financial exclusion. Thrive received grants from the Esmeé Fairbairn Trust for four years to work on civic engagement and from Comic Relief (two years) to work with people experiencing sanctions imposed by local job centres (withdrawal of welfare benefits from people out of work). Thrive was funded by Stockton and Middlesbrough Councils to do work on the impact of welfare reform (changes to the benefits systems) and undertake mentoring with people experiencing financial difficulties. Thrive is also exploring the potential of a Poverty Truth Commission in Stockton Borough, and is undertaking one-to-one community-based money mentoring based on an adapted version of the mentoring scheme developed by the Debt on Teesside project.

CAP used some of the stories from the Teesside households in its campaigns on financial exclusion and the film made by the Debt on Teesside project was part of the CAP ‘Drowning in Debt’ campaign in 2013. Although this was a theme on which CAP was already working, the findings from the project strengthened the campaign.

The Debt on Teesside work took place at the same time as the Centre for Social Justice and Community Action was developing at Durham University. Staff and volunteers from Thrive participated in events promoting PAR at the University, co-presented at conferences and contributed to raising the profile of action research for social justice at the University. Thrive and the Centre won a Durham University Research Impact Award in 2014. The work of the project became known about by senior members of the University responsible for research and policy-related impact, and was used as a case example for briefing the Vice Chancellor. Following links made at a presentation on the Debt on Teesside research by Banks at an event on churches and credit unions, a new action research project on Christian responses to debt and financial exclusion was designed by a colleague, Andrew Orton (see Barclay, Orton, and Stepanova 2016) and funded by the Leech Trust (2015–16).

6.1.2. Changing policy and practice: Stockton and Middlesbrough Councils

During 2014, Debt on Teesside findings were used in the work of the Welfare Reform Board of Stockton Borough Council. According to the Head of Policy, Improvement and Engagement (Interview, 2014):

The Welfare Reform Board has made practical changes on the basis of the Debt on Teesside research. There is an investment fund (£250k) to invest in voluntary and community sector services. It funds pilots and central costs – Thrive received money through this. So the local authority is continuing to fund mentoring through Thrive – we want Thrive to provide us with the intelligence to inform our practices.

Stockton Council also undertook intensive work on the benefits cap (limits imposed by national government on levels of welfare benefits payments), described as:

A ‘deep dive’ to look at what is happening in people’s lives around welfare and debt. This has, in part, drawn on some aspects of the Debt on Teesside project.

As mentioned earlier, Stockton Council used anonymised data from the 24 households in the project to look at the trigger points of crisis. The Head of Policy, Improvement and Engagement commented (Interview, 2014):

The data gave the intelligence to provide alternatives. We could support the Credit Union to expand capacity, for example. We blocked Wonga et al. from publicly available PCs and trained staff to signpost to the Credit Union.

Middlesbrough Council also blocked advertising of payday loans on council computers and following a Debt on Teesside policy and practice roundtable event in June 2014, followed up with Thrive on a programme of work linked to briefing papers on building financially resilient communities and the use of money mentoring (Centre for Social Justice and Community Action, Centre for Responsible Credit, and Thrive Teesside 2014; Centre for Social Justice and Community Action, and Thrive Teesside 2014).

6.1.3. Changing policy and practice: SDAIS

As already noted, the Chief Officer of SDAIS was greatly influenced by the Debt on Teesside project and commented that the findings made a ‘fundamental impact on the service set up around the benefits cap’ (Interview, 2014):

We are providing a benefits cap worker (through a council initiative) who works with 50/60 clients (rather than 500/600). Debt is part of their work. This has been developed partly on the basis of the research. Here we are taking the service out to the clients. This is a fundamental shift, to go to the clients’ houses. This has been really positive. The project was funded for one year but is now being extended and we are now looking at this model for other areas of work. The shift to long-term outcomes for clients had been thought about for a while, the research is helping to realise this.

People will only change if there is an urgent need to – something that came out of the report too. So the cycle needs to be broken.

Lots of other reports don’t stay in my in-tray but this has. The positivity of the advisory group was really important.

6.1.4. Changing policy and practice: other organisations

Many other organisations used some of the findings and policy and practice briefings, but we have not been able to follow these up. Examples include: Northern Money and a consortium of credit unions quoted the findings of the Debt on Teesside report in a funding bid; Gentoo (housing provider) used the materials on mentoring to develop their own scheme; Yale University Office of Sustainability (USA) and Monkey Money Mentors (County Durham, UK) adapted the money mentoring materials for their own use; the Institute of Money Advisers included a link to the mentoring materials on their website; the Chief Executive of the Chartered Institute of Payroll Professionals requested the money mentoring leaflet for distribution to members.

7. Collective impact

As noted already, we are using the concept of ‘collective impact’ in relation to research slightly differently from how it is used in the context of planned social interventions. In the latter context, the term is used to describe a framework for community change, which occurs when:

organizations from different sectors agree to solve a specific social problem using a common agenda, aligning their efforts, and using common measures of success.(www.fsg.org/ideas-in-action/collective-impact)

This entails organisations working together with a common agenda, shared measurement, mutually reinforcing activities, continuous communications and backbone infrastructure (Kania and Kramer 2011).

The concept and practice of collective impact is not without criticism, including the focus on large scale agencies and organisations working together to create change that they have defined as necessary or desirable (Wolff 2016). The participation of people affected by the issues is often not foregrounded in this conception of collective impact, which can turn into a targeted drive for change determined and defined by powerful individuals and organisations.

In PAR, the idea of collective impact has potential relevance as PAR aims to generate change, achieved by co-researchers and stakeholder organisations together. But the concept needs modification to fit a research context, and participatory research in particular. Although some action research may define the impact desired at the start of a project (if the research entails collecting evidence about an already-defined problem), much research is more open-ended, and precise details of impact cannot be predicted. Similarly, the idea of organisations aligning their core purposes and goals at the start of a PAR project is less likely. However, the idea of impact that is specifically planned and targeted by a group of individuals and organisations is a useful one. While, PAR is an emergent process, often unpredictable, with impact that may be ‘serendipitous’ (Pain et al. 2015, 7), it is also possible, and often important, deliberately to plan impact.

In the case of Debt on Teesside, the motivation for the research was to:

•	achieve change by collecting evidence of how and why people get into debt;
•	see if the attitudes and behaviour of indebted households might change as a result of mentoring;
•	change the unethical and exploitative practices of loan companies.

The mentoring scheme was not based on a concept of collective impact, although arguably it became more of a collective effort as organisational members of the advisory group supplemented the volunteer mentors recruited by Thrive with their paid staff. The campaigns were, of course, based on a model of collective action – drawing on a community organising approach involving: cutting (carefully selecting and framing) an issue; mobilising partner organisations with an interest in the issue; and planning and implementing targeted action for change. Two significant issues were pursued during the course of the Debt on Teesside project.

(1)

The reform of rent-to-own companies. This was led by CAP and Thrive with the Centre for Responsible Credit, focusing particularly on ensuring a more transparent lending process and securing better terms for long-standing customers. Following a series of roundtables, a voluntary code of practice was agreed by the companies in 2012 (Gibbons 2012). Members of several households involved in the debt research gave testimony at meetings, including Carter, and data from Teesside households were used to show the impact of high-cost credit on families in the area.

(2)

The affordability campaign. This was led by Thrive and CAP, designed to ensure that lenders check whether borrowers can afford loans, and encourage data-sharing between companies. Thrive provided data to CAP and the project made a film (‘Loadsadebt’) featuring several project participants, depicting a representative of a lending company stuffing money down the throat of a potential borrower (Debt on Teesside 2013). This linked with a broader national campaign, promoted by a large network of organisations (including the Anglican Church) designed to outlaw, or at least change significantly, ‘pay-day loans’ in 2013 (‘Stop the payday loan ripoff’), which eventually resulted in the Financial Conduct Authority taking action to regulate payday lenders by law in 2014 (www.fca.org.uk/news/ps14-16-detailed-rules-on-the-price-cap-on-high-cost-short-term-credit).

Both campaigns involved some of the households with experience of debt working alongside national organisations committed to combatting poverty and financial exclusion. The organisations did not align their missions, or develop backbone infrastructure, as would fit the model of collective impact developed by Kania and Kramer (2011), but they did focus together on an issue with a view to achieving change.

In community-based research we can learn from the collective impact movement about how to work together to achieve significant impact (see the Collective Impact Forum, www.fsg.org/collective-impact-forum). However, it is important to recognise that the concept and practice has some limitations. It needs to be modified in participatory research to include smaller organisations and individuals with direct experience of the issues under scrutiny, in order to ensure that the targeted change is not determined solely by powerful individuals and large organisations. The key role played by Thrive and the participating households and mentors in the Debt on Teesside project ensured that the actions taken and planned impact were not dominated by CAP or the University.

8. Concluding comments

This article has presented a three-fold model of co-impact, developed and illustrated with reference to a case study action research project. Our aim was not to present an impact evaluation of Debt on Teesside, but rather to show what kinds of individual, organisational and social changes were generated in this particular case, and what conceptual framework might be useful for organising and understanding co-impact.

The well-known complexities of PAR are evident, confirming the position outlined at the start of the article that research impact comprises more than simply take-up or use of findings; it includes micro-impacts, such as changes in the thinking and practices of co-researchers (participatory impact). Collaborative impact (change based on findings of collaborative research) is likely to be the main type of impact recognised and tracked for funders and for the UK REF. But this is a complex and dynamic process, as linear, donor-recipient models do not work when impact is embedded in cycles of action-research and distinctions between researchers, research informants and research users are blurred. In PAR research users may also be co-researchers and research participants, as evidenced by some members of participating households in the Debt on Teesside project. Hence, the use of the term ‘collaborative’ is important, signifying that the impact was achieved by people/groups working together and the attribution of cause or ownership may not only be impossible, but also unethical. Finally, we have highlighted the usefulness of the concept of ‘collective impact’ for PAR, which is particularly relevant in the case of Debt on Teesside as a community organising-based action research project, but is also relevant to other action research projects. However, we noted the tendency in current literature on collective impact to focus on social interventions determined by large multi-agency groups. Translating the concept into a PAR context requires a greater degree of power-sharing and commitment to respecting diversity.

The next step in this process of identifying co-impact would be to incorporate it into a framework for impact evaluation in a participatory research context. This would entail drawing on existing models of participatory impact evaluation (e.g. Guijt 2014), using the threefold framework outlined here to enable participatory action researchers to incorporate ongoing impact evaluation into their research.

Acknowledgements

We are grateful to everyone who has contributed to the information on which this article is based, including the households and mentors participating in the Debt on Teesside project, the Advisory Group, fellow members of the core action research team (Jan Flaherty, Greg Brown and Mark Waters), others who gave interviews about the impact of the project, and Carole Devanny and Richard Wistow for conducting additional interviews.

Disclosure statement

No potential conflict of interest was reported by the authors.

Funding

The Debt on Teesside project was funded by the Northern Rock Foundation, with additional funding from Durham University’s Wolfson Research Institute and Research Seedcorn Fund. The research on impact was supported by Durham University’s Participatory Research Hub, funded by the Economic and Social Research Council Impact Acceleration Account [grant number ES/M500586/1].