Exploring the boundaries of ‘good’ Participatory Action Research in times of increasing popularity: dealing with constraints in local policy for digital inclusion

ABSTRACT

The popularity of Participatory Action Research (PAR) increases the risk of tokenism and blurring the boundaries of what might be considered ‘good’ PAR. This became a pressing issue when we were invited by the City of Amsterdam to conduct PAR on digital inequality with vulnerable citizens in Amsterdam, within serious constraints of time and budget. We decided to take up the challenge to offer citizens an opportunity to share their needs. This paper aims to increase the transparency of the complex reality of a PAR process in order to help new researchers learn about the challenges of PAR in real-life situations, and to open up the discussion on the quality and boundaries of PAR. Though we managed to implement some core ethical principles of PAR in this project, two were particularly under pressure: democratic participation and collective action. These jeopardized collective learning and might unintentionally feed stereotypes regarding people’s capabilities. Nevertheless, this small and local study did manage to create ripples for change.

KEYWORDS:

• Participatory Action Research
• Evaluation
• Policy
• Digital inclusion
• Digital inequality
• Ethics

Introduction

The worldview of PAR, which gained much of its impetus in the 1940s from marginalized communities in low- and middle-income countries who were vulnerable to exploitation, is based on the notion that reality is not an objective truth to be discovered but ‘includes the way in which the people involved with facts perceive them’ (Freire 1982, 30). One of the core values of PAR is that knowledge is co-produced with a community; people working together to examine a problem with the goal of addressing it for the better (International Collaboration for Participatory Health Research (ICPHR) 2013; Abma et al. 2019; Reason and Bradbury 2001). The idea that this approach is more likely to have a broad social impact fits well with the increasing concern, high on the agenda of funders and governments across the world, to ensure that research has a social and economic impact (Abma et al. 2017; Penfield et al. 2014). It is increasingly acknowledged, for example, that clinical and health service research failed to translate research into practice and policy (Grimshaw et al. 2012), did not always meet people’s needs, nor do justice to their daily reality (Chalmers et al. 2014; Ioannidis et al. 2014). PAR is also considered to respond to increasing complexity; the need for multiple perspectives and brief cyclical learning processes (Ter Haar 2014). Finally, the approach fits the call for knowledge democratisation, as citizens and patients increasingly demand to have a say in policy and research affecting them (Anderson 2017; Stirling 2008), and for contextual knowledge. Moreover, it acknowledges that people are experts about their own life.

Knowledge democratisation and social impact have become important dimensions in how research is funded and evaluated (Cook et al. 2017). This is the case for funding agencies in the Netherlands; in the past, these agencies asked the researcher to simply tick a box regarding whether they would or would not include citizens/patients. Nowadays, researchers need to explain how they are going to involve the public or patients in their research and, more recently, how they involved stakeholders in writing the research proposal (ZonMw 2006). This movement places further pressure on researchers – already dealing with the stress arising from limited funding opportunities (Halffman and Radder 2015). Not already acquainted with the approach, they quickly need to educate themselves in the principles of PAR and how to enact them.

This upsurge of interest in PAR also means that it will be stretched to fields and institutes that are not yet acquainted with the values and the necessary conditions for it to flourish, such as clinics and local governments. In other words, there is a risk of tokenism and of blurring the boundaries of what might be considered ‘good’ PAR. The increasing popularity of PAR challenges inexperienced but also more experienced PAR researchers. There is recognition that some research that calls itself participatory does not align with the core principles and basic philosophical assumptions, motivations and practices that PAR practitioners would recognise (Springett 2017). Moreover, there is evidence that PAR is not necessarily a good thing in itself and has even caused some major harm, creating mistrust and participation fatigue (Cooke and Kothari 2004; Banks et al. 2013).

PAR scholars place the assessment of quality and critical reflection high on the agenda (Dedding and Slager 2013; International Collaboration for Participatory Health Research (ICPHR) 2013; Cook et al. 2017; Abma et al. 2019; Banks and Brydon-Miller 2019; Abma et al. 2017; Lennette et al. 2019). These scholars point to the ideal and rhetoric of participation and call for creative solutions, asking PAR practitioners to engage in self-analysis of the tensions they encounter in conducting democratic collaborative research. There is a need to reflect critically on questions such as whether participation is always good, and always necessary. When is it good, or good enough? Does it always need to be small, local and long term, or could it also be done in a short timespan with a bigger group, for instance with the help of social media? And if participation is based on values, and is not a method, what do these values look like in practice?

The often-reported lack of extensive case description makes it hard for new PAR researchers to become acquainted with the core values of PAR and unsure of how to put them into practice, but also limits the conceptualisation of PAR processes, its impacts and quality (Springett 2017). As Cook (2012) argues, impact is embedded in the process, which means it is not always easy to distinguish among and articulate its different forms. Moreover, scientific journals outline how researchers should report their scholarly research, set strict word limits, and work with peer reviewers who may not always see the value of extensive case descriptions. This can result in a discrepancy between what PAR researchers actually do and how they report what they do. Furthermore, in line with PAR, researchers prefer local impact and also need to please local funders and so often produce grey literature, which is harder to find in scientific databases (Jagosh et al. 2012).

The above critical reflections became pressing when we were asked by the City of Amsterdam to conduct a PAR process on digital inequality among citizens in order to feed into the upcoming policy for digital inclusion. Due to the political timeframe, the results needed to be available in seven months. As an experienced (PAR) researcher (CD) and a new scholar in the field (NG) we immediately knew that involving ‘citizens of Amsterdam’ in seven months, with funding for almost one full-time equivalent (FTE) researcher, would be ‘challenging’. Meaningful participation would be hard to achieve. The municipality, however, could not offer more time or money; their staff were curious to see what new insights participation would bring to their understanding, but could not yet sell the ‘new’ approach to higher levels to free up more funds. Above all, they had to inform the alderman about possible policy implications and priorities for action within the scheduled time.

This posed a dilemma; not accepting the assignment would mean that policy-makers would continue to take important decisions for citizens in vulnerable situations, while accepting the opportunity might lead to tokenism, devaluation of an approach that is still critically viewed for being less scientific (Cook 2012) and might harm our personal careers as scientists. After careful deliberation, we decided to take up the challenge in order a) to give citizens an opportunity to share their experiences and needs and b) to make policy-makers more aware of the value of participation and the often complicated situations in which people live. After the project was finished, we received some critical questions about the generalizability of the findings, but the report was largely well received. The findings attracted attention at higher policy levels, and helped to set a new national policy on digital inclusion. Nevertheless, the question remained for us of whether we should ever again try to do PAR under such constraints. Therefore, we decided to reflect on our own experiences in PAR process: to what extent did we manage to put (some of) the core ethical principles of PAR into practice and actually contribute to change? We invited two ‘critical’ friends in the writing process (Kember et al. 1997), namely PAR researchers (JB and TA), who have extensive professional and experiential knowledge, to critically reflect on our process and lessons learned.

This paper aims to increase the transparency of the complex reality of a PAR process in order to help new researchers learn about the messy process of PAR in real-life situations, the often problematic and contradictory relationships and the prerequisites, and at the same time to open up discussion about the quality and stretching boundaries of PAR. Before we share our experiences in a PAR project under restraining conditions, we first shortly summarise the basic principles of PAR and provide some context for the case.

Short summary of the basic principles of PAR

Participatory Action Research builds on a long and diverse history drawn from a range of disciplines, each of which has informed and developed its own understanding of what PAR is. Fundamental to PAR is a ‘shared recognition that science is more than adherence to specific epistemological or methodological criteria, but is rather a means for generating knowledge to improve people’s lives’ (International Collaboration for Participatory Health Research (ICPHR) 2013, 5). The aim of involving multiple perspectives and different forms of knowledge (experiential, practical, traditional and scientific) is to develop a richer and more meaningful portrayal of the subjects of research, and to facilitate a learning platform for representing the multiple experiences, hopes, and fears of those directly involved as route to reducing (health) inequalities (Abma et al. 2019). While this will depend on the focus of the study, stemming from an emancipatory movement the focus is on the poor and marginalized in society, who seldom have a voice in defining the problem and solutions. The focus is on shared learning and action for social change (Cornwall and Jewkes 1995; Khanlou and Peter 2005; Abma et al. 2019), within the local context. It is not a ‘quick fix’ and it does not follow a linear and predictable process. On the contrary, its endpoint is often open (Piggot-Irvine, Rowe, and Ferkins 2015).

We used the key ethical principles of PAR, as defined by the International Collaboration for Participatory Health Research (ICPHR), to reflect on the process and the outcomes. These principles, summarised in Table 1, do not suggest how to act, but need to be interpreted in the light of particular circumstances, and can be considered as being on a continuum, an ideal for which to strive – the ICPHR emphasis on seeking maximum participation and collective action based on joint insights (International Collaboration for Participatory Health Research (ICPHR) 2013).

Table 1. The seven underpinning values and ethical principles of PHR, as defined by ICPHR (2013)

Mutual respect	Developing research relationships based on mutual respect.
Equality and inclusion	Encouraging and enabling people from a range of backgrounds and identities (e.g. ethnicity, faith, class, education, sex, sexual orientation, (dis)ability, age) to lead, design and take part in the research,
Democratic participation	Encouraging and enabling all participants to contribute meaningfully to decision-making and other aspects of the research process according to skills, interests and collective needs.
Active learning	Seeing research collaboration and the process of research as providing opportunities to learn from each other
Making a difference	Promoting research that creates positive change for communities of place, interest or identity.
Collective action	Individuals and groups working together to achieve change.
Personal integrity	Participants behaving reliably, honestly and in a transparent and trustworthy fashion.

Methods: a case study concerning the development of a policy agenda for digital inclusion

The project was funded by the poverty director at the City of Amsterdam. Based on a former project with children living in poverty (Sarti et al. 2017; Sarti, Schalkers, and Dedding 2015), the first author (CD) was invited to attend a meeting in which choices would be made on how to accommodate the needs of children in relation to digital inclusion. A group of professionals attended the meeting. Many had direct experience with the so-called ‘target group’, and shared ideas on how best to spend the available money. Among others, the idea was to develop educational apps for toddlers and to expand the current free laptop arrangement to young children.¹ Practical questions of what kind of devices children actually need could not be answered, either in relation to school or to their social lives, which was also acknowledged as important. Therefore, the first author along with an industrial designer who had worked with the municipality in the field of poverty reduction, were invited to collaborate in a ‘participatory action study’. While we repeatedly stressed that for participatory and quality purposes we needed to make choices in the so-called ‘target group’, this was not possible. On the contrary, the project design rapidly expanded to not only children, but also to the experiences and needs of their parents; numbers were considered important in order to get the conclusions accepted by the policy-makers. The resulting findings should provide an overview of 1) the problem and needs of families with a low socioeconomic position (SEP) in relation to the quickly digitalizing society; and 2) concrete, and preferably swift, operational solutions.

Digital inequality is a complex social problem that transcends several policy domains, in this case poverty reduction, education and welfare, spanning the entire political spectrum, each party with its own – often conflicting – views and agenda in relation to felt responsibility, need and kind of action. The upcoming elections, 11 months after the start of the project, increased the pressure to profile themselves well – who wants to own a complex problem that defies simple solutions in an election period? This made it even more important to present a rewarding perspective for all policy-makers involved.

The budget covered for both senior and a junior researcher (together 0.9 FTE), research materials and some money to reimburse participants for their time. We also pushed for financial support to create an outstanding end product, complete with visuals. Given the complexity of the problem, the political sensitivity and the lack of time, we knew from the start that it was a huge challenge. With the help of an intern, we engaged volunteers and professionals working with families in deprived areas of Amsterdam and, through their networks, we involved parents and children living in such areas. The research question and design were developed by the researchers in cooperation with the policy-makers. In the first phase of the research, we focused on understanding the problem, as experienced by families. We conducted formal interviews with professionals (n = 42), parents (n = 10) and children (n = 3) from five deprived neighbourhoods in Amsterdam, and enlarged our understanding through participant observations, informal interviews and work sessions (Table 2). In the second phase, we focused on co-creating solutions. In total three co-creation sessions were organized with volunteers and professionals (n = 64). The findings of the co-creation sessions were discussed in six focus groups: four with mothers (n = 21), and two with children (n = 30) (Table 2).

Table 2. An overview of research activities

	Phase 1: Understanding the problem			Phase 2: Co-creating solutions
	Semi-structured interviews (n = 55)	Participatory observations (n = 18)	Work sessions (n = 2)	Co-creation sessions (n = 3)	Focus groups (n = 6)
Mothers	7	45			21
Fathers	3	20
Children	3	61			30
Volunteers/professionals	42*		39*	64*

* Some volunteers who were interviewed were also involved in the work and/or co-creation sessions

The methodology included several cycles of critical reflection. We started with critical re-reading of the journal that we (CD and NG) kept during the research process, comparing the research design and the actual process with the PAR literature and the seven ethical principles of ICPHR, identifying the main dilemmas and issues encountered. Second, we invited two critical friends (JB and TA) to read a first draft of our manuscript to identify issues and concerns with the process followed. Thereafter, we reflected on the process with the principal and the policy project leader in extended interviews, which lasted for approximately one hour and were transcribed verbatim. During the analytical process, we alternated between (re-)reading of the data and the critical questions, and open coding and final code categorisation was based on the ICPHR ethical principles.

Results

In order to do justice to the complexity of the process and for the sake of readability, we do not use the seven ethical principles as a grid, but follow the process as it has evolved in practice and the dilemma’s we faced. First, we describe the different roles of the participants in the research project. We then describe how the working relationships developed during the project. Finally, we show whether we managed to make a difference. The participants’ experiences, needs and priorities have been published in a policy report (Dedding, Goedhart, and Kattouw 2017) and a scientific article (Goedhart et al. 2019.).

The different roles of stakeholders in the project: democratic participation under pressure

Due to the short timeframe, the interest in the for policy-makers’ new approach, and the fact that the City of Amsterdam had good working relations in the field, it was decided to work closely together. We met once a week at the municipality (90–120 minutes) to discuss each step and interim findings. The policy-makers accommodated field meetings and attended some interviews and work sessions in order to learn what is at stake for whom, and to connect to other projects and departments. While working, they often reminded us of the (often shifting) focus and (interim) deadlines, as the principal explained: ‘my goal was to catch the momentum in time for the elections’, and often highlighted this in our weekly meetings: ‘If we want to submit it to the board in January, a deadline for the report in October is not on time’ [notes 4-07-2017]. Although we often explained that we needed time and flexibility for genuine participation, in order to meet and to build trust with citizens who are not familiar with research and not spontaneously willing to connect with the local government, and to learn what is actually at stake, as a team we all had to stick to these deadlines, after all: ‘The internal consultation with aldermen cannot be rescheduled […]’ [notes 7-07-2017].

Given the lack of time, it would have been easiest and most efficient to invite citizens who are already known to professional organisations, like formal educational training. However, in line with the principles of equality and inclusion, we deliberately chose also to reach people living in deprived neighbourhoods at some distance from official organizations. It was expected that they might be most affected by the digital gap, and at the same time could profit most from technological progress. We adopted no exclusion criteria in relation to language capabilities and no formal group sessions were organized. With the help of professionals and volunteers, we met people at times and places that suited their practical agendas and needs. Rather than inviting them at times and venues that suited us or clashed with theirs, we respectfully tried to fit our research activities around their schedules. We explained the purposes of the project as ensuring that future interventions and policies would fit their life-worlds and needs, and asked adults and children to share their experiences of the digital world, their needs and ideas for solutions. During the conversations, we actively explored and discussed existing interventions, such as digital training materials and a new app developed by the municipality called ‘understand the letter’. For children, we arranged an excursion to a film museum, where they learned to make a stop motion movie. We planned transport, and used the time to get to know each other and to learn about what is at stake for them. In general, these real-life meetings lasted for one or two hours.

During the project, we worked closely with volunteers linked to language and debt courses, some of them with personal experience of living in poverty. They shared their (personal) knowledge and experiences and helped to introduce the project to people living in precarious circumstances. During all phases of the project we stayed close to one welfare professional in particular, who had worked as a volunteer in her spare time for more than 10 years, to feed into the discussion, and to validate our findings and actions.

We analysed the stories using an ethnographic content analysis, ensuring that we kept an eye on the situational context and providing the opportunity iteratively to gain a more in-depth understanding. The stories and expressed needs were shared with a group of policy-makers, educators, volunteers and creative designers in three co-creation sessions in order to exchange ideas, connect to what is already happening and to develop innovative and concrete ideas. These interpretations were brought back to a group of mothers to check whether these ideas fit with their daily life, ideas, experiences, and needs. The policy-makers’ idea of investing in educational apps for young children, for example, was quickly turned down by mothers, as they believed this would damage young children’s sight. The app that was developed to help to understand difficult letters was experienced as not user-friendly, and most of all as threatening their personal comfort zone and privacy. However, the idea of developing educational vlogs about the problems they experience was seen as an opportunity some would like to try.

We tried to work together with a group of ‘experiential-experts’, who are involved in a three-year schooling trajectory in order to become professional experiential advisers in the field of poverty reduction. We visited the school to explain the purpose of the project and to see whether we could work together. At that time the schooling programme had just started, people were busy making sense of their own and others’ experiences. Some of the students were really enthusiastic about being involved in the project. Unfortunately, we could not fully invest in this relationship due to the mutually inflexible schedules. We even had to decline a request of a student for an internship, since we could not invest in a proper working relationship (integrity principle).

In Table 3 we summarize the different roles of the citizens and policy-makers in the different stages of the project. It shows that we did not reach a high level of citizens’ participation, but we did so among policy-makers and, to a lesser extent, volunteers.

Table 3. Roles played by the stakeholders in the different phases of the research project

Research phase	Role played by the different stakeholders
	Informing	Consulting	Involvement	Collaboration	Control
Planning the process and participation	Citizens Volunteers			Researchers	Policy-makers^+
Problem formulation of issue creation			Citizens	Policy-makers^+ Volunteers Researchers
Priority setting		Citizens	Volunteers	Policy-makers^+ Researchers
Search for solution			Citizens Volunteers Educators Creative designers Policy-makers^++	Policy-makers^+ Researchers
Plan development		Citizens	Volunteers	Policy-makers^+ Researchers
Implementation
Monitoring and evaluation

⁺denotes Policy-makers of the City of Amsterdam directly involved in the project.

⁺⁺denotes Policy-makers of the City of Amsterdam who were not directly involved in the project.

Working relations: mutual respect, active and shared learning, equality & inclusion

Although in their daily lives participants faced many challenges in relation to the rapidly digitalizing world, we quickly learned that for most citizens the topic was not high on their agenda. Arranging for basic needs, daily concerns, and practical caregiving tasks were more important. Besides, we often had to balance citizens’ feelings of distrust regarding the municipality, ensuring their privacy and that their information could not be used in ways they had not authorized. Rather than making private notes, we wrote on a paper tablecloth so everybody could see our notes and get an overview of the data (see Figure 1). Breaking ‘the rule’ of not writing on a tablecloth, and also learning new words and phrases, often created a relaxed atmosphere, helped to build trust and resulted in active and shared learning. We also ensured that the children were having a good time, had the opportunity to explore and expand their skills and to reflect about who we are and why we felt it is important to listen to them. Given the lack of toys in informal day care, we left our research materials behind and later sent them extra items to play with afterwards. The community centre, we closely worked with, received a gift certificate for their investment of time and sharing their knowledge, which they used for their collective food garden. Individual citizens received a gift certificate for sharing their knowledge and experiences in an interview.

Figure 1. Use of a tablecloth to increase reflexivity and learning

Different types of knowledge – e.g. experiential, theoretical, practical, intuitive, and emotional ways of knowing – were all used and valued. We familiarized ourselves with theoretical models, such as the multiple access model of Van Dijk (2005), indicating that the digital divide is a product of four factors, namely: motivational, materials, skills, and usage access. While this model turned out to be helpful for understanding the different kinds of access that might be significant, we did not use it as a rigid grid. On the contrary, we sought to familiarize ourselves with the experiences and lives of people, whether young or adult, in their specific context with the scientific theory and knowledge in our ‘backpack’.

We did not invite citizens to the co-creation session. Due to time pressure we could not create a respectful and empowering setting for them; it would have demanded extra preparation meetings with citizens in order for them to feel confident, or at least reasonably comfortable in relation to formal professionals. Indeed, two of the four volunteers who attended later commented that they felt overwhelmed by the dominance of the experts and their lack of respect, which had the effect of limiting active participation and shared learning. Though we tried to mitigate this during the meeting, we didn’t fully succeed. Partly this had to do with the fact that organizations and professionals have to compete with each other for current and future activities; (un)intentionally this increased their need to advertise their added value and ideas to the policy-makers, often overruling the more modest volunteers.

The citizens’ experiences, needs and solutions were fed into the co-creation sessions by the research team, with the help of ‘personas’, and verbally explained. Personas – semi-fictional representations – are often used in the context of industrial design to express and focus on the major needs and expectations of the most important user groups; to aid in revealing universal features and functionality; and to describe real people with their own backgrounds, goals, and values (Guo, Shamdasani, and Randa 2011). Though useful in the context of design, it didn’t feel right in the context of PAR. It was talking about people rather than with people, and it risked over-generalization of stereotypes, often portraying people negatively as passive and unknowledgeable. Moreover, it increased the risk of side-lining topics which were not considered relevant from the professionals’ perspective. And this might inadvertently have limited the position of the volunteers.

Making a difference and ‘collective’ action

It is not easy to distinguish and articulate the project’s impact since it is embedded in the process, demands time – the scarcest resource in this project – and so could not be evaluated with all stakeholders. However, in order to evaluate whether we managed to bring some of the core principles of PAR into action, we need to look at impact, since action and change is what PAR is about. Therefore, we summarized some of the benefits of the process for the different stakeholders, shown in Table 4.

Table 4. The benefits of the process for the different stakeholders

Who	Active learning and change	Concrete actions	However, not yet ….
Citizens	Valued being listened to, researchers taking the effort to come to them with genuine interest for their lives, needs and ideas: ‘we talked today about very important things about ICT, and so on’(mother after a co-creation session) and ‘Happy to take part in this’ (Father, interview)	We provided gift certificates to compensate for their time and effort, and play material for children in voluntary care	Free WIFI
		The City of Amsterdam has extended the language and parenting courses^a in elementary schools with digital skills	Tailored training in a setting near to their home
		The city of Amsterdam has purchased digital devices for three community centres in three neighbourhoods in Amsterdam.	Access to computers in all community centres in Amsterdam
		Together with a volunteer organisation we applied for a prize from a national bank and received 5,000 euro to buy digital devices.	Accessible websites for all citizens
Volunteers	Connecting to other volunteers and experts in the field and gaining newknowledge: ‘Think big with people from all different projects leads to new insights’ and ‘you can look to digital inclusion from different angles’ (Reflections of volunteers on co-creation session)	Increased visibility; the important role volunteers play in the lives of people in vulnerable circumstances	A well-organised network – good ties and working relations with volunteers and professionals working with digital under-served people
Policymakers City of Amsterdam	Have experienced the importance of listening to citizens and improved their understanding of the complexity of people in vulnerable circumstances: ‘Something we could not have imagined, we are actually in such a different world, then they, by making their living environment more transparent, you come to other insights.’ (policymaker 1, interview)	Subject is now part of one of the main programmes of the City of Amsterdam; Digital skills is a new topic next to language skills and maths skills: ’at the end we secured the topic’. (policymaker 1, interview)	New cooperation’s between public and private parties to work on this topic: ‘The result of the co-creation sessions should be implemented by themselves [volunteers and professionals], but they didn’t yet’ (policymaker 2, interview)
	Felt the need for action; empathy as catalyst for change: ‘It was the only good way to put the topic on the Agenda. The stories brought a sense of urgency. “It [low digital skills] is not so bad” No, it is. We spoke to the citizens ourselves’ (policy-maker 1, interview)	The idea of developing apps for young children has been postponed, apps as the solutions are looked at more critically, the laptop arrangement for children is broadened and offers more flexibility and choice.	No quick wins: ‘It is complicated there is no clear solution; you hope for a simple one … we need to involve several sources. I am really glad we did this’ (policymaker 1, interview)
		An inspiration day was organised in order to connect volunteers, professionals and business partners and to move the agenda forwards. A second inspiration day is planned for next year.
		Four new projects have started, all follow on from the report’s recommendations, among others the vlog project with mothers, which they wished to try.
		The City of Amsterdam launched the Agenda ‘A digital city for and of everyone’. One of the three topics is ‘inclusive city for everyone’, and the actions reflect the recommendations of this research.
Researchers	Felt a strong need for advocacy and action	Highlighted the lack of pedagogical climate and toys for children in an informal care setting. (broadening the topic)	Projects initiated by citizens.
		Purposefully invested extra time in the dissemination of findings; invited an illustrator to develop provocative illustrations to get the messages across.	A dialogue among citizens, policy- makers, educators and professionals.
		Attended Policy Strategy meetings Initiated/pushed for follow-up projects, attracted attention of the Ministry of Internal Affairs. They used the finding to set a national agenda for digital inclusion and showed willingness to collaborate in future projects.
		Successfully applied for finance to start a project in which the digital learning needs and skills of citizens will be leading.

^aWeekly meeting for the mothers of children who are in elementary school. The focus of the meeting is currently on the Dutch language and parental issues.

Interestingly, Table 4 shows a relation between the depth of involvement and impact; the greatest impact was on policy-makers. Nevertheless, the process did open up alternative perspectives and courses of action. Some of the ideas at the start, e.g. to invest in education apps for young children, were not followed up, since they were not welcomed by parents. The urge for action was clearly felt and acted upon by the professionals, the municipality and beyond.

Discussion and conclusion

With hesitation, we accepted this challenging project and made it as participatory as we could. We had three reasons. First, leaving out citizens’ voices would lead to what is called ‘epistemic injustice’ (Fricker 2007; Groot and Abma 2018). Epistemic injustice arises when people are seen as less credible than policy-makers and professionals, leading to single or one-sided views of reality (Abma et al. 2017). Second, it would lead to a situation in which (again) top-down policies and interventions were developed that do not fit the daily reality and needs of people living in precarious circumstances. Third, it would be a missed opportunity to demonstrate the importance of listening to people who seldom have a voice in defining what is at stake and what actions should be taken. The question is: 1) how far did we meet (some of) the ethical principles of PAR as set out by the ICPHR, and with what consequences? And, 2) is it wise to start a PAR approach under such constraints in the future?

We started the project with a strong motivation for democratic participation, equality and inclusion, respect, and collective action to identify needs, to stretch the boundaries of science (if necessary) and to co-create knowledge for action. To a certain extent knowledge was generated through active learning in the daily context (testing apps, going on excursions), and policy-makers increasingly learned about people’s daily reality and needs. Establishing a respectful relationship with citizens was central to most of our actions, not ‘collecting data’, but fitting in with local activities and contexts, and the experiences people wanted to share. Different forms of knowledge were put on the table and valued: the knowledge of adult citizens, children, volunteers, educators, policy-makers and researchers; experiential, theoretical, practical, intuitive, and emotional ways of knowing were all valued, exchanged in dialogue sessions and brought together in a final report and agenda for action. The research has vividly highlighted a sense of urgency, leading to ‘ripples’ (Burns 2007; Trickett and Beehler 2017) in individual, community and policy systems, as shown in Table 4. Ripples that we expect to keep spreading thanks to the greater understanding and awareness of the problem, not only by reading a report or article, but by actually getting in touch with citizens’ real life stories.

Though we managed to put some of the core principles of PAR, as set out by the ICPHR in their ethics position paper, into practice, we could not do so for all them. Two ethical principles were not achieved, namely democratic participation and collective action. They stood at odds with the available resources (in particular time), the wish for quick wins and short-term policy impact. This jeopardized collective learning, did not do justice to the participants’ qualities, and might therefore unintentionally have fed stereotypes of people’s capabilities. The project was shaped and controlled by policy-makers and researchers, and we worked more closely with policy-makers than with citizens. In fact, citizens mainly functioned at the level of consultation while policy-makers functioned at the level of cooperation in each phase of the project. In other words, the maximization of the participation of those whose life is the subject of the research in all stages of the research process (ICPHR 2013) was severely limited. Second, we could not engage enough citizens in cycles of action and reflection over time. In most cases we met citizens only once, thereby limiting participatory democracy and learning. Though we sometimes felt frustrated about the time we had to invest in keeping the policy-makers of the City of Amsterdam on board, and the continuous shifting emphasis on goals, priorities and perspectives, part of the success of this project is due to their involvement. They facilitated every step in the process, ranging from opening up their networks, organizing practical pre-conditions for the research activities, translating the findings into a policy agenda, which they managed to sell to higher levels, and arranging a major conference to create awareness and shared learning. Moreover, due to their involvement they better understood for whom they work, felt the urgency for action and the importance for (future) participation.

A third critical point is that we were unable to arrange for dialogues between citizens and professionals. Instead, the researchers and volunteers functioned as a bridge between the two worlds, thereby limiting shared learning. Though from former research projects (Sarti et al. 2017; Bijleveld, Dedding, and Bunders-Aelen 2014; Abma and Broerse 2010) we know how valuable a dialogue is to increase mutual understanding of a multifaceted problem and the willingness to share ownership and to act together; we also know that such a dialogue should be well prepared with a representative group of people. Co-creative knowledge-generation requires facilitation so that trust can be built – particularly for a group who feels less confident and does not always have positive experiences in relation to formal institutes, as was the case in this project. But also professionals should be empowered in their role in order to avoid side-lining and the use of jargon (Elberse, Caron-Flinterman, and Broerse 2011). We could not pay enough attention to power issues in such a way that each voice would have been heard, dialogue encouraged and joint ownership created. As a consequence, the power of the insiders’ voice was limited; citizens could not fully show and develop their competences as owners and solvers of the problem. Moreover, the process hardly challenged negative stereotypes of people living in poverty as passive and depending on professionals for solutions, except for the many critical questions we as researchers raised in the problem-solving skills of professionals in relation to the complexity of the daily lives of people living in poverty. Working together, preferably in a shared space, could have stimulated empathy (Castro et al. 2018) – not only for citizens’ difficult lives, but also for people working in bureaucratic institutions, and why they act in the way they do. This understanding of the working of bureaucratic systems could reduce mistrust or reluctance to contact institutions/policy-makers for necessary applications or putting issues on the agenda.

The issues described are not completely new. Back in 1998, Robert Chambers wrote: ‘a repeated experience with PR [participatory research] has been the tension and contradiction between top down bureaucratic cultures and requirements, tending as they do to standardise, simplify and control, and demands generated at the local level, tending as they do to be diverse and complex and to require local-level discretion’ (Chambers 1998, 290). In this project this partly had to do with the sensitive political context, the fact that PAR was new to the policy-makers and that they had to report to different aldermen, which intensified their wish and need to control the project. Obviously, the project’s scarcest resource was time. Building trust and genuine relationships, the development and exchange of ideas, and translating these into action demands time, as many scholars have pointed out (Pain et al. 2015; Abma et al. 2019).

So, what constitutes ‘good’ participatory action research in this specific case? Choices have to be made in any project. In this project, the limited amount of time forced us to make a choice in the route towards action for social change 1) by facilitating a shared, bottom-up learning process with citizens and other relevant stakeholders, or 2) by facilitating a shared learning process with a focus on policy-makers. Given the scope and urgency of the problem of digital inequality, and the increasing exclusion of people already living in vulnerable circumstances, we believed that creating awareness and action at higher policy levels was more urgent. By working alongside policy-makers and their deadlines, we managed to create momentum in the City of Amsterdam and attract the attention of national policy-makers – not only for the specific problem, but also for working together with all stakeholders. Nevertheless, we were not satisfied, and felt a moral duty to continue what we considered an unbalanced and unfinished process. We actively advocated a follow-up project in which we would have the time necessary for good participation with citizens, and this was granted. It shows that what constitutes ‘good’ PAR is not only situational, as many have argued before, but can also be debated. Moreover, it demanded ongoing ethical work (Banks and Brydon-Miller 2019).

Hopefully, the growing recognition of the value of participation and the conditions it requires will help in developing more inclusive practices. It demands that we dare to share our lessons, whether positive or negative. We would encourage researchers to take up the challenge, the willingness to push boundaries, to be creative and to take risks. We hope to have shown that putting at least some of the core ethical principles of PAR into practice in less than flourishing circumstances did create ripples of change: citizens who seldom get a voice at a policy level shared their preferences, needs and knowledge, and the urge for action was clearly felt and acted upon, by professionals, the municipality and beyond.

Acknowledgments

The success of the project could not have been accomplished without the support of our commissioners of the City of Amsterdam. We would like to thank Maureen van Eijk, Maartje van de Beek and Sander Havermans for their courage in initiating this project, their firmness, and commitment. The authors also thank Rolinka Kattouw and Jantine van Wijlick, who were involved in the data collection. Last, but not least, we would like to thank all the citizens, volunteers and professionals involved for their collaboration in this project and for their valuable knowledge and insights.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the City of Amsterdam.

Notes

1. The City of Amsterdam provides laptops to children in secondary education and MBO (vocational education) whose parents are on low incomes. Once in four years, parents with children between 12 and 18 years old can also receive a laptop. The idea was to expand the laptop arrangement to children between the ages of 10 and 18 years, and to offer a choice between a laptop and a tablet.