Who gets included in Collective Impact: A mixed methods study of 10 CI initiatives

ABSTRACT

This mixed methods study sought to build knowledge of inclusivity practices among 10 CI initiatives. Analyses across two strands of research revealed two distinct definitions of inclusivity: broad inclusivity, which seeks the participation of everyone; and, representative inclusivity, which seeks individuals affected by the problems being addressed. While several of the initiatives had improved inclusivity practices since adopting CI, only a few were found to be broadly inclusive and most acknowledged operating in intentionally exclusive ways at times. All of the initiatives valued representative inclusivity, but members reported struggling to achieve even minimal levels. Proponents of CI should continue to develop guides for practitioners to help ensure both forms of inclusivity.

KEYWORDS:

• Collective impact
• inclusiveness
• local governance
• public health

Introduction

Since its introduction by Kania and Kramer (2011), Collective Impact (CI) has emerged as a popular model for public-sector collaboration. The CI approach seeks to develop solutions to seemingly intractable community problems through multisector, coordinated collaboration (O’Neill, 2020). CI identifies five conditions for success, including a common agenda, shared measurement, mutually reinforcing activities, continuous communication, and backbone support (Kania & Kramer, 2011). CI initiatives have proliferated across Canada, the U.S., and beyond, leading some to reference the model as revolutionary (Wolff, 2016).

Despite its popularity – or perhaps because of it – CI has received its fair share of criticism (Spark & Impact ORS, 2018). A primary critique has charged that CI is nothing new, but rather a repackaging of long-standing community development literature and practice. Accepted definitions of community development in the literature lend credibility to this argument (Christensen & Phillips, 2016). However, in a special issue of Community Development highlighting examples of successful CI initiatives, Walzer, Weaver, and McGuire (2016) acknowledge and transcend the critique by asserting the CI framework builds on previous approaches, articulating established strategies in more accessible language.

An additional critique accuses CI of exclusivity by intentionally “engaging the most powerful organizations and partners” (Wolff, 2016, p. 3) while omitting individuals most affected by the issues being addressed (Christens & Inzeo, 2015).

In response to these critiques, efforts are now underway to revise and improve the framework. For example, Cabaj and Weaver (2016) present what they refer to as Collective Impact 3.0, which “revisits the foundational elements of the CI framework” (p. 3). In one key revision, the authors replace the fourth condition of continuous communication with “authentic community engagement” (p. 5), noting that early CI publications stressed CEO-level engagement while omitting broader inclusion, particularly among populations most affected by the issues. Similarly, DuBow and colleagues consider CI from a national movement-building paradigm, rewriting the five conditions of CI. The authors indicate this is necessary given CI’s overemphasis on existing leadership structures, which target only incremental improvements (DuBow, Hug, Serafini, & Litzler, 2018). Indeed, scholarship is coalescing around the need for CI to take aim at genuine transformational change (Weaver, 2016).

In another significant revision, the authors of When Collective Impact Makes and Impact (Spark Policy Institute & ORS Impact, 2018) highlight eight principles of practice, developed by practitioners with implementation experience. These principles stress the importance of implementing CI in a complex community context and help guide the work with design elements such as inclusion of community members in the collaborative, placing priority on equity, and careful customization to fit the local environment. The authors’ emphasis on equity highlights a common theme of seeking to realize a more inclusive and equity-oriented version of CI that strives for authentic community engagement (LeChasseur, 2016; O’Neill, 2020; Raderstrong & Boyea-Robinson, 2016; Wolff et al., 2017; Wood, 2016). To be sure, these changes are not easily accomplished. However, as LeChasseur suggests, concrete steps can be taken now, which will move CI practice in the right direction. These include requirements issued by funders, technical assistance provided to CI practitioners from expert consultants, and the very fundamental act of including CI members who value inclusion and bring diverse perspectives (p. 236).

Other scholars are less content with corrective modifications to the CI framework. The most notable example includes a group of researchers and community organizers who adeptly argue that “we cannot repair a model that is so heavily flawed regarding equity and justice. It is time to move beyond Collective Impact” (Wolff et al., 2017, para, 8). The authors put forward six principles – rather than a specified model to be followed in whole – designed to aid coalitions in achieving the systemic change needed to realize equity and justice for all community members. The principles (collectively termed: Collaborating for Equity and Justice) appear to have traction in practice and scholarship (see especially Health Education & Behavior supplemental issue: “Collaborating for Equity & Justice”, Kegler, 2019) and are supported with an online toolkit complete with resources, case studies, and more.

Whether the pendulum swings toward continued improvement of CI or abandonment of it in favor of a better alternative, one thing is clear: proponents of CI are now emphasizing the importance of equity, justice, and inclusiveness, even citing inclusivity as a “fundamental democratic and moral principle” (Cabaj & Weaver, 2016, p. 5). It remains to be seen if practitioners adopt these revisions with the same receptivity they demonstrated for CI at its initial publication.

Research aims and questions

The aim of this research study was to examine inclusivity practices in active CI initiatives. Particular attention was given to two important aspects of CI activity: participant selection and decision-making. Ten public health initiatives were examined, all of which were following the standard CI approach without any special emphasis on recent CI revisions (e.g., CI 3.0). A mixed methods design was used for the study and the following research questions guided the analysis:

• Overall research question: Is participant selection and decision-making in CI practice inclusive?

  • Strand 1 sub research questions (qualitative): How is inclusiveness defined among the CI initiatives? What strategies are used to achieve inclusiveness?

  • Strand 2 sub research questions (quantitative): Are findings from Strand 1 of the study supported across the full sample of CI participants? Do CI participants believe their initiatives are achieving inclusiveness?

Methods

The study was initiated in the fall of 2017 after approval from the researchers’ institutional review board. The exploratory sequential design (Creswell & Plano, 2011) was used and included two research strands. The first was a qualitative strand featuring key informant interviews. The second was a quantitative strand that made use of a survey administered to all participants of CI initiatives. The two strands of research were designed to complement one another. As the exploratory sequential design begins with and emphasizes qualitative data, the study’s quantitative analysis is intended to provide supplementary data, helping to extend and validate the qualitatively derived findings across the study’s full sample.

Key informant interviews in Strand 1 of the study were exploratory in nature, asking interviewees general questions about how their CI initiatives selected participants and made decisions. Analysis of the interviews focused on identifying themes related to how inclusiveness was defined and practiced during actual CI operations. For comparative reference, Leach’s (2006) definition was used: “an inclusive process places few formal restrictions on participation” (p. 101). The quantitative strand was then used to test the accuracy of the qualitative themes across the larger sample. A 20-question survey was designed to assess levels of participant agreement. Questions utilized a seven-point Likert format and were roughly divided between five categories that included: (1) definitions of inclusiveness; (2) levels of inclusiveness achieved in the CI initiatives; (3) impact of administrative procedures on inclusiveness; (4) how the CI framework itself was perceived to impact inclusiveness; and (5) demographic questions. Quantitative analyses focused on descriptive statistics and are used as a validation check of qualitative findings.

Research participants

The CI initiatives represented in this study were identified through CityMatCH, a national Maternal and Child Health (MCH) organization working with a network of public health departments in 180 cities across the country. As such, all of the CI initiatives were operating in the field of MCH, which has made extensive use of CI through federal grant programs worth billions of dollars annually (e.g., HHS, 2014, 2016). Although each initiative focused on improving MCH outcomes, their precise objectives were determined locally and, therefore, differed somewhat, as did their slate of collaborative partners and community contexts.

To identify a study set for this research, CityMatCH conducted outreach to its membership to identify active CI initiatives. Of the 180 member health departments, a total of 12 currently functioning CI initiatives with an MCH focus were identified. Of these, one CI initiative did not agree to participate in the study, indicating they were too early in their work and wanted to be further along before participating in a research project. A second maintained only lose affiliation through an online forum, which precluded participation in the study. The remaining CI initiatives each agreed to participate, with leadership from each initiative providing a full membership list with contact information for participation the study. This yielded a study set of 10 CI initiatives with a total of 226 participants.

The study set was diverse, with all four U.S. regions represented and population sizes ranging from 100,000 residents to over 2,000,000. All 10 of the CI initiatives were early in their work, having established common agendas and beginning to organize for action. Several of the initiatives had previously functioned as generic collaborations before adopting the CI model. (Table 1) presents an overview of the 10 CI initiatives included in the study with key descriptive details on each.

Table 1. Overview of the 10 CI initiatives included in the study

	Community Population	Number of CI Members	Inclusion of Members with Lived Experience (Yes/No)	New CI Initiative or Modified Collaborative	Main Project Focus
CI Initiative #1	625,000	7	No	New	Health Equity
CI Initiative #2	1,500,000	13	No	New	Health Equity
CI Initiative #3	165,000	61	No	Modified	Perinatal Mental Health
CI Initiative #4	113,000	9	Yes	Modified	Group-based Prenatal Care
CI Initiative #5	269,000	10	No	New	Substance Use in Pregnancy
CI Initiative #6	692,000	13	Yes	Modified	Infant Vitality
CI Initiative #7	874,000	7	No	Modified	Financial Literacy for New Mothers
CI Initiative #8	2,100,000	11	Yes	New	Infant Mortality
CI Initiative #9	488,000	32	No	Modified	Maternal Mortality
CI Initiative #10	571,000	8	No	New	Breast Feeding

To accommodate the mixed methods design, participants were divided into two samples. The qualitative sample (Strand 1) included 10 key informants – one from each participating site. Variation in key informants was intentionally sought to ensure diversity of perspective. This aided qualitative theme identification, surfacing themes that would likely not have emerged with a more homogenous qualitative sample. All 10 selected key informants agreed to the interview resulting in: (1) demographic diversity by race, ethnicity, age, and gender; (2) representation from each of the four U.S. regions; (2) variability in jurisdiction size that mirrored the full study set (i.e., 100,000–2,000,000); and, (3) six interviews with public (i.e., health department) employees, two with private/for-profit participants, and two with participants from nonprofit organizations. All remaining study participants (Strand 2) were then included in the quantitative methods and analysis. Strand 1 participants were removed from the quantitative strand to avoid response bias that may have resulted due to their prior in-depth conversations on the subject matter with the researchers.

(Figure 1) presents the full study set and samples used in the research.

Figure 1. Study set and samples.

Data collection and analysis

Data collection for the study included a qualitative interview protocol and a survey instrument. Data collection began with 10 key informant interviews, which were conducted in November and December 2017. Given the geographic distribution of sites included in this study, eight interviews were conducted via telephone (Drabble, Trocki, Salcedo, Walker, & Korcha, 2016; Rahman, 2015) and two were done in-person at a CityMatCH event where two key informants were present. No differences were noted between the two interview techniques (e.g., depth, rapport, interview length, etc.).

The qualitative interviews made use of the responsive interviewing technique (Rubin & Rubin, 2012), which utilizes preplanned main questions and context-dependent follow-up questions, supplemented with response probes to solicit detail and context. All interviews were recorded with interviewee permission and later transcribed for thematic analysis. Interviews averaged one hour in length and were divided into four sections. The first section asked interviewees to give a general overview of their CI initiatives. Sections two and three explored participant selection and decision-making processes, respectively. The final section addressed the research questions in the most direct manner, asking interviewees specifically about inclusiveness practices in their CI initiatives. Field notes were taken throughout the interview process to note novel information, identify possible developing themes, and to actively engage the emerging data. Thematic coding utilized etic analysis (Berg & Lune, 2011) to discover how the CI processes discussed by interviewees impacted inclusiveness within their CI initiatives. Codes were validated across three researchers.

Survey measures made use of previously published research (Leach, 2006), replicating measurement scales and questions from the prior study’s survey instrument. This previous study was well-suited for use in the present study because it investigated democratic values, including inclusiveness, in collaborative initiatives. Findings from the qualitative strand were also used to create additional survey questions. Questions on the survey utilized Likert-style responses and asked about broad and representative inclusion in the initiative’s participant selection and decision making (e.g., Participation is open to everyone in our CI initiative; When decisions are made in our CI initiative, everyone is included, etc.). Additional questions on the survey assessed: (1) the impact CI had on partnerships (e.g., Because of CI, I have new partnerships in my community, etc.); (2) the level of formality in CI processes (e.g., Our Collective Impact initiative operates without the need for formal rules, etc.); and, (3) the types of strategies used to achieve inclusivity (e.g., In order to add new participants to our group, we need to have an existing relationship of trust with them).

The survey was pilot tested by two scholars with expertise in quantitative methods and two local public health professionals with expertise in MCH and CI. The final version of the survey was launched in January 2018 via SurveyMonkey and remained open through April 2018. The survey was sent to all participants of the CI initiatives who were not part of the qualitative sample, and was conducted online with links sent to participants via e-mail. Incentives were not provided; however, follow-up e-mails were used to achieve a response rate of 79%. Item order was randomized by the survey software to avoid order bias. All questions were single-item Likert questions with the exception of demographic questions and a final open field response question (i.e., would you like to provide any additional comments). Results were assessed between CI initiatives and found not to vary significantly across the 10 sites.

Results

To better understand inclusiveness practices in CI initiatives, this study examined participant selection and decision-making processes in 10 CI initiatives convened to improve maternal and child health. Following the sequential nature of the research, findings are presented in two strands: qualitative followed by quantitative.

Strand 1: Qualitative findings

The research questions for Strand 1 of this project asked: How is inclusiveness defined and what are the strategies used to achieve it? Qualitative analysis yielded four thematic findings: (1) inclusiveness was defined differently among the CI initiatives; (2) inclusiveness was partially achieved through new and deepened partnerships; (3) inclusiveness was impeded by informal CI processes; and, (4) inclusiveness was rarely open, but rather strategic and relational.

Differing definitions

Qualitative analysis revealed that interviewees held two different definitions of inclusiveness. The first definition (hereafter, broad inclusiveness) closely adhered to the reference definitions of inclusiveness found in the literature (i.e., “an inclusive process places few formal restrictions on participation” (Leach, 2006, p. 101)). In their own words, interviewees expressed this understanding of inclusiveness in the following quotes.

We manage to be constantly inclusive, so we’re always educating community partners on the work and asking them to join in the collaborative effort.

We started by brainstorming the list of everyone who should be involved and then we just went for it. We invited everyone (emphasis by interviewee) and it just went from there.

We initially brainstormed a list of people and organization that might be interested in being involved and have a stake … And then it’s just grown and branched out from there. Now people from the group email me and say, ‘Hey I’m thinking about inviting this person, is that okay?’ And I say, ‘Sure.’

A second group of interviewees defined inclusiveness by the intentional inclusion of specific individuals (hereafter, representative inclusiveness). Namely, people with first-hand knowledge of the challenges the CI initiatives hoped to improve. Interviewees commonly referred to this as “lived experience.”

Through the use of data, we understand that we have these huge disparities, and we have to get to the root, underlying cause. And data shows us where the challenges are in our communities. For us, there are three zip codes where we need to focus. [But], if we don’t have lived experience around the table, how are we going to know what to do?

One interviewee, who serves on her CI initiative as a community representative, discussed lived experience, explaining: “The passion I have for this work comes from lived experience. I went through these struggles as a new mother, so I know it firsthand.” Apart from her comments, inclusion of persons with lived experience was referenced as something missing from their CI initiatives. The interviewees acknowledged it was important, but in practice, this kind of inclusiveness was lacking.

We’re very fortunate here to have our own biostatistician who seeds the infant health network with all of the data that we need around infant mortality and all of the birth certificate data and everything like that … So we’re pretty much in a good place, but not when it came to collective impact. And what I say by that is, we were missing the hardest people at the table: the women we’re focused on … who we’ve brought to the table since then has been [more] subject matter experts.

Communities complain about this [not including lived experience], we go into communities where there is need, do a grant, grant ends, and we leave. This is so disempowering to communities.

New and deepened partnerships

A second theme noted in the qualitative analysis highlighted how the CI process improved inclusiveness over previous collaborative efforts. For the 10 CI initiatives included in the study, collaborative projects were often already underway. When they transitioned these collaborative projects to the CI approach, one result was a growing and more inclusive group.

We were working in this area for about the last three years, but we didn’t take it on as precisely Collective Impact until about a year and half ago. It [our previous work] was a pretty small working group made up of the key players. When we moved into the Collective Impact model, we expanded our partnerships.

Collective Impact certainly helped us to expand our partnerships. If it wasn’t for Collective Impact, I don’t think we would be working with some of the partners we are today.

In addition to creating new partnerships, the interviewees reported that previously non-engaged partners became more engaged and active when the work turned to the CI model.

With it [CI] we’ve seen the number of people who participate increase and the number of people who attend regularly increase.

People started calling and saying, ‘are we doing Collective Impact at the meeting tomorrow?’ So, they wanted to know that it was happening. And then a number of people started to come to the meetings more regularly; they were just on the rolls before but not really coming to the meetings. It’s really made a huge difference.

Informal processes

The third theme was less optimistic, noting that inclusiveness across the CI initiatives appeared to be impeded by informal administrative processes. The 10 CI initiatives studied here struggled to implement clearly defined processes, defaulting instead to informal and ad hoc operations. As one interviewee explained: “Collective Impact does help to formalize some of things you run into, but other stuff just comes up and we just kind of have to figure it out.”

This theme was present across all of the interviews and strongly impacted the interviewees’ perception of decision-making in the CI initiatives. From their perspective, procedures were not specified, and decisions were made without a clear understanding of who made the decision or why it was made. As one interviewee succinctly shared, “From my perspective, there appears to be several different decision-making channels.”

The use of informal administrative practices also included the creation of leadership structures and the selection of participants for leadership positions. One interviewee casually explained, “We found we needed a mechanism to organize what this work might look like, so we created a leadership team.” In some instances, the level of informality resulted in confusion about even the interviewee’s own inclusion in their initiative’s leadership.

[We have] a core set of governance organizations. The health department is part of it along with several other organizations, probably 15. I’m not sure how that group was formed [contemplative pause]. I know I was added, but I’m not sure why or how.

Strategic and relational inclusion

The final qualitative theme identified common approaches used by the initiatives to add participants. Stated simply, participants were most frequently included if they knew someone or were thought to add strategic value.

If a subgroup says, hey we need to have a banker join our group, then that’s decided in that subgroup, and if the group knows someone to invite; maybe Lisa says, ‘I know a banker.’ The group says, ‘ok great.’ Then Lisa’s going to invite the banker. And then it will come back to the core team to say, ‘FYI, Lisa’s inviting a banker.

In their planning and implementation, the subgroups may identify a new partner or gaps in who they have … and we have dialogue and shoot out possible names.

The above quotes demonstrate how strategic and relational inclusion operated to add participants. However, other interviewees surfaced this theme in a markedly less inclusive manner, being strategic about whom not to include.

Adding partners is not about the fishnet approach, but about being very targeted while leaving the door open. You could invite as many people as you want and there could be someone real screwed up at the table who could really hijack this work and put it in government bureaucracy quicker than you can blink by adding rules or ego into the mix.

How members are added to the group is what the core team thinks is their alignment with our values, and also how do they fill a strategic role in the Collective Impact initiative. So, let’s say someone who is a political champion; they may fill a strategic role, but they also have to be a political champion whose values align with the core team and the larger group.

Trust is an unspoken value for us. Who we can add to our group has to do if we have a relationship of trust with them … There are organizations that will chase money and take credit for work that is not theirs. Sometimes we don’t invite those organizations, and sometimes we have to; the old cliché about keeping your enemies close.

(Table 2) presents a summary of the study’s qualitative themes.

Table 2. Qualitative themes summary

Differing Definitions	Qualitative analysis revealed that interviewees held two different definitions of inclusiveness. For some, being inclusive meant being open to all. For others, it meant including persons with lived experience.
New and Deepened Partnerships	The CI process improved inclusiveness over previous collaborative efforts. Many new partners were added, existing partners became more engaged, and working relationships were enhanced.
Informal Processes	Informal administrative processes impeded inclusiveness practices. People were added or excluded, and decisions made without everyone’s involvement.
Strategic and Relational Inclusion	Participants were included if they knew someone or added strategic value. Others were intentionally excluded when trust was lacking, or values were misaligned.

Strand 2: Quantitative findings

The research questions for Strand 2 of this project asked: Are findings from Strand 1 of the study supported across the full sample of CI participants, and do the CI participants believe their initiatives are achieving inclusiveness?

Results for this portion of the analysis are derived from survey respondents representing the same diverse cohort of 10 CI initiatives considered in Strand 1 of the research. The survey’s overall response rate was 79% (N = 216) with site-specific response rates ranging from 33% to 100% (M = .75, SD = .25). The number of participants in the CI initiatives ranged from seven to 61 (M = 21.6, SD = 18.55). Given variations in the number of respondents per initiative, data in this section of the analysis were weighted to give equal representation to each of the 10 CI sites. (Note: for the measures reported here, data were not found to vary significantly by site).

With a few notable exceptions, quantitative data largely validated the themes and provided additional detail for the overall analysis. Concerning the first theme of Differing Definitions, the survey assessed the importance CI participants placed on each of the two forms of inclusion (i.e., broad inclusivity is important in CI?; representative inclusivity is important in CI?). Respondents agreed, with nearly identical ratings, that both forms of inclusivity were important in CI practice (92% and 94% respectively, either somewhat agreed, agreed, or strongly agreed). Support was also noted for the theme of New and Deepened Partnerships, meaning respondents generally agreed that the CI framework had served to advance partnerships (83% agreement); however, a sizable percentage of neutral responses (14%) and a few negative responses (3%) were noted, indicating some ambivalence within the sample.

Support for the third theme of Informal Practices was mixed. While respondents agreed that administrative processes were created along the way in their CI initiatives (86% agreement), they did not uniformly view the resulting structures as informal (32% agreed their CI initiatives operated informally, 28% were neutral, and 39% disagreed). In a subsequent question, most respondents felt CI had provided formal structure for their work (74% agreed). Finally, the theme of Strategic and Relational Inclusion was tested. Respondents felt both approaches to inclusion were used in their CI initiatives, and in relatively equal measures (78% and 75% agreed respectively, that these inclusion strategies were used in their CI initiatives).

The second research question in Strand 2 of the study asked whether the CI participants believed their initiatives were achieving inclusiveness. Given Strand 1 findings, this part of the quantitative analysis considered both broad inclusiveness and representative inclusiveness. Mirroring the qualitative findings, survey data found that a majority of respondents believed their CI initiatives were achieving broad inclusiveness, but not representative inclusiveness. Eighty-four percent of respondents positively assessed their initiative’s broad inclusion, with less than 2% percent indicating they disagreed or strongly disagreed. Conversely, when asked about inclusion of “affected groups,” respondents generally rated their CI initiatives negatively. Just over two thirds (68%) felt they were not adequately included, while only 4% agreed or strongly agree they were included.

Discussion

The qualitative analysis revealed a complex picture of inclusiveness practices among 10 CI initiatives operating in the field of MCH. Viewed from one perspective, the CI process facilitated new and closer connections. From another perspective, without clearly articulated rules of engagement, the initiatives created their own common-sense rules for participant selection and decision-making. These rules worked to include likeminded colleagues, exclude others when trust or values were in question, and routinely miss populations with direct experience of the issues being addressed.

The quantitative analysis demonstrated support in most instances for the themes identified during Strand 1 of the study. However, not all survey respondents agreed that their CI initiatives had facilitated new partnerships or had relied on strategic and relational inclusion. Additionally, they felt CI had provided structure for their efforts and disagreed that administrative processes were conducted informally.

These findings should inform the field of MCH, which has relied heavily on the CI model for almost a decade. Other analyses have focused on the programmatic efficacy of CI for MCH practice (Landry, Collie-Akers, Foster, Pecha, & Abresch, 2020); however, the question of who gets included in CI efforts to improve the health of women and children has not been addressed until now.

A key question raised by this study is: Does the CI approach facilitate inclusiveness in real-world practice? The data indicate that CI did not specifically promote or prohibit inclusiveness. Depending on the context and parties involved, CI practices were inclusive in some ways and exclusive in other ways. Additionally, in the absence of specific instruction, CI initiatives appear to often develop their administrative procedures based on expedience, sometimes resulting in exclusive strategic and relational strategies.

Broad inclusion, as defined in the literature, is neither strategic nor relational: “an inclusive process places few formal restrictions on participation” (Leach, 2006, p. 101). In other words, participants are not handpicked for their expertise or because an existing member of the team knows them. They are also not intentionally excluded, even when their values are perceived to be different from those of the initiative’s leadership. To be sure, broad inclusivity was largely missing from the CI initiatives included in this study. Additionally, while representative inclusiveness was valued, participants uniformly discussed it as missing.

These findings reflect those of others (Landsman & Roimi, 2018) and make it clear that if CI is to endure as the go-to model for collaboration, this challenge must be addressed. For example, a recent study of 25 CI sites found meaningful inclusion to be a challenge area, noting that “most sites struggled with implementing inclusion strategies that ensured adequate representation and shifted power to the communities being affected” (Spark Policy Institute & ORS Impact, 2018, p. 70). Proponents of CI should increasingly emphasize inclusivity, noting the importance of both forms, and providing practical aids for practitioners to use. One such resource, now available to practitioners, does exactly this, providing 10 key questions to help CI initiatives assess and improve their readiness for engaging people with lived experience (Homer, 2019). Similarly, in discussing the importance of authentic community engagement, Attygalle (2017) should be informative to practitioners who often recognize the importance of including content experts but overlook the need for context experts. As Attygalle concludes, “context experts are necessary partners in determining new ways of working that will lead to more social inclusion and less poverty, and more vibrant and healthy communities” (p. 7).

A second key question raised by this study is: Does CI provide a way to structure collaborative activity or does it need more specificity? Data from the study are mixed. Qualitative findings demonstrated a high level of informality across the CI initiatives. However, from the participants’ survey responses, findings demonstrated that CI was viewed as structured. This could, in part, stem from the fact that several initiatives had been existing collaborations that were repurposed to fit the CI model. Applying CI’s five conditions resonated with participants and felt like needed structure. As CI continues to mature, an opportunity exists to lean into the wealth of knowledge already established in literature from the fields of community development, network governance, citizen participation, and related areas. Gleaning insights from this work will allow CI to avoid needless reinvention.

The ambiguous findings on CI structure in this study may also reflect the relatively early stage of work for these 10 initiatives. In the planning stages, CI appears to be well structured, particularly the work to develop a common agenda. Perhaps this recent history was what participants reflected on during the study. When the CI initiatives moved beyond the common agenda stage, things changed. At this point in the CI process, a period of reorganization took place, which was not expressly addressed by the CI framework. Proponents may argue that CI’s conditions of continuous communication and shared measurement are designed to structure this phase of the work; however, these conditions do not address the need felt by all 10 initiatives to delineate various substructures within their initiatives (e.g., working groups, leadership councils, co-leads, etc.) as administrative strategies for planning and carrying out activities.

The need to create this missing structure is nothing new to the community development literature. Bowen (2005) for example calls upon community development practitioners to “create institutional mechanisms” (p. 87), which will enhance the impacts of collaborative community improvement efforts. CI researchers have noted the need as well. However, the Collective Impact Forum, an initiative formed by two consulting agencies designed to support and advance CI, actually encourages and describes what they refer to as “loose structure” (Preskill, Parkhurst, & Juster, 2014, p. 4). While loose structure may not appear problematic at first glance, we ought to take issue given the fact that many CI initiatives are working to advance critical aspects of public wellbeing. As was the case here, these initiatives are also commonly funded with taxpayer dollars, heightening the need to ensure their processes are fair and conform to our larger democratic ideals. Viewed from this perspective, concerns stemming from informal processes are manifold and likely to include some of the following challenges:

• Informal processes are unlikely to be transparent because they lack consistency and clarity and may not be communicated to the full group every time.

• Informal processes are unlikely to be impartial because decision-making is unstructured and may involve hidden rules operating without the knowledge of all participants or the public at large.

• Informal processes are unlikely to be fully deliberative (i.e., thoroughly discussed/debated) because some decisions get made in small groups without all-party consideration.

• Informal processes that haphazardly create leadership and working subgroups are unlikely to have representative inclusivity because appointments to the groups are not open and scrutinized.

These potential challenges suggest that CI initiatives need assistance in creating administrative structures that promote inclusivity. If division into workgroups and leadership committees is a practical need, perhaps some democratic assurances could be instilled. These might include full-initiative voting on leadership positions with defined term limits, workgroup minutes on decisional items with full-initiative review and ratification, and processes for ensuring representative inclusivity within all workgroups and the leadership committee. Processes such as these, would better ensure inclusion in CI practice.

Study limitations

Given the nuanced, at time even individualized, findings of this study, caution should be stressed in generalizing. Results here are best interpreted as a particularized view of inclusivity practices within 10 CI initiatives. Perhaps an even more fine-gained approach would limit these findings to the public policy area of MCH. Methods employed here could be replicated to study additional CI initiatives in other sectors to determine if the findings hold.

Additionally, it must be acknowledged that the picture painted of these 10 CI initiatives is still unfolding. As indicated earlier, the initiatives were relatively early in their work, having established a common agenda and beginning to organize for action. As the initiatives mature, perception and practice are likely to change. In fact, some changes in practice were already evident during this investigation. For example, the initiatives reported using voting as a practice during the creation of their common agendas but had moved away from that as time progressed. Other important changes that impact inclusivity may take place as the sites get further into implementation efforts.

Conclusion

The rise of CI in public-sector collaboration in the 2010s was not only dramatic, it was also needed if collaborative models were going to produce results. As this study makes clear, CI captured the attention of practitioners and created a renewed sense of engagement and excitement. Given the popularity of CI, efforts should be made to ensure the model is truly inclusive. Practitioners appear to hold this value as well. Across the interviews, respondents made clear that their CI initiatives placed a high conceptual value on inclusion; however, in practice not all groups were invited to participate.

Without the involvement of everyone, especially affected populations, CI is likely to misunderstand the problem, develop ill-fitting solutions, and maintain existing patterns of dominance and privilege (Wolff, 2016). Results here lend empirical support to such critiques. To be sure, the intentional exclusion of some and the common absence of affected populations are the study’s most troubling findings and pose a clear threat to inclusiveness. In this way, CI may promote technocratic, rather than democratic ideals, working to solve a community’s complex issues, but supplanting “the people” with an array of colleagues and experts. One interviewee unwittingly summarized this concerning pattern when she explained: “FYI, Lisa’s inviting a banker.”

The decades’ old call remains relevant today, “Many worthy projects and skillful practitioners utilize coalitions as a health promotion strategy, a firmer basis in research is still warranted” (Butterfoss, Goodman, & Wandersman, 1993, p. 315). Research specifically addressing inclusiveness in coalition work, including CI, must continue, not only to better understand the inclusivity practices being used, but as a strategy to maintain pressure on CI proponents to provide workable solutions to well-intentioned practitioners who continue to make use of their powerful model. Finally, we must continue to emphasize the principles for collaborating for equity and justice, especially relevant here is principle 2, which calls for the use of a community development approach in which residents have equal power. While “sharing power equally with those most affected by issues is a significant challenge for collaboratives” (Christens et al., 2019, p. 112S), solutions must be discovered, not only to honor our democratic values, but to drive systemic change and coalition success.

Disclosure of potential conflicts of interest

No potential conflict of interest was reported by the author(s).