Collective impact: We can’t take it for granted

Collaboration is one of those ideas whose value seems obvious, but is notoriously difficult to pin down.  Many significant social problems that plague children and families—violence, lack of economic opportunity, poor school achievement, unhealthy water and air, substance abuse, social isolation—cannot be turned around by a single program. The reason is that the causes of these problems are complex and multidimensional, and they call for population-level changes, not just the progress—however significant—of clients who participate in a particular program. Instead, they call for an “all hands on deck” response that includes participation from multiple sectors of a community, and not just formal programs, but the informal efforts of residents themselves, acting as friends, neighbors, and citizens.

It’s to this kind of community-wide effort[1] that many experts attribute, at least in part, the remarkable decline in rates of cigarette smoking among youth, or smaller, but still important, decreases in drunk driving, and unprotected sex. In recent years, “collective impact” is the term often applied to describe these kinds of results. According to its best-known exponents, there are five conditions that are common to successful collective-impact initiatives:

• A common agenda
• Shared measurement systems
• Mutually reinforcing activities
• Continuous communication
• A backbone support organization

But the scientists (among other skeptics) rightly say, “prove it.” Even if something is intuitively compelling, that’s rarely sufficient where scarce resources and political jousting are realities. It’s not enough to simply claim credit; there always seem to be plenty of folks ready to do that.

We must confront the counterfactual: how do we know any positive impact we observe wouldn’t have happened anyway, without the partnership/ collaboration? After all, communities can change for all sorts of reasons (population influx, outflow, as well as other trends unassociated with the intervention). And, further, maybe it was just one or two components/ partners that were responsible for all the heavy lifting, and the rest were just along for the ride?  Did we really need all those partners? All those meetings?

This is important, because collaboration comes with its own costs. Organizational resources (certainly time, but also often money) are shared—and shifted from other established functions or plans. It takes time to establish ways of working together that are respectful of the various partners, that reward leadership, that ensure accountability, that make for sustainability, and—most of all—that build trust.

So, demonstrating convincingly that collective, coordinated activities create collective impact is one of the holy grails of evaluation. Indeed, evaluators have wrestled long and hard with these issues, and have produced few easy solutions. What is the added value of such coalitions? Are they just a “feel-good” strategy, a way for everyone to pat each other on the back? Do they make a difference?

There are a few tantalizing results from studies that suggest the answer is “yes.”

Recognizing both the potential and the need to learn more, the current Administration has given at least implicit endorsement to the collective impact model, by supporting a number of place-based initiatives (which often include a collective impact approach), starting with Promise Neighborhoods, and extending to some of the Social Innovation Fund projects, including two where Child Trends is an evaluator. In these, we’re essentially asking whether the whole is greater than the sum of its parts—that is, whether intentional partnerships organized around shared goals (and some degree of shared measurement) can achieve what organizations working separately cannot. It is important, too, to understand the specific conditions (of place, population, and partners) under which collaborative partnerships are successful—or not.

Child Trends will continue to look for opportunities to advance our knowledge about common purpose and collective impact. This work will require both rigor and creativity. The “gold standard” randomized controlled trial is not likely to be feasible for this kind of study, but there are other quasi-experimental techniques (for example, interrupted time-series, difference-in-difference analyses) that can be appropriate. The field needs more evaluations that carefully document the theories of change, the implementation, and the results associated with collective impact efforts, and that may build the case for their catalytic value.

Note: Child Trends is an evaluation partner for Venture Philanthropy Partners and the United Way for Southeastern Michigan.

[1] Multidimensional efforts that often address policy changes, changes in social norms, and individual behaviors, and may include public media campaigns, as well as one or more evidence-based programs focused on youth, parents, or the public at-large.