Peacebuilding Accountability: The United Nations Peacebuilding Fund and Community-Based Monitoring and Evaluation

International peacebuilding as a discourse and practice has expanded rapidly in the nearly three decades since UN Secretary General Boutros Boutros-Ghali's Agenda for Peace (1992) was published. Alongside the growth of peacebuilding efforts and the expansion of the concept from its narrow roots of post-conflict reconstruction has come the realization that many peacebuilding projects conceived of and sponsored by the international community have failed to meet their own objectives. More importantly, many peacebuilding projects have failed to be embraced fully by those whom they were supposed to help: the individuals and communities attempting to rebuild their lives in post-conflict countries.

This resistance, known as hybridity in peacebuilding, has led to a more thorough examination of issues surrounding local ownership in peacebuilding processes. One seminal work in this area is by Donais (2012), who delineates local ownership into maximalist and minimalist approaches, with international funders essentially writing unlimited checks at the maximalist end of the spectrum and reducing locals to turnkey operators at the minimalist end. Other authors, such as Richmond (2010, 2012), argue that internationally driven efforts to inculcate local ownership have foundered on the shoals of an overfocus on institutional statebuilding and the conflation of the national with the subnational as defining the local. Another strain of thought on local ownership in peacebuilding focuses on questions of agency, which Richmond (2013) describes as liberating and which Hancock (2017) describes as a form of basic human need. As a basic need, agency would be seen as ontological, a drive that cannot be deterred, only denied, with the consequences of denial connecting directly to the resistance that leads to hybrid outcomes.

The conclusion that this basic need for agency is, arguably, at the heart of local resistance to international imperatives resulting in hybrid or less than satisfactory outcomes might then lead to the proposition that to have effective peacebuilding, one should push for a more maximalist vision of local ownership. A maximalist vision of local ownership, or local agency, requires that beneficiary communities have some say over the goals of peacebuilding programming (cf. Hancock 2017). The issue of who has input into defining programmatic goals, like that of beneficiary involvement in evaluation, is a source of tension and conflict between beneficiary communities and those in the hierarchies of liberal peacebuilding organizations. The mindsets of those embedded in multilateral organizations and even national governments may be guided by a sense of responsibility for outcomes over an understanding of the importance of process. This understanding, as well as the processes used to develop peacebuilding programs and projects, tends toward outcome-oriented thinking, focusing on rapid impacts, concrete deliverables, and quantifiable and measurable changes that can be used by these individuals and agencies to report to those higher on the food chain.

Furthermore, the push for this maximalist vision of local agency runs into the reality that many, if not most, peacebuilding efforts require outside support, either in the form of technical assistance, or at the very least, in the form of resources—financial or otherwise. The need for outside assistance brings with it the expectation that this assistance will bring measurable benefit and should not be wasted, misused, used ineffectively, or at worst, siphoned off for corrupt purposes. To address this expectation, the field of evaluation was developed to measure the impact and effectiveness of resources used for development or improvement, with the goal of ensuring that those receiving resources are accountable for their use (Rossi, Freeman, and Lipsey 1999).

Program evaluation is the main way that international funding agencies operationalize the idea of accountability. Despite the fact that accountability mechanisms are considered by some peacebuilding programs to be onerous burdens and poorly connected to positive outcomes (Hancock 2020a), or that many funding agencies feel that traditional evaluations do not accurately answer their questions about the impact and effectiveness of the programs that they fund (Bush and Duggan 2013), there have been few major changes to evaluation and accountability models in the peacebuilding field. Even with the ubiquity and importance of funder accountability mechanisms, there is little research on their impact on local ownership, community agency, and peacebuilding success. In the development field, and in the realm of anti-corruption research, some work has been done to examine accountability models and to try to develop new ones (Galtung and Tisné 2009). However, even though new accountability models, such as community accountability (accountability down) or mutual accountability mechanisms have been used occasionally, little to no work has been done to examine their use, or lack of use, by larger international funding organizations (Wilén 2009).

This article examines the nexus of issues around accountability, local ownership, and community agency, and their connection to evaluation models and practices. The connection between these areas provides fertile ground for the development of a more community-oriented, mutual accountability system designed to meet the needs of all stakeholders involved in peacebuilding in conflict-affected societies.

The context chosen for this application is the UN Secretary General's Peacebuilding Fund (PBF), which serves as the UN's “financial instrument of first resort to sustain peace in countries or situations at risk of violent conflict” (United Nations Peacebuilding Fund, n.d.). As an organization that provides peacebuilding funding to both UN and non-UN agencies, the PBF is required to provide a plethora of reports to many of its own stakeholders, including the Secretary General, General Assembly, and the donor countries that replenish its resources on a regular basis. The PBF continues to invest in innovative systems and capacities designed to improve learning and provide accountability to this array of stakeholders. In its 2020–2024 strategic plan, the BPF committed to testing new and innovative approaches to monitoring and evaluating projects, programs, and whole country portfolio evaluations (United Nations 2020a).

My part in this project came from my good fortune in being awarded a year-long fellowship from the Council on Foreign Relations to work in a policy-related position with either a US or international governmental organization. With CfR's support I was placed at the UN PBF for the 2021–2022 academic year as a Local Peacebuilding Advisor to the PBF's monitoring and evaluation team, a small group of professionals commissioning and overseeing the evaluations for all PBF-funded projects and tasked with contracting and overseeing thematic reviews as well as drafting policies and guidance for evaluation of UNPBF-funded programs. My main role as a member of this team was to lead the effort to develop a more collaborative form of peacebuilding evaluation. The goals for this project were to focus on providing opportunities for beneficiary communities to participate in evaluation efforts and collaborate with other stakeholders in order to enable quicker responses by peacebuilding providers and funders on the ground—through closing accountability loops—and to develop methods that were less extractive and more collaborative than those previously used by the PBF. In doing so, I was not only able to learn a great deal about how the PBF operates and about the processes it uses to commission and review evaluations, but I was also able to learn about some of its institutional goals as well as the requirements that even large organizations have in justifying their work and the use of resources to other UN organs and, most importantly, to the many donor countries that continue to fund this important work.

Thus, this article represents both a scoping analysis of the strengths and weaknesses inherent in the interactions between large international donor agencies and local peacebuilding efforts alongside a preview of one agency's efforts to bridge those gaps and implement programs and processes designed to promote local agency while supporting transparent accountability.

Because of the nature of peacebuilding as representing a sort of kitchen-sink approach, there is often a great deal of confusion about the concepts used to delineate its contours and boundaries. So much so that as peacebuilding has gained in institutional acceptance, more entities are describing what they do as peacebuilding (Barnett et al. 2007). Therefore, it is prudent to review briefly a few of the concepts that drive this analysis and underpin the approach to peacebuilding evaluation developed later in the article.

Accountability is the process by which those charged with providing services report back to others to demonstrate the quality of the work being done or the services being delivered. Accountability mechanisms are at the center of governance and the work done by nonprofit groups, whether governmental or civil society. All evaluations, whether of peacebuilding projects or development initiatives, are exercises in accountability, and should be designed to demonstrate the appropriateness of policies and processes, the efficiency of the work being done, and the overall quality and effectiveness of the outcomes.

In development and peacebuilding there are typically three different forms of accountability. Upward accountability is when the provider organization reports to those above it on the food chain, namely to national or international funding agencies. This is the typical form of accountability practiced by peacebuilding funders and delivery organizations and has led to widespread complaints that it tends to distance purported beneficiaries from peacebuilding providers (see Paffenholz 2010; Richmond 2010; Donais 2015). Downward accountability is when “local communities or constituents hold governmental or other institutional leaders accountable for the performance of their projects” (Hancock 2020a: 142). Downward accountability contributes both to increased agency and, where service providers are responsive, to the perceived legitimacy of the service providers and their organizations, empowering local communities to root out and address corruption (Galtung and Tisné 2009). A third form, known as horizontal or mutual accountability, allows beneficiary communities direct access to donors and decision makers, increasing oversight and transparency in aid delivery and peacebuilding projects (Pinnington 2014; Arandel, Brinkerhoff, and Bell 2015).

Local ownership is an important goal for international agents, who want to ensure the adoption of programmatic goals by their intended beneficiaries. The success of peacebuilding programs rests upon some level of local cooperation. One of the main complaints about liberal peacebuilding has been the high levels of local resistance to peacebuilding plans devised by outsiders. Both parts of the term express a lack of clarity, with “local” encompassing everything from the smallest village to the national government and “ownership” meaning everything from total local control to what Mac Ginty (2008: 145) describes as “flat-pack” peacebuilding (cf. Adjei and Hancock 2020).

When peacebuilding recipients, or local peacebuilding organizations, believe that they do not have sufficient input into decision-making about peacebuilding planning and implementation, they tend to resist the execution of those plans, either by refusing to implement them fully or by bending them to their own goals, an outcome known as hybrid peacebuilding (Mac Ginty 2011). The prevalence of hybrid peacebuilding as an outcome of internationally planned, and ostensibly locally owned, peacebuilding plans leads to questions about both the goals and outcomes of these projects. To begin to address this problem, one area to think about in terms of local ownership is the question of its worth. Is meaningful local ownership only of value to international peacebuilders who wish to ensure that their projects meet programmatic objectives, or should it also be of value to local peacebuilding providers and, especially, to local beneficiaries? If the latter, then we need to think a little more broadly about the kinds of benefits that local ownership, sometimes characterized as local agency, can bring to these stakeholders.

A basic reason for developing mutual accountability systems stems from the nature of peacebuilding and conflict resolution work. In this work there is an inevitable tension between resolving issues and addressing problems of social justice to develop structures supporting sustainable peace (Clark and Coy 2015). In peacebuilding this tension can be seen in the desire to deliver effective peacebuilding services, while empowering ownership of peacebuilding projects, processes, and outcomes. It is not always easy to determine whether effectiveness or empowerment should be more important in each situation, but there is much to be said for an increased focus on empowerment. As Coy et al. put it:

The key to bridging the tension between effectiveness and empowerment is to focus on the ethos of peacebuilding. By focusing on empowerment, one is not rejecting the idea of best practices, but instead is interrogating those practices with respect to both effectiveness and empowerment, asking questions like how do our practices engender empowerment of the parties involved? How does the peacebuilding work respect the needs of stakeholders who are and are not at the table? And how do our peacebuilding projects and programs engender sustainable peace? (Coy et al. 2019).

An important reason for addressing ethos in peacebuilding work is supporting the need to get away from traditional forms of evaluation, which focus largely on the benefits of the intervention itself, and to move toward methods of evaluation that can engender empowerment and generate benefits in terms of social capital. Social capital is typically defined as the presence of information about and trust in one's social networks (Woolcock 1998). Scholars like Putnam (2000) and Skocpol (2003) note the importance of social capital to the functioning of democratic societies. Scholars of peacebuilding note that developing social capital can help to address sources of violence and to instill a democratic ethos among community members. They also note how mutual accountability mechanisms empowered local residents not only to hold peacebuilding providers accountable for their actions, but to develop social networks and skills that enabled them to build resilient communities and address other social problems (Arandel et al. 2015). Social capital is also noted as an important product, or byproduct, of local peacebuilding and violence prevention efforts in local zones of peace in places as far flung as Colombia, the Philippines, El Salvador, and Northern Ireland, where it can be seen as an outcome of deliberative processes and public participation that result in increased local agency (Hancock 2007, 2018).

Peacebuilding programs are typically funded by donor communities rather than by fee-for-service models. Positively, this provides beneficiaries with resources that they could not otherwise access. Negatively, this exposes them to forms of accountability that tend to minimize their agency and subject them to the needs of funders to ensure that the resources they are providing are being used appropriately (Richmond 2011; Donais 2012).

Standing athwart the concept of accountability is the practice of evaluation, a branch of research that has been active for decades and plays a central role in the fields of aid, development, and peacebuilding. The reliance on certain forms of evaluation is partly responsible for the lack of local ownership and agency by peacebuilding beneficiary communities and for the tension between top-down directives and bottom-up needs resulting in hybrid peacebuilding.

While this article will not include a complete review of evaluation practices, it is worth looking at both standard evaluation practices and some newer methods that have been introduced to improve evaluation practices and results. Following this, I will add issues of agency and procedural justice into the accountability mix and evaluation practices.

According to Rossi et al. (1999) there are three main flavors of evaluation: formative, summative, and process. Formative evaluations look to see if a project is meeting the needs of its beneficiary community as well as its own goals. When used as needs assessments, these evaluations are designed to determine whether a need exists in a potential beneficiary community and to help program officials determine how to best meet those needs. Process or cost-effectiveness evaluations are designed to test whether a project is meeting its own goals as regards to the processes it is supposed to be running, and whether the implementation of the program matches its plans.

Summative or impact evaluation is the main form of evaluation used to implement program accountability. It is designed to determine the extent to which a program has either met its own goals as laid out in project documents or needs assessments or the extent to which it has met the needs of the beneficiary community or had a positive (or negative) impact upon that community (Rossi et al. 1999). In practice what these kinds of evaluations do is to develop a series of project documents that are then reviewed and approved by the funding agency. The project documents can include things like the rationale for the service being provided and, most importantly, what is known as a theory of change, or the reason why the project implementer thinks that the proposed activities will have the effect that they are aiming for.

The structures of these various traditional evaluation forms tend to favor the continuation of accountability upward from peacebuilding provider to funding organization. Typical interactions with peacebuilding beneficiaries may be limited to data gathering processes where outside experts travel to the program locale and either conduct surveys or interview beneficiaries about their experiences with the programs undertaken. At times the kinds of data collected consisted of the number of workshops undertaken, individuals trained, or interventions conducted.

As peacebuilding has expanded, the difficulties in achieving desired programmatic goals have begun to creep in, encouraging the development and use of novel evaluation methods. Two popular methods are outcome mapping and outcome harvesting. Both start with the premise that it may be impossible to directly connect any specific changes taking place in target communities to the programs and activities being evaluated, instead working backward “to identify changes in attitudes, behaviours, knowledge and perceptions at the outcome level,” then analyzing “the extent of the intervention's contribution towards outcome level changes rather than attribution” (Corlazzoli and White 2013: 32; emphasis in original).

Outcome mapping is included at the outset in project design and systematically used throughout the project implementation period. It is divided into three main stages: intentional design; outcome and performance monitoring development—often using self-assessed “outcome journals”—and evaluation planning to identify priorities and develop an overall plan (Earl, Carden, and Smutylo 2001).

Outcome harvesting can be used in a post hoc fashion, allowing for more flexibility. It focuses on observable outcomes and then works backward to determine their connection to project activities. These outcomes are defined as observable changes in the “behavior, relationships, actions, activities, policies, or practices of an individual, group, community, organization, or institution” (Wilson-Grau and Britt 2013). They are collected both in real time and afterward. In traditional outcome harvesting, outcomes are collected by project staff or by partners.

Outcome harvesting consists of an iterative process, beginning with harvest design, then data gathering and drafting of initial harvest descriptions. These are followed by the harvesters engaging directly with “change agents”—members of the stakeholder communities—to refine outcome descriptions. Then the substantiation process engages with independent individuals, who are knowledgeable about the outcomes and use them to validate the credibility of the findings. This is followed by an analysis and interpretation of the findings to make recommendations and by meetings with the harvest users to determine how the findings will be used (Wilson-Grau and Britt 2013).

Both approaches rely upon stakeholder involvement in the planning and data collection portions of their development but are less connected to stakeholders during the evaluation processes. As such they incorporate a great deal more agency for stakeholder groups in the evaluation process than do traditional evaluation methods, but, for the most part, still leave much of the analytical understanding and decision-making in the hands of the evaluators and funders.

A key element in thinking about the intersection of social science theorizing and peacebuilding practice has been the application of motivational theories such as basic human needs or procedural justice to expand our understanding of peacebuilding practice and impacts. Two studies by Hancock focus on how an understanding of procedural justice can be applied to peacebuilding practice to better explain peacebuilding successes and failures. The first analysis argues that failures of local ownership take place because of a lack of understanding of the motivations driving local peacebuilding implementation organizations and beneficiaries to engage in resistance to international plans, leading to hybrid outcomes. In essence, Hancock argues that instead of just being a social good, local ownership is the expression of a basic need for agency, and like other basic needs posited by Maslow (1962), Burton (1990), and others (e.g., Max-Neef 1991), agency as a basic need “represents a universal constant” inasmuch as all people have basic needs and “will struggle to have those needs met, even if those struggles result in antisocial or violent behaviour” (Hancock 2017: 259).

In the peacebuilding realm one way to determine whether a basic need for agency is being met is to apply another theory of human motivation, that of procedural justice. Procedural justice is a theoretical paradigm drawn from work on courts and legal systems that focuses on the processes involved in the resolution of disputes rather than just the outcomes (Thibaut and Walker 1975).¹ Procedural justice also has echoes in the mediation movement dating back to the 1970s in the United States. Here the focus was also on justice, but much less so on retributive or distributive justice than on what later became known as restorative justice (Bush and Folger 1994). However, even with a focus on the restoration of relationships, pioneers in mediation understood that the process itself was as important, if not more so, than the outcomes (Moore 1986).

In essence, procedural justice shows that “people are often more concerned with their perceptions of treatment than with achieving specific outcomes” and are willing to accept suboptimal outcomes when the processes used have been deemed fair (Hancock 2018: 25). Procedural justice is made up of four main elements: the neutrality of the forum, trustworthiness of the authorities, treatment with dignity and respect, and opportunity for voice (Tyler 2000). Each of these is a key variable that can be measured and applied to both the development and delivery of peacebuilding programs, as well as to the accountability mechanisms used to evaluate their successes or shortcomings. They are also useful indicators for the development of both peacebuilding programs and evaluation models, because mechanisms for both of these arenas can be designed to fulfill these requirements alongside of technical requirements for the effective delivery of services and accurate evaluation of that effectiveness.

Hancock's second analysis focuses on one example of a peacebuilding organization whose functioning incorporated elements of procedural justice, the Suffolk Lenadoon Interface Group (SLIG), located in west Belfast, Northern Ireland (Hancock 2020a). SLIG, located along one of Belfast's many interfaces, sat between two historically hostile communities, Catholic-Nationalist Lenadoon and Protestant-Loyalist Suffolk. Because of the lack of trust between these two communities, the development of SLIG as an intercommunal peacebuilding organization took several years and was marked by high levels of consultation and transparency (Hall 2007). For most of its inception and early stages, all four indicators of procedural justice were present in SLIG's organizational structure and function. The only time when these indicators were absent was when SLIG was offered the opportunity to apply for millions of dollars of funding from Atlantic Philanthropies—in recognition of its high profile and excellent work. Unfortunately, much as with some of the special development areas in the Philippines (Avruch and Jose 2007), the infusion of massive amounts of funding destabilized the proposal process, creating mistrust between the two communities (Hancock 2020a).

There is a clear arc from the problems of liberal peacebuilding acting in a top-down fashion to the creation of community-based monitoring systems like community-based monitoring and evaluation (CBM&E). At the farthest end of that arc are those local peacebuilding instances described as zones of peace (ZoPs), first developed in the Philippines in the 1980s, spreading to Colombia, with analogues in places like El Salvador, Peru, Sri Lanka, Nepal, Sierra Leone, and Belfast (Hancock 2020b). ZoPs, first developed as places of sanctuary in the midst of violent conflict, rapidly transformed themselves into locally owned and locally driven peacebuilding programs, capable of addressing a wide variety of communities' needs. For the most part these efforts were largely free from external guidance or external constraints, with El Salvador's Local Zone of Peace going so far as to create its own funding arm. But in some instances, problems arose when ZoPs accepted funding from outsiders, whether governments or NGOs, and found themselves caught up in outside requirements or torn apart by the influx of new resources (cf. Avruch and Jose 2007; Hancock 2020a). Local ownership and control over the peacebuilding processes, especially broad community inclusion in governance and oversight, have contributed to the resilience of successful ZoPs and have helped to ensure their longevity (Mitchell and Hancock 2007; Hancock and Mitchell 2012; Hancock 2018).

Parallel to issues of local ownership in peacebuilding efforts, the development field has been grappling with issues related to the impact of funder-sponsored reporting requirements as well as the need to address issues of corruption and low return on donor investment (Galtung 1998). Both of these concerns speak to the need to, as Donais (2012) puts it, move beyond the top-down/bottom-up dichotomy of programmatic ownership in both peacebuilding and development. Galtung and Tisné (2009) argue that moving beyond accountability downward by including beneficiary communities in the accountability process can effectively address problems of corruption, while Arandel et al. (2015) showed how USAID nurtured a mutual accountability system in order to address issues of local governmental legitimacy.

CBM&E emerges out of the combination of these two approaches to community involvement in monitoring and evaluation. It is guided by Donais's (2014) call for vertically integrated peacebuilding efforts and Hancock's (2017, 2020a) recognition that successful local ownership requires that international agencies pay attention to the human motivations that drive beneficiary communities to need agency in the development, implementation, and monitoring of peacebuilding efforts. CBM&E emerges out of the application of the tenets of procedural justice and local agency to the monitoring and evaluation process. This application might best be thought of as inculcating a series of practices that are tied to indicators of procedural justice, which, when satisfied, lead to the development of local agency and productive peacebuilding. The relationship of these indicators to each other and to productive peacebuilding is illustrated in Hancock's model of deliberative peacebuilding. Here the four indicators of procedural justice are captured by practices of democratic governance, accountability down, and public participation (Hancock 2020a). As detailed in Hancock's examination of SLIG, the model aptly shows how deliberative peacebuilding is possible when strong links are forged between the peacebuilding provider/delivery agency and the beneficiary community. However, Hancock's analysis of SLIG, as well as numerous other examples, shows that true accountability down is difficult to obtain and maintain for many peacebuilding providers because of the drive to respond to funder initiatives and donor requirements; in essence it is quite difficult to expend energy and resources on accountability down mechanisms when the power relationships require tending to accountability up mechanisms (Figure One).

To address these power relationships, it is necessary to reimagine the conceptual model of deliberative peacebuilding to incorporate a different form of accountability as a method for including the concerns of funding agencies and their donors while still amplifying the voices of beneficiary communities and including the concerns of peacebuilding delivery agencies. So instead of substituting accountability down for accountability up, we need to develop a mutual accountability mechanism. The first step in this process was to develop a mutual accountability framework that would link together the beneficiary community, peacebuilding implementing or delivery organization, and other stakeholders such as governmental officials, international officials, and funding representatives.

In this context the other stakeholders include members of the UN Secretary General's PBF, the UN Country Team (UNCT), and the country's Joint Steering Committee (JSC), among others. Figure Two below shows how these three are connected by a mutual accountability system we are describing as CBM&E.

Conceptually CBM&E has much in common with the ideals of deliberative peacebuilding. However, it has a narrower focus on accountability streams as well as a wider focus through its inclusion of funding agencies and other peacebuilding stakeholders in the process. Figure Three shows how CBM&E's practices of indicator development, community data gathering, and collaborative evaluation fulfill the elements of procedural justice by providing neutrality of forums, trust for authorities, treatment with respect for beneficiary community members, and most importantly, opportunity for voice to beneficiary community members as well as members of the peacebuilding implementation or delivery organization. The fulfillment of the tenets of procedural justice creates systems that lead to beneficiary empowerment, and thus to the perceived legitimacy for the delivery agency and, finally, sustainable peacebuilding.

Naturally enough, the connection between CBM&E and sustainable peacebuilding—and sustainable peace itself—is not as linear as laid out in the conceptual framework. However, the ultimate goal of sustainable peace rests upon the inculcation of a vibrant civil society wherein community members are empowered to participate in their own governance.

The CBM&E model laid out above in Figure Two shows how its tools of indicator development, community-based data gathering, and collaborative evaluation bind together the beneficiary community, implementing organization, and other peacebuilding stakeholders.

One typically thinks of indicators in connection with project or program results frameworks that are identified by implementing agents when an intervention is being designed. While these types of indicators may help recipients track some aspects of implementation, they usually are less helpful in determining whether the intervention is meeting the needs or goals that conflict-affected populations would prioritize. This is a well-known problem identified by analysts who argue that such data reflect the priorities and experiences of international agencies or donors rather than those who live in conflict or post-conflict situations (Fast 2017; United Nations Peacebuilding Fund 2022).

To correct this problem, Pamina Firchow and Roger Mac Ginty argue for the inclusion of so-called everyday peace indicators (EPI), which are datapoints developed by people in conflict-affected or post-agreement situations that measure agreed upon and uniform measurements of positive or negative social change (Firchow and Mac Ginty 2017). The methodology seeks communities' input on the goals of peacebuilding projects as well as the development of indicators to use to evaluate them. By placing indicator development before project development, the EPI model interrupts the tendency of international peacebuilding funders and agencies—even when in consultation with national governmental officials—to set peacebuilding goals and objectives. CBM&E somewhat alters this replacement of top-down goal setting with bottom-up goal setting by working within a larger framework laid out by the PBF. As described in more detail below, CBM&E is designed to be used in instances where a country has requested eligibility to the PBF's Peacebuilding and Recovery Facility (PRF), which grants a five-year window of time to access large amounts of funding that can be “projectized.” As a part of that process, the PBF and national government develop what is known as a Strategic Results Framework (SRF), an overview of national goals and objectives along with high-level indicators for measuring their impact and effectiveness (United Nations 2021). The local objectives and goals developed by CBM&E's indicator development process are designed to be harmonized within the SRF's overarching objectives, and the outcomes of CBM&E's collaborative evaluation process (described below) will provide a base of data to the SRF-level evaluations, contributing to broader and deeper levels of data, while at the same time valuing local objective-making and harmonizing it with national-level objectives developed by the SRF process.

Indicator development consists of several discrete steps that must be implemented over time, thus it needs to begin early in the project cycle. EPI founder Firchow recommends that indicator development start before local partners have been chosen or hired.² Indicator development processes begin with the hiring or training of an indicator development team. This team may have national or international members, or both, but it requires members who are familiar with the locale in which the work will take place and able to understand local customs and vernaculars.³

After establishing contacts and gaining entry to the locale, indicator development begins with a series of focus groups, ideally stratified along important divisions such as age and gender. Other divisions such as ethnic affiliation or geographic location may also be salient. Initial focus groups are designed to elicit the beneficiary community's views on peace indicators, problems, and needs. Following these focus groups, the indicator development team will need to produce and analyze a transcript of the meetings to develop a unified sense of what the beneficiaries think are their main problems, community goals, indicators for success, and potential obstacles. Once these have been developed a subsequent round of verification focus groups should be held with a cross-section of those in the initial, stratified groups. The members of this focus group will be presented with a list of options, and the indicator development team will then select those that resonate the most with the group as the indicators to be used for data gathering. These indicators will then need to be coded into categories to aid in setting peacebuilding priorities, tracking changes over time, and for use in final community evaluation processes.

Once indicators have been developed, the beneficiary community is organized to collect data on those indicators on a routine or periodic basis. Data collection can take any number of forms, including surveys, interviews, observation, or focus groups. Data can include reflections on peacebuilding projects or activities, impacts of those activities, or other positive or negative changes to the social situation.

There is no one best way to collect data, and it is recommended that more than one method be implemented to achieve triangulation. Multiple methods have been used to collect field data including some of the most tried and tested as well as newer, more creative approaches, including:

• individual and group interviews,

• survey instruments, whether electronic or paper-based, and in-person or remote,

• focus groups, stratified or mixed,

• structured observation,

• reflection journaling, and

• participatory visual methods, such as photovoice, picturevoice, paintvoice, and comicvoice.⁴

The selection of individual data collection methods should rely upon asking and answering a series of questions about the context and kind of project being evaluated, the kinds of things that one wants to know about that project, and the kinds of populations with which one wants to interact. Oftentimes researchers might be familiar or comfortable with certain methods and will tend to use those methods to the exclusion of others, even to the extent of framing their research questions based on the methods that they wish to use. Instead, it is strongly suggested that these questions be placed at the forefront and that the answers drive the kinds of methods selected.

The idea of collaborative evaluation processes is rooted in participatory action research and draws from it in two ways. The first is in the participatory aspect, understanding that to identify peacebuilding needs as well as potential solutions, the community needs to be a part of the process from an intervention's inception (Lykes and Crosby 2014). The second is in reference to how research connects with action. In both action research and participatory action research, a project's research goals derive from the kinds of social impacts and social changes that drive the work. Both research methods rely upon a range of collaborative processes between outsiders—typically a researcher of some sort—and those in the community. Collaboration begins at the outset by identifying the problems or needs of the community, drawing up the intervention program to address those needs, and finally analyzing and reporting the results of the intervention.

Collaboration takes place in all aspects of the research process, from problem identification to project development, data gathering, and evaluation report writing. The main rationale for this is the same as for the ethos of peacebuilding described above, to empower members of the beneficiary community by conducting research that is democratic in nature, equitable to the beneficiary community, liberating in the sense of freeing them from oppressive structures, and life enhancing in the sense of facilitating their full human potential (Stringer 2014; United Nations Peacebuilding Fund 2022).

Collaborative evaluation practices are also rooted in the field of deliberative and small group democracy, which focuses on the ways that small groups can meaningfully involve all stakeholders in analysis and decision-making around important or contentious topics (cf. Chambers 2003; Gastil and Levine 2005; Thompson 2008). Collaborative processes are centered around the transparent use of power and the use of consensus-type decision-making processes, wherein all stakeholders need to come to agreement in order for any decision to move forward (Gastil 1993). These types of processes have been used in a variety of settings, including environmental dispute resolution, development planning, and other forms of conflict resolution, but there is little evidence that they have been used in either the peacebuilding or program evaluation fields (Wondolleck and Yaffee 2000; Linden 2002; Snyder 2003).

For CBM&E, these practices can take many forms, including small group discussions, focus groups, study circles, town hall-type meetings, fishbowl discussions, and many others. In the kind of international peacebuilding locales that the PBF funds, it is recommended that collaborative evaluation sessions be adapted to the local culture, or that local cultural practices be adapted to include elements of collaborative evaluation in order to ensure familiarity and legitimacy while attending to potentially illiberal aspects of indigenous approaches to peacebuilding (Mac Ginty 2008).

CBM&E is envisioned as both an approach to encourage and facilitate agency among peacebuilding beneficiaries as well as an approach to address concerns about a dearth of rich data and understanding about how peacebuilding interventions are perceived by conflict-affected beneficiaries. At the time of this writing, however, CBM&E remains a set of theoretical concepts that require field testing to collect the empirical data that will tell us whether and how the application of these concepts is borne out in the results. However, given that many of CBM&E's practices derive from other sources, we can, to some extent, examine how these different elements have operated under other conditions and extrapolate, again to some extent, their adaptability to the context of an overarching program like CBM&E.

In terms of community input into program goals, there are several places where we can see everything from total community control over project choices and objectives to more structured forms of input into goals and programs designed by others. Zones of Peace, frequently operating separately from international peacebuilding structures, often have the most complete control over objectives and programmatic goals. This includes everything from the Constituent Assembly of Mogotes requiring candidates for Community Manager to receive complete approval for their peacebuilding plans from community members (Hancock 2018) to the insistence of total control over the use of incoming funds by peace communities in El Salvador and Colombia's San Jose de Apartadó (Hancock 2007; Burnyeat 2017). More mixed models, where locals have a large, but not total, say over the development of peacebuilding goals are to be found in the UN-funded projects in Somalia—Midnimo (unity) I and II—which used large Community Action Groups to develop action plans and set goals for the peacebuilding projects, and EPI's indicator development strategy (described above), which is premised upon developing projects based upon community-derived needs and goals (United Nations 2017). Beside these more successful models of community-led goal- and objective-setting we can place the more confused outcomes of the Suffolk Lenadoon Interface Group's attempt to develop a plan to spend the close to $6 million offered by Atlantic Philanthropies. Prior goal development processes with high levels of intercommunal cooperation were replaced by ad hoc intracommunal meetings, followed up by a more constrained joint meeting where the two plans were integrated. This disconnect between the two communities, like the awarding of special development funds to the Filipino peace zone in Tulunan, fractured hard-won community cohesion, reminding us that community inclusion and cohesion are important components for any kind of objective or goal development process for beneficiary communities (Avruch and Jose 2007; Hancock 2020a).

There are several examples of community-based data collection, including some official programs that were specifically designed to include beneficiary community members in the data collection process. Like the development of program objectives and goals, community-based data collection can range from informal to formal and official processes. At the informal end, the close connection between community members and the staff at SLIG meant that when the community needed services, such as suicide prevention, they were able to access staff easily and tell them of these needs. Alongside the use of town hall-type meetings and routine surveys, this close connection meant that community needs and perceptions were readily accessible to staff members (Hancock 2020a). In Niger, the CSO Search for Common Ground developed a plan for creating Community Monitoring Committees (CSDs in French) in 68 target communities to be used in the UN's priority plan for peacebuilding in that country.⁵ The CSDs were to be responsible for data collection, with members receiving training in the use of collection tools, especially smartphones, which would then transmit data to central representatives (Moutari and SFCG 2017). It is unclear the extent to which CSDs have been implemented in Niger's peacebuilding projects or programs, though there are indications from PBF reporting documents that there are plans for local project monitoring committees to be set up for conflict prevention and resilience programs⁶ as well as for a specific program focusing on youth activism in supporting peacebuilding.⁷ The most complete example of community monitoring comes from the UN's Community-Based Monitoring (CBM) process, developed in a 2020 guidance note and implemented in Guatemala, where it was integrated into a project focused on building social cohesion in communities receiving young returnees (United Nations 2020b).⁸ During this project, ten young people from the affected communities were selected to conduct data collection and preliminary analysis, carrying out more than 200 surveys alongside interviews with public officials, NGO workers, and others (United Nations 2022). The project's final evaluation noted that CBM was “valued as an innovative strategy,” that the young people who participated in data collection were able to “empower themselves and recognize themselves as leaders,” and that the process provided valuable feedback to the project coordinator (United Nations 2022). With the exception of SLIG's informal methods, the main difference between CBM&E and these methods is their reliance upon indicators provided by program staff or international funders. This perceived shortcoming was one of the rationales for the more beneficiary-oriented approaches of EPI and CBM&E, both of which are described as less extractive due to the inclusion of the beneficiary community in program planning and objectives, and not just data collection efforts.

Community involvement in data analysis and evaluation is the key element that sets CBM&E apart from many of the other community-led monitoring and evaluation systems used in peacebuilding programming. The closest analog to the kind of community-based evaluations and assessments proposed by CBM&E's mutual accountability model comes from community-driven accountability efforts made by groups most focused on anti-corruption efforts in developing states. Galtung and Tisné (2009) assessed a number of community-led anti-corruption efforts in Afghanistan, Timor-Leste, and Congo, noting that where these community-driven initiatives are able to partner with local governmental officials, they have managed to collaborate in corruption reduction rather than engage in conflict-enhancing behaviors. They conclude that community-driven accountability mechanisms help to “restore trust in war-torn societies” and “can balance citizens' expectations with states' constrained ability to deliver needed services” (Galtung and Tisné 2009: 107). This dovetails with findings from the Midnimo projects, which showed that inclusion in the monitoring and evaluation process resulted in higher levels of satisfaction, even when insufficient resources existed to complete projects as desired.

It is beyond the purview of this article to predict when and how a CBM&E pilot test will take place. However, initial requirements for a successful pilot revolve around three main conditions. The first of these is the necessity for funding and support over an extended time period. Because CBM&E is an extended evaluation process, and because the EPI indicator development process can be implemented both at the beginning of a project and at the midway point, the implementation of CBM&E requires a multi-year commitment—typically a minimum of three to five years. In the UN context, this would require that recipient countries qualify for the PBF's PRF funding scheme described above (United Nations 2020a). The second is that the country or program would need to develop a dedicated team to oversee the CBM&E program and to coordinate the activities of all of the various component groups making up the program, including indicator development teams, data collection groups, and community evaluation facilitators. In the UN context these functions would be the responsibility of the country's PBF Secretariat, whose members would coordinate with CBM&E team members, community members, peacebuilding delivery organizations, government officials, and UN Headquarters staff. The third requirement is that CBM&E needs to be implemented in accordance with the principle of do no harm (Anderson, Olson, and Doughty 2003). This requires the ability to negate or mitigate any potential adverse effects that might increase a population's vulnerability to either physical or psychosocial risks. CBM&E is designed to “flatten” hierarchies of power, giving members of the beneficiary community access to—and voice within—decision-making forums. Unless there is clarity about CBM&E and support for power sharing and equality among all stakeholder parties, implementing CBM&E can run the risk of lower power stakeholders facing backlash from higher power stakeholders. In these instances, lower power stakeholders may feel constrained from fully participating or voicing their concerns, negating one of the main benefits of CBM&E, its ability to amplify and consider the voices of those usually neglected in the halls of power.

When I joined the monitoring and evaluation team at the PBF and was tasked with the development of what later became CBM&E, it was with the explicit goal of developing a community-level monitoring and evaluation system that would foreground the agency of beneficiary communities and engage them in a partnership. Because CBM&E was explicitly designed to be inclusive rather than extractive, and to be a mutual accountability system, incorporating local voices from the village to the national capital, it was understood from the outset that decision-making about programmatic goals and evaluation methods would have to be distributed among the different levels of the CBM&E architecture. The original contribution of CBM&E is the shift in mindset that it will require of those in positions of power—a reenvisioning of their accountability relationships with other members in the CBM&E network. In essence, these individuals are being asked to shift their mindset from being responsible for outputs to being responsible for inclusive processes, and to trusting in those processes to provide the kinds of positive outputs that will satisfy both the beneficiary community as well as donors in northern capitals. This is a significant challenge, and the fact that an organization as central to international peacebuilding as the United Nations has developed this process speaks well for the future of peacebuilding, but as with any other new initiative, the proof will be in the implementation.