Bias in participatory evaluation: friend or foe
The post incisively critiques current participatory evaluation methods, exposing their false neutrality and inherent biases that merely perpetuate exclusionary power structures. It advocates for a paradigm shift towards an “empty middle” approach, emphasizing indeterminate judgments over logocentric or relativist judgments. The central argument is that participatory evaluation must shift its foundation from a logocentric emphasis on truth-based neutrality to neutrality achieved through indeterminate judgments. This shift aims to promote genuine inclusivity, ensuring that evaluations serve the broader populace rather than a select minority (donors, policymakers, or project leaders). Such a transformation is imperative for achieving collectively rational assessments of complex interventions targeting social transformation and sustainable development.
The Empty Middle as a Foundation for Evaluating Complex Interventions
The empty middle is a pioneering concept in evaluation theory, signifying an epistemic space that transcends polarized, logocentric assessments. Drawing from Eastern philosophies like Buddhism’s Middle Way and Western thinkers such as Heidegger, Sartre, and Derrida, this approach acknowledges the significance of voids or gaps in comprehending complex phenomena. By engaging with the empty middle, evaluations can transform inherent biases and blind spots into constructive elements for synthesis. Rather than anchoring participatory evaluations of complex policy interventions to predefined notions of truth or succumbing to relativism, this paradigm embraces the fundamental indeterminacy of interventions, treating bias as an epistemic resource rather than an impediment.
The conceptual framework of the empty middle is particularly relevant for the evaluation of complex interventions, where neither pure rationality nor unstructured participation can fully capture the dynamics at play. While participatory evaluation seeks inclusivity and collective rationality, it often fails to achieve genuine neutrality due to inherent biases. This paper examines four participatory tools — Most Significant Change (MSC), Causal Mapping (CM), SenseMaker (SM), and Outcome Harvesting (OH) — assessing their effectiveness in addressing these biases. The four tools typically attempt to suppress or bypass biases rather than engage with them productively. Although designed to incorporate diverse perspectives, they frequently fall short due to methodological inconsistencies, selective aggregation, and an inability to capture the indeterminate realities of a multiplural world. Moreover, their weak theoretical foundations often result in either a refusal to synthesize fragmented evaluation findings or a tendency toward overgeneralization of its fractional conclusions.
The empty middle offers a novel framework for participatory evaluation, enabling biases to converge and be addressed more inclusively and rationally. By situating evaluations within this space, participatory methods can transition from merely suppressing or compensating for biases to actively engaging with them. This approach acknowledges the coexistence of order and chaos, fostering a more comprehensive understanding of complex interventions.
Addressing Bias in Participatory Evaluation
Bias is an inevitable feature of participatory evaluation, yet mainstream approaches often mishandle it. Realist epistemologies view bias as a defect to be corrected, while constructivist epistemologies consider it an integral part of socially constructed knowledge that should remain unchallenged. Both perspectives fail to meaningfully engage with the complexities inherent in participatory processes. The empty middle framework offers a mesoscopic alternative by introducing the concept of double exposure of bias. For instance, in sustainable development contexts, the economic biases of ecologists and the ecological biases of economists, when intersected, reveal a further void — the social dimension of sustainability — that neither perspective alone can address.
Rather than striving for absolute neutrality through double-blind techniques — which triggers exclusionary effects — the “double exposure” approach is proposed in participatory evaluation. It positions biases in direct opposition, allowing their interplay to be observed and understood. In this framework, the evaluator’s role is neither to eliminate nor to acquiesce to biases but to cultivate an environment where these biases can neutralize each other through their interaction.
Conventional participatory evaluation methods often attempt to eliminate bias through rigid methodologies that paradoxically introduce new biases. For instance, CM addresses confirmation bias using robustness checks, including cross-validation of biased participatory contributions with evidence-based arguments and expert knowledge. However, these checks can undermine the democratic function of a participatory process. The primary goal of democratic processes should not be to compete with or even supersede empirically derived knowledge, but to legitimize rational arguments. Similarly, evidence-based knowledge prioritizes truth-seeking, serving to inform participatory processes ex-ante rather than validate their outcomes ex-post. If the verification process is inappropriate, its outcome — bias mitigation — will necessarily be biased.
Serving the Excluded Majority Rather Than the Privileged Few
A key limitation of conventional participatory evaluation approaches is their tendency to serve the interests of a privileged minority rather than the excluded majority.
Mainstream inclusion efforts focus on minority representation (e.g., disabled, ethnic, and economically disadvantaged groups) while ignoring the systemic exclusion of the broader population. However, studies indicate that 60–80% of people worldwide feel politically unrepresented, highlighting a paradox in participatory methodologies: they amplify specific marginalized voices while overlooking broader systemic exclusions. The dominant logic of inclusion assumes that representation within existing structures equates to genuine participation. Yet, as Foucault has shown, marginalized groups often strive to achieve a state in which they are no longer excluded more than the majority of the population. This merely reinforces systemic inequalities. Despite their democratic intent, participatory methods frequently reinforce existing power dynamics. They fail to account for the structural exclusion of the included majority.
The empty middle approach addresses this by integrating perspectives not through synthetic consensus but by navigating the intersections of different exclusions, revealing new insights. As a part of this strategy, evaluation must exclude excluders — those who systematically undermine inclusive deliberation. Drawing from Rawls’s justice theory and Mouffe’s agonistic democracy, it suggests that certain actors (e.g., hate groups, misinformation campaigns, and those who reject democratic dialogue) should be excluded not for their views but because their participation obstructs collective sensemaking. They are not excluded from evaluation but from the synthesis of participatory contributions, which is necessary to safeguard the integrity of democratic deliberation.
Practical Implications and Conclusion
The shift to an empty middle evaluation framework has significant implications for practice. It challenges evaluators to:
- Reframe Neutrality: Move beyond the illusion of objective neutrality and embrace neutrality as an emergent property of bias interplay.
- Recognize Bias as a Resource: Instead of eliminating bias, evaluators should map its intersections to reveal underlying patterns of exclusion and epistemic blindness.
- Prioritize the Majority Over the De/Privileged Few: Ensure that participatory evaluation is structured around the needs of the systematically excluded, rather than the stakeholders on the margin.
- Reject Absolute Aggregation: Avoid reductionist synthesis methods that favour predefined indicators and embrace iterative, multilevel synthesis in the empty middle.
- Enforce Selective Exclusion: Prevent actors from undermining inclusive deliberation from shaping evaluative conclusions while maintaining openness to opposing viewpoints within a democratic framework.
In sum, that participatory evaluation must undergo an anti-postmodern turn — one that recognizes the limitations of both rationalist and constructivist approaches while embedding itself in the indeterminate space of the empty middle. The empty middle provides a new foundation for participatory evaluation, enabling a more inclusive and collectively rational approach. By engaging biases productively and prioritizing the needs of the excluded majority, this framework seeks to transcend traditional limitations and advance genuine social inclusion and democratic decision-making. Neutrality in participatory evaluation is best achieved by recognizing and engaging biases within the framework of the empty middle, resulting in more effective and fair evaluations of complex interventions. In this way, evaluation becomes a tool for uncovering new forms of collective rationality — not imposed from above but emerging through the structured interplay of multiple biases within an epistemic void that enables truly inclusive deliberation.
Language editing: ChatGPT
Summary of the Working Paper of the Slovenian Evaluation Society 1/XVII(2024): https://www.sdeval.si/2025/01/08/inclusive-or-rational-participatory-evaluation/
