Order or Chaos? Evaluation needs both
In order to explain the concept of complexity, professor Ralph Stacey developed a very convenient tool, the Agreement Certainty Matrix. It situated complexity between the concept of order and the concept of chaos. Order assumes high agreement and certainty about what is going on, while chaos comprises situations in which uncertainty rules with agreement and certainty both low. Stacey presents complexity as a hybrid concept, which capacitates it for bridging between order and chaos, modern and postmodern understanding of the world, as well as between facts and values.
The mainstream schools define complexity in incompatible frames (Castelani). The first understands it in a disorganised state, located ‘on the edge of chaos’. It addresses the lowest level of description of a larger system with randomly behaving individual parts. Global structures emerge unpredictably exclusively by means of the local interactions of individual components. Disorganised complexity is typically studied with formal language and by quantitative means such as in theories of informational entropy, or with theories of computational complexity.
The opposite is the school of organised complexity which locates its object of concern ‘on the edge of order’. In this case, complexity emerges from a moderate number of macroscopic variables that are related to one another in interdependent ways, such as between billiard balls or between domains of sustainable development: economic, social, and environmental. The concept of organised complexity has found its place in the philosophy of science and in the qualitative methods of social research, among others.
Each school develops exclusively, and often neglects or ignores other schools of thought. Polarisation contradicts the very meaning of complexity as a bridging principle (Malaina). Complexity cannot be properly understood in binary terms, nor with binary means, but with mesoscopic ones — partly ordered and partly disordered — by application of partly systemic and partly antisystemic logic. In turn, systems and complexity do not belong to the same level of generality. The theory of complex systems contributes very relevant and indeed indispensable insights into many aspects of social complexity but only as a special case. Complex systems cannot explain genuine complexity: fundamental coalescence of the rational and the irrational.
If so, then the evaluation of complex matters needs to be theorised as a mesoscopic undertaking, integrating systemic and chaotic reasoning at the middle level. On the one hand, evaluation is undoubtedly a rationally systemic effort with a distinctive logical frame, methodology, and standard procedures. On the other, it is also value-based, ‘irrational’, and chaotic when it seeks to include incommensurable stakeholders in participatory evaluation in the search for some coherent understanding from incompatible grounds. Mesoscopic methodology is integrative precisely because it is evaluative: it is based on scientific evidence without being dominated by scientific (systemic) reasoning (Radej). Evaluative and mesoscopic reasoning are identical.
So what advice can complexity research give to the policy impact evaluator about good practice? It seems to advise an integration of systemic and participative approaches to evaluation by applying a mesoscopic methodology that relies on both:
- Systemic step: The evaluated matter is first defined as complex at the meso level, which is at the intersection between foundational contradictions, the vertical and horizontal axes of development from which a given complex matter has emerged.
- Participatory step: Stakeholders then convene to identify horizontally three main domains of a given complex matter on the meso level, such as the physical, economic, and social domains of territorial cohesion (EU); or education, research, and innovation in the triangle of knowledge (European Institute of Innovation and Technology), etc. This is the constructivist part of the evaluation that invites creative design thinking, for instance with the three-part Venn diagram.
- Systemic step: The evaluator continues by rigorous analysis of the impacts of the evaluated phenomenon on three domains and their cross-sectional, secondary links in the meso-matrix — a tool that locates synergies between domains through their correlations (Radej).
- Participatory step: The evaluation concludes with stakeholders discussing synthesised findings and evaluatively interpreting them in the pursuit of shared understanding.
It seems that the evaluation of complex matters does not demand very sophisticated interventions into standard evaluation procedures. Nothing really new is required except application of a consistent approach to connecting several compatible features of conceptually opposite approaches to evaluation at the meso level, where order and chaos can naturally overlap. Complexity is after all only a specific ordering principle. It produces simplicity, order from disorder, only in a characteristically uncharacteristic, and unpredictable but nevertheless surprisingly reasonable way.
Sources:
Castellani B., F. W. Hafferty. 2009. Sociology and Complexity Science. Berlin, Springer.
Malaina A. Two complexities: The need to link complex thinking and complex adaptive systems science. Litchfield Park, Emergence: Complexity and Organization, 17/1(March 2015).
Radej B., M. Golobič. 2021. Complex Society In the Middle of a Middle World. Vernon Press. Presentation: https://www.linkedin.com/pulse/social-complexity-complex-society-middle-world-bojan-bojan-radej/
Stacey R.D. 2011. Strategic management and organisational dynamics. The challenge of complexity to ways of thinking about organisations. Sixth Edition. Harlow, Pearson.
*** Discussion: LinkedIn Group: Complex Public Governance, https://www.linkedin.com/groups/8522366/ ***
