1



Meso-Matrical Synthesis of the Incommensurable #

Bojan Radej*

Abstract: Many of those currently involved in the assessment of government interventions’ contribution to the social welfare have had significant difficulties in summarizing known but sometimes contradictory facts into summary conclusions. There are incommensurable viewpoints with regard to many social realities, and they provide us with very different ‘numeraires’ and macro-views of the world which are not reducible to common denominator. In particular, there is a disagreement over assumptions about the aggregation of the assessed policy impacts (micro) into summary conclusions (macro) that inform decision-makers operating at meso level. A new method is proposed of meso-matrical impact assessment (MIA) of policy interventions to cope with social incommensurability in scale (micro-meso-macro) and scope (economic-social-human-natural). It is based on Leontief's square input-output matrix that evaluate overlaps between incommensurable sets on their margin. In MIA secondary issues play a central role. Practical example illustrates the achievement, schematic presentation generalises the principle. Paper explains that social or political common ground is not the condition of unity and cohesion with the meso-matrical perspective in mind.

Keywords: Incommensurability, scope, scale, meso, matrix, evaluation.

JEL Code: A13, A10, F12, A19, C50, H40

# Revised 15. December 2008; submited in October 2008.

* Bojan Radej, MSc in economics; independent social researcher; Slovenian Evaluation Society; Ljubljana, Slovenia, bojan.radej@

1. Introduction

Sustainable development is a norm that obliges governments to put forward policies that ‘meet the needs of the present generation without compromising the ability of future generations to meet their own’ (WCED, 1987). This calls for multi-criteria assessment of policy interventions which significantly complicates summative evaluation. There are incommensurable viewpoints and values with regard to different social realities, such as economic, social and environmental, or local and global, which provide us with very different ‘numeraires’ and views of the world, and they are not reducible to one common denominator (Funtowicz and Ravetz, 1994, in Martinez-Alier et al., 1998).[1] For the fact that variegated forms of welfare need to be taken into account implies that ‘different principles of social primacy and legitimacy must be reckoned with and reconciled’ (Wacquant, 1997) in a multi-criteria impact assessment (IA).

Incommensurability is not accompanied only by value comparisons but also by scientific ones since its formal systems decisively differ in the propositions they take as their axioms. When confronted with multidisciplinary issues involving scientific controversy, even competent, honest and disinterested scientists may arrive at different conclusions because of systematic differences in the way they summarise available information of one and the same world (Kuhn, 1970; Mumpower et al., 1996). Kuhn (1993 in Sankey, 1998) concludes that theories and scientists are unable to truly reflect reality. When values and facts are based on incommensurable oppositions, there is no objective basis for rational choice between theories and than no neutral observation of social reality is possible. No individual person or theory is than able to fully comprehend social complexity – such as sustainable development - and its multiple meanings are persistently disagreed collectively. This alleges that multiplicity is one of the determining conditions not only of values and facts but also of social systems (Munda, 2004).

The current welfarist model is on the other hand unable to conceptualize and operationally apply substantively different aspirations in a public sphere. Bar-Yam (2003) thinks that governments have experienced systemic failures in addressing complex welfare issues such as large-scale and multi-scope matters. They rely on reductionist approaches, which simplify the welfare concern by dividing it into independently ministered sub-problems. The very act of simplifying by sub-divisions loses the interconnections and therefore cannot tackle overlapping concerns (Chapman, 2001). The analytical concept of reality, is for example based upon the idea that the total is equal to the sum of its parts and that the parts can be quantitatively described, is insufficient for studying complex issues. A complex phenomenon is a structure of sets, connected by ordering relations of ‘super-, sub- and co-‘, where each sub-system (itself a system) has a plurality of relations of all three sorts with other subsystems (Ravetz, 2006). Reasoning on different levels is discontinuous and transgression between them (Weaver, Rotmans, 2006) breaks linearity. For complex social considerations, where ‘sum differs from the aggregate of their parts’ (Veen, Otter, 2002), the summation procedure is far from trivial (Veen, Otter, 2002), so there is an aggregation problem (Foster, Potts, 2007) in the summative IA. For sustainable development to flourish, an approach to welfare considerations is needed that is summative in a way that is equally sensitive for all incommensurable aspects.

Existence of an aggregation problem in policy studies is probably less obvious than the vast consequences it manufactures. In policy-making even good individual policies, based on strong values and common sense, often lead to disappointing results. Inability of a government to coordinate and aggregate opposite claims can be hold responsible for generating growth antagonisms since 1960’s such as income inequality (Giraud, 1996; Milanović, 2006), and happiness paradox (Easterlin, 1974) where increased material wealth is not accompanied by a subjective satisfaction; an ecological footprint (Wackernagel, Rees, 1994) shows that the wealthiest nations actually consume resources at a volume that could be sustained only if the planet Earth were two to almost six times larger. It was not that growth antagonisms took place in the absence of policy evaluation. Evaluation has increasingly obtained its role in a policy cycle since the sixties, but it failed to address deep conflicts that accompany large-scale and multi-scope policy interventions. So ‘ill development’ took place with our informed knowledge. Thus the question how to think intelligently about the conflict nature of trade-offs between different aspects of welfare that are characterised by the incommensurable oppositions (Williams, 1972, in Martinez-Alier et al., 1998) arises again.

Arrow explained using his impossibility theorem (1951, Nobel Prize for Economics in 1972), that it is not possible to scale up from all individual preference functions to produce a social welfare or “public interest” function that satisfies desirable properties of an aggregation process (Evans et al, 2002) such as non-dictatorship. The theorem has challenged the presumption that an aggregation of individuals’ wants such as in Condorcet’s majority voting rule or Bentham’s ‘the largest utility for all’ can simultaneously meet individual and collective expectations. It further demonstrated that there is a discontinuity between rationality at the individual and group level (Evans et al., 2002) that mirrors the tension between private and public in the governance of the social. Coleman (1986 in Åberg, 2000) maintains that this micro-to-macro link, also referred to as social causation (Sawyer, 2003) is a controversial and most poorly developed part of sociological theory.

Associated with these meta-theoretical concerns, there is an apparent paradigm crisis in IA (Virtanen, Uusikylä, 2004; Hertin et al, 2007). IA aims to suggest more consistent policy interventions (Hertin et al, 2007). However, evidence shows that policy response to evaluation recommendations is very limited (Picciotto, 1999). Existing IA methods are designed for the appraisal of homogeneous interventions or ‘projects’ with only limited diversity of impacts (Elbers et al., 2007; Rotmans, 2002), while governments work with heterogeneous issues. IA conventionally forwards two-valued or binary logic which is focused on only two scopes of the assessed phenomena - cause and effect -, such as economy’s impact on environment. But government operates in multi-value context, where causes and effects are not straightforward. The problem is also that majority of IA models assess processes only at one scale level, micro or macro. This all results in a ‘evaluation deficit’ – the situation in which most assessments provide the kind of information that does not inform policy-makers whether a global objectives can be met – consequently, evaluations remain under-utilized (Stame, 2004) or even get mis-used.

Summation is the Achilles heel of the evaluation effort (Scriven, 1994). It arises from a disagreement over assumptions about the aggregation of numerous policy impacts when assessment phenomenon is multiple in scope (economic, social…) and scale (micro, macro). As elaborated by Carlsson (2000), different evaluators assessing the same policy interventions may come to opposite conclusions solely because of differences in their assumptions about the summation of the assessed impacts across incommensurable scopes and scales. We often think of them as coupled because of the most common ways in which we encounter them (Bar-Yam, 2004): consider observing a system through a camera that has a zoom lens. For a fixed aperture camera, the use of a zoom couples scope and resolution (scale) in the image it provides. “As we zoom in on the image we see a smaller part of the world at a progressively greater resolution. This leads to a particular relationship of observations of parts and wholes, suggesting that when observing details of the system, the whole is not being observed. We must allow a decoupling of scope and resolution, so that the system as a whole can be considered at differing resolutions as well as part by part. For this purpose scale can be considered as related to the focus of a camera—a blurry image is a larger scale image—whereas scope is related to the aperture size and choice of direction of observation” (Bar-Yam, 2004).

For evaluator this means that what one sees is always predefined in scope and scale of his or her observation. Recognition of this is central for the methodology of meso-matrical impact assessment (MIA) that is proposed here. It builds on previous observations that to summarise the trade-offs between intrinsic sustainable values, evaluator needs to derive conclusions across different scales and scopes at which appraisal takes place (Weaver, Rotmans, 2006; Dopfer et al, 2004; Dopfer, 2006; Easterling, Kok, 2002). As Chapman (2001) elaborated: the theory of complex systems offers an alternative to conventional IA strategy for studying large-scale issues: instead of going down to the elementary level as in the standard reductionist model, the system theory justifies the opposite, namely going up a level of abstraction which establishes a multi-scale view.

The aggregation problem in IA that apply the impact matrix method has been first addressed by Luna Leopold et al. (1971). They proposed a detailed expert-based impact matrix at the micro-level from which the macro aspect remains absent. It is concerned with only two scopes – economic and ecological – and assesses the possible impact of the former on the latter (second chapter). Recently Ekins and Medhurst (2003) proposed a method for IA in the complete multi-scope perspective. They developed a reduced version of Leopold matrix, but with the scope dimension expanded on the four scopes of sustainability (Leopold-Ekins-Medhurst impact matrix – LEM); they further allowed for aggregation of all impacts, represented in rows of their matrix on each particular scope – by column of the LEM. An analogous approach is conventionally applied in various standard IA procedures, such as the strategic IA (2001/42/EC), territorial IA (ESPON – 3.2, 2006), and ex-ante IA of the contribution of the EU structural funds to sustainability of regional development (GHK et al, 2002). Nevertheless, LEM’s summation approach is inappropriate, as the effects of individual policy measures on each appraised scope are not homogenous (Rotmans, 2006), and therefore not commensurable. So impacts could be aggregated by column in LEM only by source and area of impact, i.e. partially. This reorganises LEM into square matrix of Leontief (1951) with equal number of rows and columns (third chapter). We learn that social complexity does not prohibit aggregation in IA when one takes apart weak from strong incommensurability. This permits two step summation procedure in IA - the second step is synthesis with the correlation of intersections between different scopes that are obtained from the Leontief matrix. Both summative steps are illustrated with a practical example. The regional development programme for the Slovenian region of Pomurje for 2007-2013 (RDPP) is first assessed ex-ante with LEM and then with the MIA. Their summary results are found diverging. The second part of the paper studies reasons for this incompatibility. To accomplish this task a complete multiple-scale perspective is introduced into the IA (fourth chapter). When scale and scope aspects of IA are combined it becomes obvious that inconsistency in standard IA arises from inappropriate interpretation of assessment results across scope and scale. MIA offers new understanding on how to integrate policy concerns without imposing uniform norms. The implications for the governance of the social close up the paper. Work continues.

2. Summation in standard approaches

The first impact matrix generation methodologies for the ex-ante assessment of large scale policy interventions such as the one elaborated by geologist Luna Leopold et al. (1971; for review see Munn, 1979) explicitly reject the summation of multifarious policy impact into aggregate indicator. They were concerned with the impact of economic policy measures on the components of the environment. Their impact matrix lists the 100 most important economic policy actions horizontally and 88 environmental fields of impact vertically. The intersection between these two scopes creates a detailed matrix with 8800 fields – each further divided into four subfields that characterise each impact by its size (strong, medium, small), direction (positive, negative), probability and risk factor (critical or not). In this way, all possible impacts are presented in sufficient detail to enable informed decision. In the finest analytical manner, Leopold aims to provide the large picture of complex policy issue with a detailed description of all its elementary parts. The first generation methodologies claim that impacts should be presented disaggregated, leaving policy-makers with full responsibility for the synthesis conclusions in the light of a policy decision. There must be a clear demarcation in IA between the evaluator and policy-makers to ensure that value judgement rests with policy-makers and is not inappropriately shifted towards the evaluator. Refusal of aggregation is then important because it protects the evaluator from the political interference (Kunseler, 2007) that accompanies decision-making.

The rejection of summation in evaluation of large-scale policy interventions is problematic. Let us recall the impossibility theorem, which implies that the social optimum can not emerge from a reduction procedure. It is precisely the inability of policy-makers as social aggregators that calls for a policy evaluation in the first place. If evaluators fully accept their role in the policy cycle, they should not refuse summation of evaluation results! Another difficulty is that Leopold and even many contemporary authors observe incommensurable values as antagonistic, so they can not explain how opposing views work together, without which it is impossible to say anything about integrative capacity of a policy proposal. Failing to summarize impacts in the assessments is “letting the client down at exactly the moment they need you most” (Scriven, 1994). Without aggregation, an appraisal will usually not generate clear-cut conclusions but ‘information overload’ fostering a piecemeal rationality. Results that are too fragmentary and unrelated offer little value, fail to satisfy information needs at the strategic level and cannot provide an informed basis for a decision. All too often disaggregated results produce banal answers to complex and multidimensional societal problems (Virtanen, Uusikylä, 2004). Evaluation methodology hewing simply to the production of non-overlapping information tends to underplay the need for learning and substantiation of findings, legitimating the disregard of stakeholder issues (Stake, 2001), which alone is sufficient to leave an evaluation utterly exposed to a political interference.

To synthesize or not to synthesize – this is how the question is again posed for Scriven (1994). In somewhat less tragic storyline he explains that we need to distinguish between cases in which it is improper to push for a synthesis and cases in which it is improper not to. This fuzzy instruction may not be exciting for writers of tragedy, but it is very helpful for an evaluator. To see how this advice helps us, we first need to broaden the conceptual framework of social incommensurability to identify the conditions under which summation is feasible even though commensurability of welfare indicators is abandoned. Incommensurability is of two kinds: weak for individual considerations and strong for collective ones. Two arguments aim to justify the distinction below: uniqueness of individual elements of the system must to be delimited from incommensurability of social issues; under certain conditions even different aspects of social incommensurability can be traded between each other.

At the elementary or micro level the social system does not entail incommensurability but uniqueness (Li Xiaorong, 2007): for example, even though individuals are unique, they can be compared because their unique formation of characteristics is obtained from the shared base of human characteristics. The result is that individuals, their statements and acts are in some characteristics similar and comparable to some other individuals, but not to all others. So similar sorts of individual uniqueness are partly commensurable and can be locally averaged.

The aggregation of diverse impacts takes not only that the positive (or negative) effects of one particular policy measures can be summed up. More crucially it also assumes that a positive effect of one policy intervention outweighs its own or another policy’s negative effect. Is it, for example, sustainable to trade tons of greenhouse emissions for euros in EU’s scheme of tradable pollution permits market, when we know that greenhouse emissions can cause irreversible changes in the atmospheric conditions? Such a trade-off is not adequate as a general or macro principle, because economic and environmental aspects of welfare are equally important. However, one needs to reason in a multi-scale perspective, where micro is different from macro. Trade-offs between economy and environment are not accompanied by incommensurability in every single case, or at least people and their communities are not willing to treat them as such.

To incorporate this peculiarity in policy evaluation, system thresholds of sustainability – such as ecological, human and social ones – have emerged as a new assessment determinant (for a survey of literature see Muradian, 2001). System thresholds exist that should not be crossed for this would endanger the basic integrity of the system. Policy interventions that could jeopardize the thresholds should be avoided in order to circumvent (even more) antagonistic confrontation between, say, climate conditions and economic growth. So the concept of thresholds is closely linked to the concept of incommensurability. As Wiggins (1997) explains, two values are incommensurable if “there is no general way in which A and B trade-off in the whole range of situations of choice and comparison in which they figure”. Measurable social phenomena are incommensurable in IA ‘only’ beyond (or sometimes below) their threshold values. However, within safety limits, an agent either does not sense the difference between two conditions, or refuses to declare a preference for one or the other (Luce, 1956 in Munda, 2006) such as in the cases of minor damages that stay within ‘safe’ ecological standards. The practical consequence for policy evaluation is that positive and negative impacts can be aggregated (averaged, regressed, correlated, benchmarked etc.) only if they occur in the safe interval of system normality. Reliance on thresholds also simplifies evaluation because risk factors would be already incorporated in the definition of thresholds.

Where the appropriate threshold level is in every particular case, that is certainly a hotly debated issue, but this does not obviate the need for their imposition in policy evaluation. They simply account for the fact that there are discontinuities in the measurement of a value-based phenomenon and value addition (Mason, 2006) that comply with discontinuity in individual and social values. Thresholds set absolute limits to the possibility of trade-offs between different aspects of incommensurability (sustainability) on the micro level. This further necessitates a distinction between weak and strong incommensurability of the appraised impacts. Griffin (1986 in Morgan, 2007) argues against strong incommensurability because it repeatedly fails at every level of discourse except for debates about the nature of reality and truth (Morgan, 2007). Martinez-Alier et al. (1998, in Stagl, 2004) pointed out that in complex social systems, when there is an irreducible value conflict, we can only search for weak comparability in marginal, non-core issues as a facilitator of collective discourse. A consequence of this distinction for IA is that the summation algorithm should follow separate procedures for the aggregation of weakly incommensurable micro elements of the assessment and for the macro synthesis of conclusions that are based on components that are strongly incommensurable. The first step is addressed in the remainder of this chapter, and the second one in the forthcoming chapter.

Acknowledgment of weak incommensurability on the micro level enabled a methodological progress in the IA aggregation procedures. Among the first proposals was the methodology of strategic environmental assessment (SEA Directive, 2001/42/EC), that broadened the assessment focus to strategic issues that go beyond particular programmes and policy jurisdiction. Further progress came from Ekins and Medhurst (2003) in their proposal for the IA of the contribution of the EU structural funds to regional sustainable development in the light of their four capitals – economic, social, environmental and human. This four-valued logic of sustainability can be traced back to Brundtland report (WCED, 1987) and others (Munashinghe, 1992; Ekins, 1992). Proposal for the territorial IA in EU (ESPON Project 3.2, 2006; Camagni, 2007a, b) goes even further, proposing a composite indicator of territorial impact which is assuming commensurability of assessed impacts, and is therefore altogether contrary to Leopold.

For the reasons given earlier (refusal of composite as well as disaggregated measures), the most important recent proposal in the summative methodology may be attributed to Ekins and Medhurst. They acknowledged the existence of social incommensurability (their four capital model) and also properly allowed for a partial aggregation of diverse policy induced impacts of a similar sort. They have proposed highly compacted form of the Leopold matrix. The columns are reduced from 88 fields of possible environmental impacts to four scopes of regional sustainability. The first step in their procedure is standard for an assessment: composition of LEM starts when stakeholders of the appraisal select a smaller number of representative evaluation criteria for the assessment of policy impacts on each of the four scopes. In the second step participating experts judge how the implementation of each of the proposed policy instruments impacts each evaluation criterion. Impacts are expressed as relative differences on a scale between neutral impact (0; either no impact or positive and negative impacts mutually neutralised), weakly, medium and strongly positive (+, ++, +++) or negative (-, --, ---) impacts. In the last step, all appraised impacts are aggregated within each scope for all criteria (horizontally) and than these aggregates are further aggregated (vertically) for each of the four scopes (vertical). The result is not one composite indicator but four composite measures of impact, in four column sums.

Table 1 presents aggregate results of LEM, from practical example that will be also used in the next chapter to elaborate summation problems in IA. Regional development programme for Pomurje region (2007-2013; RDPP) consists of 47 policy measures. Their impacts were assessed against eight evaluation criteria (two for each scope: economic – E, human – H, social – S, and nature – N). All appraised impacts (47*8 fields; see Radej, 2007) were aggregated in two respects: the first impacts for both indicators of each scope (obtaining four columns in Table 1) and than also impacts for all 47 measures into six main policies (six rows in Table 1). Finally, the impacts of the RDPP are aggregated by columns to obtain four aggregate impact indicators.

Table 1: The LEM matrix of the RDPP’s impact on regional sustainability

|                             Incommensurable scope|E |H |S |N |

|                                          of | | | | |

|regional SD* | | | | |

|  Sectoral policy | | | | |

|Value added growth |+++ |+ |- |+ |

|Tourism |+ |0 |0 |0 |

|Health |0 |+ |+ |0 |

|Rural development |+++ |+ |+ |++ |

|Infrastructure |+++ |+ |++ |++ |

|Environment |+ |+ |+ |+ |

|                     Summary impact of RDPP    |++ |+ |+ |+ |

Source: Radej, 2007. Note: * SD - sustainable development

Table 1 suggests the following overall conclusions. Impacts of RDPP on the four scopes of sustainability as identified in the summary row do not appear unacceptable, they are all positive, and the differences are relatively small. The most problematic thing is that growths of income will negatively impact social development. RDPP will most improve economic situation (++), while it will have only a weak positive impact on the improvement of social, human and nature sustainability. Individual policies support each others’ goals too weakly; nevertheless, for evaluator RDPP is satisfactory proposal because it promises a rather balanced and therefore sustainable improvement of all scopes of regional sustainability.

By deriving these conclusions, the broad purpose of the LEM is fulfilled. But assessment results can be analysed further if properly rearranged. Ekins and Medhurst incorporated social incommensurability into the evaluation, but only for policy goals and not for policy interventions. This is important because sectoral policies do not produce homogenous impacts; they are not neutral in scope, so they should not be aggregated by columns.

3. Partial aggregation

If detailed expert assessments of policy intervention’s impacts are not summarized, as is the case in Leopold, the assessment produces findings that are too fragmented. In contrast, full aggregation, such as in a LEM, causes that findings are too much amassed. Both approaches waste information on the complex constitution of the appraised phenomenon. In consequence both wade in inconsistent interpretation of the conclusions. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download