Reviewing the Ambiguous : Examining the Typologies of Public Participation Towards Its Evaluation

Arguments surrounding the epistemology, as well as practical manifestation of various typologies of public participation are enormous. This highlights not only the fluid and complex nature of the concept but also how strongly tied it is to time and place. While the typology of public participation is often viewed from the perspective of engagement levels, this paper uses a dual perspective approach to discuss the various forms of participation from the perspective of both engagement levels and motives, to that of specific contextual applications. The paper also draws from the practical experiences of planners in Malaysia and Nigeria to examine the relationship between evaluation approaches for public participation and the successes of participatory processes in planning projects. The perceived contribution of participatory mechanisms to a project’s success is found to be inadequate in explaining the technique’s contribution to the overall success of planning projects. Also, the motivation of (ex-ante) evaluation is more a determinant of the project success than the focus of evaluation. There is therefore a need for coherent frameworks to integrate previous evaluation experiences in to subsequent policy guides to improve further evaluation efforts as well as planning projects.


Introduction
Public participation, as it is commonly known, is a famous concept.However, scholarly disagreement still exists on the two key words contained in the term.Neither definition of 'the public' nor that of 'participation' is fortunate enough to be unanimously agreed upon by scholars.The different expressions used to denote public participation in different local contexts point to the fluid nature of the concept.While some advocates of public participation (for example Rowe & Frewer, 2000;Maidin, 2011) have criticized the use of the term consultation as an approach to participation of the public, in some countries such as South Africa and Australia, common terminologies used to denote public participation are consultation and public consultation respectively.What commonly comes to mind however, when public participation is mentioned, is the process of making collective decisions between initiators of programmes and projects and those that may impact or be impacted by the action.Mouratiadou and Moran (2007: 67) define public participation as a process whereby people are allowed to influence the outcomes of plans and working processes.Laurian and Shaw (2008: 294), on the other hand, have defined public participation as a "mode of relationship between the state and civil society that involves the public in decision making" or "mechanisms intentionally instituted by government to involve the lay public, or their representatives, in administrative decision making".Historically, the process is a transition from initiating and executing programmes/projects through executive orders and purely expert judgements, to a situation where the "public" are expected to make inputs in the design and implementation of programmes that will likely affect them or be affected by them.The ubiquitous nature of public participation therefore explains its relevance in many fields of literature.
political affiliation which ties it to decision making.According to Brodie et al. (2009), citizens who volunteer to participate in community issues (regardless of their motivation) must be prepared to face some sort of collective decision making challenges.Participation involves deciding on alternatives and making choices between them.Whether or not the process reflects conventional democratic principles, the decision that is finally taken by a focus group, citizen jury, or a planning cell is supposedly in the interest of the wider community.The aim in any case is to implement the best (agreed upon) alternatives and provide a balance between individual and public interests.In a nut shell, the implication of participation motives is generally the same across the different fields in which it is applied.
Although the confusion which previous scholars experienced when characterizing public participation is still evident in contemporary debates (such as Arnstein, 1969;Healy, 2009), and that sceptics have also cautioned the unrealistic rosy promises of participation (for example, in Brody, 2003) and the consequences of over-participation (Parkins & Mitchell, 2005;Caves, 2012), some agreement does exist among scholars concerning the potential positive outcomes of meaningful engagement.Smith (2003) summarized the need for public participation to include (but not be limited to) the following:

*Establish legitimacy in democracy
In the same vein, Abiona and Bello (2013) highlight the existence of a positive relationship between grassroots participation in development programmes' decision-making processes, and the sustainability of physical development programmes in Osun and Kwara states of Nigeria.That is to say, commitment from citizens towards participation at the grassroots levels leads to more sustainable physical development.The usefulness of community participation has also been recorded in a number of water resource management projects globally (Kujinga, 2004;Mouratiadou & Moran, 2007;Boakye & Akpor, 2012).The work of Christian, Fraser, Gyawali, and Scott (2013) highlights the contextual utility of some participation approaches, by indicating a successful navigation through obstacles of participation in Alabama's cost-share programmes by undeserved landowners in the region, through such strategies as landowners' workshops.
Studies on the evaluation of public participation including specific professional practices (such as those among urban planners) are quite relevant in that, they provide an insight on whether experts or professionals do actually care to look back at previous efforts being made to engage the public.The answer to this will certainly be beneficial to the design of subsequent policies and programmes.Also, with the fact that more public input is expected in subsequent policies, the input which should be expected from planners can be reflected through their perception in say a 'comfort zone' for a more realistic level of engagement.An explanation for why public input in Malaysia's planning has previously been after the preparation of draft plan for example, could be as a result of it being the most practically (in that physical context) realistic option.

Method
At first, a dual perspective approach was taken to review the different typologies of public participation, beginning with categorization such as indicating levels of influence/public impact and information flows, to the specific levels of particular programmes in which such participation can be achieved.Secondly, from the results of a pilot study conducted among 21 planners from the Federal Department of Town and Country Planning (FDTCP) peninsular Malaysia, as well as 76 members of the Nigerian Institute of Town Planners (NITP), planners' evaluation experiences are examined from the perspective of the utilization of engagement approaches, planners' views on the role of citizens, and the contribution of engagement approaches to the success of participatory processes.The two surveys were conducted in July 2012 (in the case of Malaysia) and November 2012 (at the Annual General Meeting of NITP in Abuja, Nigeria).The survey and follow up interviews adopted In the International Association of Public Participation's (IAP2) (2004) public participation spectrum, Arnstein's concept is represented as: inform, consult, involve, collaborate, and empower.In a nut shell, the entire spectrum or continuum (in a planning sense) can be seen to reflect the transition from conventional or traditional forms of planning, to interactive planning.

Forms of Participation: Specific Contextual Applications
The second perspective through which the typology of public participation can be discussed is in the forms which the participation may take in specific cases.Scholars' varying approaches to understanding and researching participation are reflected in the terms they use.The term participation, as earlier noted, is contested and used in different ways by different authors within the body of literature.An array of prefixes (such as civic, civil, public, community, and citizen), often qualify the term participation.The motive however generally denotes a collective decision making process between initiators of programmes or projects and the 'public'.Bowman and Roberts' (2001) used the contextual application approach in a public participation study in the area of environmental decision making.Using instances from environmental decision making, Bowman and Roberts described the following participation types: The extension of the above classification in planning decision making can be seen in the different levels of plan formulation and in the evaluation of alternatives.Citizens can participate in the initial design stage, more detailed design stage, preparation of draft reports, the review stage, and the decision making stage.However, one may ask the question whether the citizens in question possess the capacity to make a meaningful input at the different stages or even whether the possibility exists for their inputs to be utilised.These questions can be linked to the form of participation that characterizes the planning system in question.In places where participation is still characterized by a dominant one-way communication, the desire for actual public input is initially not intended, but as Arnstein suggests, the aim is to educate and cure the citizens.

Evaluation of Public Participation
We have seen from the previous sections that the complex issues involved in public participation mean that every experience in a particular participatory process can serve as a ruler for subsequent initiatives.Several authors have noted this fact including the context-specific nature of the concept (Hillier 2000;Healey, 2003;Laurian & Shaw, 2008).The evaluation of deliberate participatory processes, in order to improve subsequent programmes and projects, is therefore not a new thing (see Rowe & Frewer 2000, 2004, 2005;Laurian & Shaw, 2008;Brown, 2012).Among Malaysian planners involved in public participation for example, criteria for judging the success of a participatory process include the number of participants (for instance at the focus group or publicity (seranta)) and the volume of objections from the public.In each case, the higher the number, the more successful the process is seen to be.Objections however may not always be an indicator of success, except if the intent is only to carry large numbers along.Its persistence may also point to dissatisfaction.
Still with regards to the public participation evaluation debate, Abelson et al. (2003) asserts that increasingly complex decision making processes require a more informed citizenry that has weighed the evidence on the issue, discussed and debated potential decision and options, and arrived at a mutually agreed upon decision or at least one by which all parties can abide.Their study explores the fascination with deliberative methods for public involvement by examining their origins within democratic theory, and then by focussing on the experiences with deliberative methods.By so doing, the authors have highlighted the need for evaluation criteria (not only the intended action but by itself) to be agreed upon.However, the application of Beierle's components for evaluating a deliberative process by the authors was carried out in the health sector and the participants were "informed" citizens who could weigh evidences and had the capacity to debate and discuss potential decision options.Rowe and Frewer (2000) assert that, a general lack of empirical consideration on the quality of existing public participation methods arises from confusion regarding the benchmarks that are appropriate for evaluation.Their study suggested the need to consider which aspects of the process are desirable and then to measure the presence or quality of these aspects of the process.By so doing, the authors specified a number of theoretical evaluation criteria, comprising of acceptance and process.Combining acceptance (representing the outcome of the exercise) and process may lead to a fair evaluation being it two-fold.However, the science and technology policy (which Rowe and Frewer studied) can in itself be context specific, particularly if differences are considered in terms of citizens' capacity.Additionally, the 'early involvement' component of the acceptance criteria may only be possible where the participants have the required capacity to be involved early.In another instance, Rowe and Frewer (2004) state that in theory, evaluation of public participation exercises appear to be largely concerned with the outcome rather than the process.This, according to Rowe and Frewer, is a result of the "uncertainty in the research community as to how to conduct evaluations".The agenda presented by their study might lead to a better evaluation of public participation, through identifying the credence of defining effectiveness and operationalizing one's definition.However, although the study includes an analysis of a number of evaluations in the past, the promise to establish which mechanism works best in which situation still remains open ended.This is because although the cases of evaluation reviewed are numerous, certain important urban development projects/programmes (for example those considered largely profit-oriented and initiated monarchically) were not captured.Charnley and Engelbert (2005) state that written mail surveys were an effective and economical tool for obtaining feedback on the environmental protection community involvement programme.Their study disaggregated the citizen's understanding and satisfaction into four components: understanding of environmental and human health risks (associated with the studied site), satisfaction with information provided, opportunities for community input, and the agency's response to community input.This can be helpful in evaluating an agency's public participation programme.However, Charnley and Engelbert's evaluation appeared to be outcome based, since the focus was on citizen understanding and satisfaction alone.Gelders, Marleen, Jeroen, and Nathalie (2010), on the other hand, state that participation and collaboration, resources, policy involvement, communication, context, method, and continuity are relevant for evaluating citizens' engagement and provide important guidelines for government communication policies.Their study provides a pretext for a more versatile evaluation, in that, resources were considered alongside context, method, and continuity.Therefore, it may not be difficult to identify the contribution of the various factors towards the success or otherwise of a participatory exercise.However, the authors' focus on individual citizens, as well as citizens in associations (for instance, community organizations), has meant that a different combination of indices may be required should the two components be considered in isolation.
To sum up, Laurian and Shaw (2008) summarized Charnley and Engelbert's approach to public participation evaluation in the form of: a. Focus: This entails what the evaluators are actually targeting.This could be process, outcome or participants' satisfaction.
b. Motivation: This reflects the source of the initial urge to evaluate, either as a result of it being a requirement by the government or the donor agency, reactions from residents or a failure of previous projects.It could also be as a result of the evaluators' personal initiative.
c. Criteria for judging success: This reflects the criteria which allow us to judge the success of a participatory process.For example, in some cases (such as Malaysia) number of participants and volume of objections are often considered as indicators of success.In the following subsection, we shall see how some of the indices relate to the overall success of a participatory process in practice, using planners' evaluation experiences in Nigeria and Malaysia as examples.

Engagement Approaches and the Success of Participatory Processes: Planners' Experience From Nigeria and Malaysia
In general, the Malaysian case showed a 76:24% ratio of planners with public participation evaluation experience, and those without.This is in direct contrast with the Nigerian situation where the ratio is 29:71%.
Similarly, the on-going survey in Malaysia (which so far has recorded 126 responses across different categories of planners) showed a 44:56% ratio.The variation in the Malaysian pilot case can be explained by the fact that respondents were drawn only from the FDTCP project office at Kuala Lumpur, and largely from the monitoring and evaluation unit.A more realistic picture is therefore that fewer planners are experienced in public participation evaluation.Non evaluation was explained largely by the fact that either the respondents had never had the opportunity or perhaps the task of evaluation is a specific person's responsibility in the organization.In all cases however, the respondents indicated their intent to evaluate in the future, should opportunities exist.
To examine the role of the public in a planning process, from the perspective of planners, in line with the different theoretical levels of engagement outlined by Arnstein, we found that planners are sceptical about the highest rungs of full project control by the public, or even to have a majority say (see Figure 3).
Figure 3. Role of public in the planning process Negotiating alternatives, engaging in trade-offs, or for the public to be consulted with so that they can outline concerns about the project, are actually preferred.This could be a reflection of the existing citizens' capacity as well as social organization.What happens, for instance, if the elected officials (who have the decision making

Planners' level of agreement with the roles
Roles of the public mandate from citizens) have a majority say or full control of the projects in question?Will this not be synonymous with the citizen control?Again, the issue of adequate capacity will still prompt the need for a balanced consultation between experts and the lay public or their representatives (as identified by Healy, 2009) as a more realistic position.
The following is the pattern of relationship between focus and motivation of evaluation, and the overall success of projects/programmes with participatory processes.At the time the Malaysian pilot study was conducted (in July 2012), the overall project success was not included among the indices examined.Therefore, the explanations were based on the contribution of various techniques for participation to the success of participatory processes, and the relationship of those techniques with the focus and motivation of evaluation.
The experiences of Nigerian planners in public participation evaluation have shown a moderately negative relationship between participants' satisfaction as a focus of (an ex-ante) evaluation and the overall project success (see the correlation coefficients in Table 2).This implies that participatory processes for which the focus is participants' satisfaction seldom yield successful projects.For process and outcome, there is no significant relationship with project success.That is to say, when designated as the focus of an evaluation (which is designed prior to project implementation) process or outcome does not affect the project's overall success.

Overall success 1
In Malaysia, a question was asked to participants, to indicate the contribution of different techniques to the success of participatory processes.The responses show that focus group discussions appeared to be the dominant engagement technique contributing to the success of participatory processes, with meetings potentially having the least impact (see Figure 2).Even though generally lower than focus groups, public hearings and workshops are also significantly relevant.In the Malaysian context for example, the success of a participatory process is determined by its volume of attendance at the focus group discussion (in the initial stage of the process) as well as the number of objections received after the draft plan is prepared and open for publicity.This perhaps indicates citizens' commitment, which is generally lower in rural communities (regarding planning decision making as reported by FDTCP planners) than in urban or sub-urban communities, and as confirmed by studies elsewhere (Taylor, 2003;Harrison & Singer, 2007;Brodie et al., 2009) and also reflects differences in social characterization.Similarly, participation, being an incremental activity, often requires an experimental learning process (Brodie et al., 2009) to provide the affected citizens with the required capacity to make meaningful contributions to the process.A relationship was then examined between the focus of evaluation and focus group discussion (FGD) as a contributor to the project success.The result showed that all the indices, with the exception of 'process' (with low negative relationship), have no significant relationship with focus groups as a major contributor to the project success.This means that planners' perception of a particular participatory technique, as having a greater contribution to the success of participatory may not necessarily be an indication that overall, projects in which the participatory process utilizes focus group discussions are often successful.This further indicates, in relation to the contribution of participatory techniques to a project's success, that the relationship between a technique which explains the success of participatory process more and other variables only indicates higher instances of using the said technique under an evaluation which has outcome or process as its focus.It is not a necessary indicator of the same relationship with overall project success.Outcome 1 In the Nigerian situation, meetings displayed the highest contribution to the success of participatory processes and have a moderately positive relationship with outcome and participants' satisfaction.However, when correlated with overall project success, participants' satisfaction showed a negative relationship while outcome indicates no significant relationship.Meetings, which happen to be the major contributor to the success of participatory process, (as a technique) showed almost no relationship with overall project success.This further confirms the earlier notion that contribution to the success of participatory processes, as observed by planners, is not synonymous with contributing to the overall success of the project.
With regard to the motivation for evaluation, the Nigerian case indicates that projects with externally motivated evaluations, particularly as a requirement by (external) donor agencies, possess greater potential for success.It is not strange however, that externally motivated evaluations which are conducted as a result of either failure of previous projects or reaction from residents showed a negative relationship with overall project success.

Conclusion
Contemporary debates surrounding the epistemology of public participation reflect the complex nature of engaging citizens in 'public good' programmes.While the idea of public participation can be disaggregated in the form of engagement levels, which is often linked with levels of influence or public impact and information flows, specific contexts in which public inputs are needed also contain diverse forms which the said participation can actually take.The diverse forms are not necessarily tailored along the line of the engagement levels (for example as identified by Arnstein (1969)) but are often related to citizens' capacity.We have seen from the perspective of professional practice that, planners do not perceive the role of the public in the planning process to be mere information exchange, nor are they comfortable with the highest rungs of full project control.Also, the relationship, between a technique which appeared to contribute more to the success of participatory processes with focus and the motivation of evaluation, does not reflect the existence of such a relationship between focus or motivation with the overall project success as we had expected.This is because the Nigerian case showed almost no relationship (Pearson's r = 0.05) between meetings as a participatory technique and overall project success, even though meetings contributed more than any other technique to the success of participatory processes.The dynamic nature of stakeholders in participatory processes requires a continuous feedback process, in order for evaluation criteria to have temporal relevance.A flexible framework in which a bridge is provided, to link planners' evaluation experiences with subsequent operational policies for organizing participatory processes, is therefore required to guide the circular relationship between theory and practice in the evaluation of public participation.
responsibility: establish priorities and find partners *Enhance public knowledge, understanding, and awareness *Meet legal and policy requirements *International and local agreements: -Principle 10 of the Rio declaration on environment and development -The E.U's Aarhus convention of 1993 and the 6 th environment action programme of 2002 -Europe's Water Framework Directive (WFD) 2000, etc.

Figure 4 .
Figure 4. Contribution of citizen engagement techniques success of participatory processes

Table 1 .
Types of public participation in environmental decision making

Table 2 .
Correlation coefficients for focus of evaluation and project's success

Table 3 .
Correlation coefficients for focus of evaluation and Focus Group Discussion as a major contributor to success of participatory process

Table 4 .
Correlation coefficients for 'overall project success' and motivation