The Planning and Appraisal of Mega Transport Infrastructure Projects Delivered by Public–Private Partnerships: The Case for the Use of Policy-Led Multi-Criteria Analysis
Article Category: Research Article
Published Online: Jul 30, 2019
Page range: 1992 - 2008
Received: Sep 05, 2018
Accepted: Nov 05, 2018
DOI: https://doi.org/10.2478/otmcj-2019-0007
Keywords
© 2019 John Ward et al., published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Mega transport infrastructure investment in the form of large-scale bridges, tunnels, road and rail links or combinations of such projects has in recent decades rapidly grown in number, size, complexity and cost. These projects, typically associated with capital expenditures of well over US$1 billion, are frequently perceived as critical to the “success” of major urban, metropolitan, regional and/or national development because of their potential to affect significant socioeconomic and territorial changes (OMEGA Centre 2011, 2012). According to many infrastructure experts, these projects should not be considered merely as larger and more expensive versions of traditional transport infrastructure investments. Rather, they should be regarded as a totally different “breed” of project (Capka 2004) because of their diverse outcomes and impacts that frequently go well beyond the physical assets that are being delivered (OMEGA Centre 2011, 2012).
Notwithstanding that mega infrastructure investments, and mega transport projects (MTPs) in particular, currently overshadow much of the development agendas worldwide, those projects deemed “unsuccessful” because they have been unable to meet their original expectations have tended to dominate the international literature narrative about infrastructure development (Hall 1980; Morris and Hough 1987; Altshuler and Luberoff 2003; Flyvbjerg et al. 2003; Samset 2012). A major cause for such perceived underperformance has been attributed to the inadequacies (and inappropriate applications) of
Reinforcing this critique, there is also a growing body of literature expressing concern regarding the excessive importance given during project appraisal to overtly economic tools such as cost–benefit analysis (CBA) (Brown et al. 2001; Alexander 2006; Naess 2006; Metz 2008; Dimitriou et al. 2016) and the exclusion of many project stakeholders from the planning and appraisal process (Haezendonck 2007; Macharis et al. 2009; Colomb 2010; Macharis and Bernardini 2015; Ward et al. 2016). Many authors of this body of literature have emphasized the need to ensure more holistic and transparent assessment of project proposals by employing multi-criteria analysis (MCA) methodologies more extensively and increasing the stakeholder participatory character of the appraisal exercise (Dimitriou et al. 2016). This is seen as not only potentially enhancing the agent of change and sustainable development attributes in the evaluation of such mega projects but also better linking their policy and planning aims with their performance, when compared with measurement by more traditional metrics of project appraisal that focus on the so-called “iron triangle”, i.e. “on time, on budget and to specification” delivery concerns, frequently referred to as the project management metrics of the iron triangle (Weaver 2009).
The growing calls for broader and more transparent project appraisal frameworks coincide with a period when public private partnerships (PPPs) are growing in importance globally as the favoured procurement route for governments looking to undertake new mega transport infrastructure developments (see Section 3). Defining here, PPPs “as a long-term contractual relationship between a government or public agency and a private-sector entity, where the latter delivers and funds public services using a capital asset, sharing the associated risks with the public sector participant/client” (Dimitriou and Field 2017), some see the practicalities of PPPs as placing them at odds with aspirations for more inclusive and open project appraisal with adequate consideration of the public interest (Shaoul et al. 2006; Siemiatycki 2009). It is, nonetheless, the authors’ contention that if introduced with broader and more systematically presented sustainability concerns, PPPs can remain compatible with such ambitions. This is notwithstanding the fact that there remain a number of hurdles discussed in Section 3 to be overcome before PPPs can realise their full potential.
Towards this end, this paper presents the rudimentaries of a policy-led multi-criteria analysis (PLMCA) approach to project appraisal (outlined in Section 4), developed by the OMEGA Centre at University College London (UCL), as a means by which PLMCA can contribute to more holistic PPP procurement practice. PLMCA is defined here “as a form of Multi-Criteria-Analysis (MCA) to decision-making that is policy-led and that can be applied at any stage in the project lifecycle, providing a means for the (re)structuring of complex decisions by allowing for both quantitative (not necessarily monetized) and qualitative data, plus other information to be taken into account in appraisal (Dimitriou et al. 2013). It comprises both a framework and set of attendant processes that are informed by lessons of practice and research findings of the OMEGA Centre regarding the planning and appraisal of MTPs and seeks to explicitly and directly incorporate different (often conflicting) interests of key stakeholders side-by-side with multiple public policy objectives and related appraisal criteria with a view to identifying commonalities, differences and potential areas of future stake-holder collaboration and compromise (Dimitriou 2016).
The authors contend in the latter part of the paper that PLMCA addresses many of the limitations associated with the application of narrower decision-making and project appraisal approaches currently supporting PPPs and other more conventional procurement practices. A tentative outline of the advocated PLMCA approach for application to an MTP delivered by a PPP is offered in the penultimate section of the paper. It is referred to as tentative because it has yet to be applied/operationalized in a PPP context. Without feedback from project stakeholders engaged in a real-life PPP, the principles, steps and stages advocated must at this stage be deemed theoretical for such project procurement paradigms.
The development of PLMCA as presented here has its roots in the findings of several research projects undertaken by the OMEGA Centre at UCL, in particular research conducted for the UK Institution of Civil Engineers (ICE) and UK Actuary Profession (AP), known alternatively as the OMEGA 3 Project (OMEGA Centre 2010), where the Centre concluded that the employment of a policy-led MCA approach was better able to incorporate social, environmental and institutional dimensions of sustainable development in the appraisal of major infrastructure projects than traditional appraisal methods. The original MCA framework proposed as the cornerstone for this study (which incidentally subsequently contributed to supplementary guidance to the UK Government’s
This research included a critical review of decision-making in the fields of military, earthquake engineering, banking and finance, insurance, agriculture and contagious diseases, public health and medicine and the corporate worlds of business management and innovation, as well as in the fields of project management for construction, planning and appraisal in territorial strategic and regional planning, city and transport planning and real estate and property development. It highlighted the importance of the treatment of risk and uncertainty, the clarity of the boundaries of mega projects and the influence of contextual factors impacting project decisions to the success of mega projects.
Of the three sets of OMEGA Centre research cited earlier, PLMCA as applied to the planning and appraisal of MTPs has been most informed by the nine clusters of lessons arrived at from an extensive critical analysis of the 30 MTP case studies in the OMEGA 2 Project (Figure 1), the findings of which have been published in a variety of publications (OMEGA Centre 2012; Dimitriou et al. 2013; Dimitriou and Field 2017) and are summarized in Figure 1. The three sets of research together concluded that conventional wisdom mega project appraisal methods are not only too narrow in scope (to capture more holistic perspectives of project outputs/outcomes) but also inadequate in their treatment of risks, uncertainties and complexities outside limited boundaries to suitably inform a full spectrum of key project stakeholders as to what constitutes a successful project beyond the metrics offered by iron triangle project management concerns. These conclusions especially point to CBA and its derivatives being incapable of responding adequately and robustly to unexpected events (including those generated by policy changes) external to project management decision-making (Dimitriou et al. 2016).
Fig. 1
OMEGA 2 nine clusters of lessons of relevance to PLMCA application to PPPs. Source: Adapted from OMEGA Centre (2012)
As conveyed in Figure 1, the findings of the OMEGA 2 Project suggest that many MTPs conceived as a coordinated series of major infrastructure interventions have the potential to become strategic agents of change for the territories, economies and communities they serve. This is on account of the multiple spatial, economic, environmental, political, financial and social impacts that such projects often produce over time in the areas in which they are located. Their strategic influence, interdependency relationships with other infrastructure projects and associate developments, plus long-term and operational phases further reinforce the case for the need for their broader framing and appraisal beyond more immediate project delivery concerns, as important as the latter may be.
The two-way relationship between mega project decision-making and the external policy environment of this decision-making not only has these projects responding to economic and political changes over time and the policy changes they spawn but also has them frequently acting as catalysts for major transformational changes either by design or default. Such projects can be affected by (and contribute to) unexpected developments leading to unanticipated outcomes (both positive and negative), which reinforce the case for the need of more open systems’ planning treatment of decision-making than project management typically permits. The OMEGA 2 Project from which the abovementioned observations were derived (OMEGA Centre 2011) conceives this interaction (between project and context) as “exploratory” and almost “organic”, allowing for unexpected outcomes to become recognized and accepted as part of an “emergent order”, replacing in some instances more planned orders.
In these terms, the real challenge to MTP decision-making is not represented by how to overcome the irreducible complexity and uncertainty that characterize the decision-making for such large-scale infrastructures but rather how to shape them, framing and reframing the strategic mission of a given project in light of its changing contextual forces and trends. The same OMEGA Centre research observed from its case studies that “a period of reflection” is often beneficial during the preparation (even execution) of such projects, particularly during periods of prolonged enforced delays. This provides an opportunity to ensure both a re-examination of past decisions against contextual changes (context scanning) and the incorporation of newly “emerging” project objectives, thus reshaping the original project scope to the realities of the day in light of new developments and knowledge/information provided.
Identifying such robust measures has the advantage of building into decision-making more resilience. It should be appreciated that it also ensures an adequate redundancy of information is provided, which is inevitably necessary when involving multiple different stake-holder groups in dynamic complex decision-making. While the identification of the relevant actors and the analysis of their mutual relationships and agendas can be very challenging and lengthen the decision-making process, this advocated “opening up” to a plurality of voices and lines of argument makes for more effective consultation and is more likely to capture key issues that need to be addressed. The employment of PLMCA (discussed in Section 4) not only accommodates the abovementioned dynamics and complexities but also helps highlight differing values of multiple stakeholders that a given project should reflect. It can also inform the need for the adjustment of project objectives plus the different kinds of costs and benefits produced by more possible initiatives, and the avoidance of the premature discarding of options.
What immediately follows (in Section 3) is an introduction to the principles and modes of public–private partnerships (PPPs) as investment frameworks and vehicles to deliver mega infrastructure projects, MTPs in particular. This begins with an introduction to the principles and application of PPPs to mega infrastructure projects in the following section, with a view later in the paper (in Section 4) to theorizing how PLMCA might be applied to the appraisal of an MTP delivered by a PPP.
There is no single universally accepted definition of PPP either as a shared decision-making process or as a project delivery contract (OECD 2008; Marin 2009). For the purposes of this paper however, and as earlier already indicated, our working definition of a PPP alludes to “a long-term contractual relationship between a government or public agency and a private-sector entity, where the latter delivers and funds public services using a capital asset, sharing the associated risks with the public-sector participant/client” (Dimitriou and Field 2017). PPPs in this respect are seen to possess two important characteristics: they provide an emphasis on service provision by the public sector and, to this end, offer investment by the private sector. In so doing, they are, furthermore, claimed to significantly transfer risks from the public sector to the private sector in the delivery of the public infrastructure and services.
Although very basic PPP modalities can be traced back historically to earlier centuries (Aldcroft 1968), governments began experimenting seriously with PPPs in the UK in their more contemporary incarnation in the 1980s as an alternative to traditional public-sector construction procurement contracts. Driven by the prospect of transferring fund-raising responsibilities to the private sector, and achieving perceived efficiency gains during both project delivery and operation, several different PPP models have evolved in the UK and elsewhere (especially in Australia and the USA) of the kind generically illustrated in Figure 2. According to some sources (Marin 2009), PPPs can range from design and build schemes or design, build and maintain (DBM) contracts with no investment obligations to concessions/contracts with significant investment obligations in addition to operational and management responsibilities that include typically design, build, operate and maintain (DBOM) schemes or build, own operation/transfer (BOO/T) style contracts. DM and DBM contracts tend to be closest to traditional procurement structures, and by assigning primary responsibility to the public sector, they offer the least risk-transfer away from the public sector. In contrast, BOT and DBOM style contracts are less traditional procurement structures, seeking to transfer much of the control, but also risks, to the private sector.
Fig. 2
PPP models and the risk-sharing continuum. Source: adapted from Rodrigue (2017)
During the 1990s, a PPP contract type emerged out of the UK that sought to bundle design, construction, financing and operation into a single long-term concession (Siemiatycki 2009). Referred to as design, build, finance and maintain/operate (DBFO or DBFMO) schemes, this particular model has gained global attention for delivering major infrastructure projects, especially MTPS, claiming to both introduce greater competition into the development process and to achieve a better match between the aims, risks and rewards of the private and public entities by using long-term contractual agreements. For the purposes of this paper (and the discussion which follows), the authors see PLMCA as particularly suited for application to PPP models adopting the DBFO structure – these being seen “as a long-term contractual relationship between a government or public agency and a private-sector entity, where the latter delivers and funds public services using a capital asset, sharing the associated risks and responsibilities, resources and competences with the public sector participant/client” (EPEC 2017).
As earlier indicated, PPPs of this kind focus on the service provision by the public sector and some form of investment by the private sector in support of this service provision, with both parties looking to share risks thereby transferred from the public sector to the private sector and apportioned as appropriate. Since mega infrastructure investments procured via this style of PPP are typically very capital intensive, they commonly use a special purpose vehicle (SPV) for their delivery (Figure 3), which, depending on how the latter is structured, resourced and managed, has implications for the levels of risk ultimately taken on by the private sector and retained by the public sector.
Fig. 3
A typical PPP structure. Source: Dimitriou and Field (2017)
An SPV is a legal entity typically proposed by the consortium of infrastructure experts and associated parties that bid for a tender and which, if successful, undertakes the project. All contractual agreements among the various parties are negotiated between themselves and, once set up formally, the SPV. So, for example, in the case of an MTP procured via a DBFO, the SPV enters into a contract to design, build, finance and operate the facility for an agreed period (typically 25–30 years). The SPV will then raise the finance to pay for the provision of the facility without carrying the debt or other liabilities of the parent organization. The client (typically from the public sector) then pays the SPV for the availability of that facility and the services it provides (e.g. for maintenance, facilities management, etc.) through a periodic charge (often monthly) after construction is completed and occupancy commences. As far as the evaluation of outcomes from the SPV, performance measurement becomes key, leading to the use of highly tailored service-level agreements and key performance indicators. In the case of DBFOs, the definition of such performance criteria is often left to the private consortium itself (Siemiatycki 2009).
While PPPs are seen by many involved in the infrastructure investment and development worldwide as an increasingly important vehicle in project delivery, particularly for mega infrastructure projects (and especially MTPs), their introduction has been scrutinized from many perspectives; and not just in terms of the value for money (VLM) for public sector/users. Questions have, for example, been raised regarding:
the ability of PPPs to meaningfully reduce public-sector debt (Quiggin 2004); the extent to which the PPP model gives private financiers and investors undue influence over national and/or regional strategic development priorities (Siemiatycki 2006); whether PPPs afford the high level of transparency necessary to achieve meaningful public consultation and involvement (Daniels and Trebilcock 1996); whether in PPPs, risk transfer from the public sector to private sector actually occurs in all cases in line with the promises/claims made by PPP promoters (Siemiatycki 2009); whether the typically long-term contractual arrangements between PPP partners limit the flexibility of future decision makers to respond to changing conditions that have significant impact on the project and, in light of this, make plans that best meet evolving public interests (Siemiatycki 2009) and whether PPPs offer the required levels of open communication and trust as platforms for project stakeholders to make compromises and collaborate as a means of reducing potential conflicts between private and public parties that would otherwise threaten a project’s success (Jacobson and Choi 2008).
In addition to the abovementioned questions, recent research drawing on case studies and analysis undertaken by the European PPP Expertise Centre (EPEC) at the European Investment Bank (EIB) has identified the following seven key issues related to PPP implementation (after EPEC 2016):
the degree of political commitment needed to meaningfully engage in PPPs; the suitability of the legal, regulatory and institutional frameworks and policies in place to deliver PPPs; the adequacy/suitability of the capacity of the authority(ies) responsible for delivering the PPP; the adequacy of the capacity and buy-in of the private-sector party(ies); the inflexibility of prevailing public-sector procurement protocols; the funding constraints (of both public- and private-sector parties) as against financing problems and the statistical treatment and management of PPP fiscal risk.
Both the findings of EPEC and the earlier questions raised suggest that a focus on addressing these issues/questions is of
Unlike CBA, the authors contend that a PLMCA approach offers such a framework and facilitates stakeholder dialogue at different stages of the project lifecycle so that all/most involved parties can better understand project aims, risks, opportunities, outputs, outcomes and impacts from different stakeholder perspectives over time (and in different scenarios) and how these relate to their own agendas, interests and responsibilities, as well as, most importantly perhaps, those of other key stakeholders (Dimitriou et al. 2016). The following section seeks to elaborate on this potential application of PLMCA to the appraisal of MTPs delivered by a PPP contract drawing in part on some of the findings of the OMEGA 2 Project alluded to in Section 2 of the paper, plus other works undertaken by the OMEGA Centre.
Because PPP approaches are deemed to be optimized when the private sector is given specific output requirements (in terms of asset performance criteria for the user of the asset) when it can use innovation and past experience to better design and deliver “the optimum” asset and service integration against cost (Medda et al. 2013), the authors see a PLMCA appraisal approach potentially assisting project stakeholders arrive at a fuller range of required/expected “outputs, with their suitability and robustness assessed against pre-stated policies, aims and objectives” translated into project performance criteria. This contrasts with other project appraisal practices that typically only refer directly to the user experience of the performance of the built asset.
While it is of course recognized that this kind of “output based approach” to project development ultimately seeks to transfer the risk of project delivery of this output from the public sector to the private sector, the proposed widening of the scope of consideration of such outputs is likely only to prove to be VLM if the private sector feels it has the ability to manage this risk. If this is not the case, the availability of funding is likely to be adversely affected or requires too high an investment return given that more risk equity would be needed in the overall funding mix. PLMCA can prove helpful in this context by identifying which outcomes are best managed by the private sector and those which are best delivered (and funded) by the public sector, with a clear interdependency agreement between these two options. In so doing, PLMCA can provide support to PPP funding by helping to demonstrate to the private sector the level of wider stakeholder support and, therefore, the consequential reduction in political risk, which is often seen as a major deterrent to many infrastructure investors and contractors (World Economic Forum 2015).
PLMCA, as applied to project appraisal, can be defined as a framework plus attendant process that allows the inclusion in the complex decision-making process of multi-stakeholders from multi-sector perspectives directed by policy guidance with a view to facilitating the necessary trade-offs among major stakeholder objectives (OMEGA Centre 2013). A key element of the framework as applied to project infrastructure appraisal is the explicit link formed for decision-making between the numerous public policies framing and shaping such developments and the cost and performance metrics that make such projects viable and efficient.
As in the case of MCA, PLMCA is a decision-making process that can be applied at any stage in the project life-cycle. It provides a means for the (re)structuring of decisions by allowing for both quantitative (not necessarily monetized) and qualitative data/information to be taken into account in a consistent, systematic and transparent manner over time (including alternative scenarios, where needed). Applied to project appraisal, PLMCA provides an invaluable means for the holistic assessment of project performance in all of its dimensions (technical, economic, financial, environmental, etc.) when set against complex and often conflicting objectives. It also offers opportunities to examine multiple project impacts, especially distributional consequences, over time and space. It has, furthermore, the potential to incorporate findings of various other appraisal methods such as CBA, economic effective appraisal, strategic environmental impact assessment (SEIA) and community impact assessment (CIA), all of which can be embedded within a PLMCA framework if so desired to inform the overall appraisal analysis.
As earlier mentioned, the PLMCA approach presented here builds on the original MCA framework proposed by the OMEGA Centre as the cornerstone of its study for the ICE and AP completed in 2010. This framework was successively enhanced by subsequent works undertaken by the OMEGA Centre during a number of other research assignments (Rosenburg et al. 2013; OMEGA Centre 2014a, 2014b). The most notable of these were the application of PLMCA to the planned extension of the London Underground Northern Line to Battersea (Ward et al. 2016), and the application of PLMCA to the appraisal of major urban and regional development investment proposals considered by the EIB for investment support (EIB 2015). Subsequent applications have been made to the appraisal of urban regeneration and urban heritage assets for the Government of Malta. The development of PLMCA has thus assumed a somewhat organic process. It is still seen to be in a relatively early stage of maturity where new elements learnt during each study have been incorporated to enrich and improve the overall framework and where the value of bespoke applications has become more apparent.
In generic terms, a PLMCA approach to project appraisal is characterized by the following features:
A more holistic and transparent framing of the project appraised: While many MCA methodologies seem to deal with weighting and scoring procedures in a decision-making context void of any meaningful issue identification, PLMCA seeks to place the framing of the problems it is required to address at the heart of the appraisal exercise. In so doing, it aims at establishing the wider policy framework guiding the decision, identifying the key stakeholders who can influence or can be affected by the process and incorporating a variety of economic, political, social and cultural values into the analysis. A more inclusive appraisal process: To better identify, inform and expand on the insights of the problems under consideration to be addressed by the proposed project, the PLMCA approach to appraisal views meaningful and timely engagement of all key stakeholder groups as fundamental. Only effective consultation of this kind, it is contended, allows a comprehensive identification of the array of policies and objectives that a project should reflect and the issues that need to be addressed. It should be noted, however, where a powerful/influential stakeholder refuses to participate, the value of the PLMCA exercised is inevitably significantly devalued, except perhaps in mapping out the different policies against different stakeholder positions, as a basis for informing all stakeholders (including the non-participating party) of the negotiation challenges ahead. It can even, perhaps, provide a basis for the non-participating party to belatedly see the value/merit of joining the dialogue (Dean 2017). A more policy-led process: Notwithstanding the stakeholder participatory process advocated earlier, it is very evident from the realities of actual decision-making that there are limits to negotiation and mediation exercises among stakeholders. This is no less true during a project appraisal process. Setting and managing these limits can be greatly assisted by the public policy and government plans/guidelines, which project appraisal decision-makers need to consider, alongside stakeholder agendas and priorities, in choosing a specific course of action to follow. In this sense, the entire PLMCA appraisal process is policy led involving a deep scan of key public policies, plans and guidance that have an important bearing on the appraisal and its related decision-making context. This facilitates the identification of, on the one hand, “non-negotiable” areas that different stakeholder groups have no choice but to comply with and, on the other hand, areas open for negotiation/compromise among stakeholders. Apart from a non-compliance by key stakeholders to participate in the PLMCA process, further limitations to project appraisal employing PLMCA include data availability, budgetary limits and time and contextual constraints, which, together with policy guidelines, in effect define the boundaries of the appraisal decision-making space within which stakeholder trade-offs in appraisal decisions can ultimately take place (Figure 4). A continuous balance between “opening up” and “closing down” the appraisal: A significant finding of the OMEGA 2 Project (Figure 1) is that the best way to deal with emergent threats (and opportunities) to the success of an MTP during its planning and appraisal is to adopt an “open systems” approach that focuses not only on the issues, problems and influences that occur within the project but also those external to them. In addition, given that the same research concluded it is prudent to advocate a commensurate opening up of the decision-making process to a plurality of stakeholder agendas in these same phases, it is contended here that the broader swath of potential challenges and issues that can be identified (and ultimately addressed) by PLMCA has a consequence in allowing underlying potential common and conflicting values to be better understood and addressed. This in turn facilitates the consideration of different types of costs and benefits produced by project alternatives and also helps avoid the premature discarding of options (Allport 2011; Bickerstaff et al. 2002; Miller and Lessard 2000; Stough and Rietveld 1997). As earlier argued, by opening up the analysis, this facilitates strategic thinking and sense-making of the context of the appraisal exercise. This in turn helps the multiple project investors (and planners) become more aware of the various levels of risk, uncertainty and complexity presented in the decision-making space they have been left, allowing them then to select and deploy the tools and techniques appropriate for the circumstances. This process of opening up and closing down forms the backbone of the PLMCA framework. It is supported by an iterative and dynamic set of attendant procedures that have various grades of “openness” (reflected in the different types and numbers of participants, different types of procedures to elicit scoring and weightings and different treatments of uncertainty). These are provided according to the needs of each stage of the project lifecycle and level of importance of the issue/challenge under consideration. It should be stressed here that the proposed framework does not attempt to draw a rigid/fixed line of demarcation between the “open” and the “closed” phases of decision-making. Conversely, the framework accommodates the opportunity for re-opening the analysis as a monitoring and/or evaluation tool (after project implementation) in response to new contextual changes. An integrated appraisal process: As an appraisal exercise, PLMCA in effect looks to provide a form of “knowledge platform” where information/perceptions/interests regarding a proposed project from various scientific disciplines and different stakeholders are provided together and made available to all parties deciding among options for action and choice. PLMCA is not intended to be a substitute for CBA or any other more focussed project appraisal methodology but instead represents a broader decision-making framework that can be populated by the findings of other appraisal techniques such as environmental impact analyses, SEIAs and CIAs provided to inform stakeholder negotiations. An alternative to having such specialist appraisal exercises inform a PLMCA appraisal exercise is to employ PLMCA as a form of resilience test, offering a risk and opportunity register against which specialist appraisal exercises are scrutinized with a view to examining/identifying issues drawn from a broader perspective that have not been registered on the radar screen of the specialist appraisals. A focus on learning aspects of the appraisal process rather than complicated algorithms: PLMCA as applied to appraisal, indeed to any stage of the project lifecycle, should not be seen as an approach that aims to “solve” a problem by means of complicated algorithms. Rather, it affords project stakeholders the opportunity of becoming more aware of other stakeholders’ perspectives of the challenges the project being appraised is expected to address. It looks to encourage/facilitate stakeholders’ re-thinking about issues and reframing of problems as a basis of possibly reconsidering their objectives and interests and how the project under appraisal can best address the challenges confronted. The fact that this shared journey of lesson learning and sharing has been found from the research of OMEGA Centre (Ward et al. 2016) to facilitate the creation of stakeholder alliances and consensus is an important attribute of PLMCA that adds significant value in the appraisal process. A context-sensitive appraisal method: The result of any single PLMCA exercise provides merely a “snapshot” of the outcomes of a particular phase/step of the appraisal decision-making process. However, after a decision on a given issue has been made, the proposed framework provides an opportunity for potentially re-opening the analysis in response to new contextual change if needed. The acknowledgement, as part of the OMEA 2 Project, of MTPs representing organic phenomena with unexpected outcomes as part of an “emergent order” in line with complexity theory implies that the continuous monitoring of the context within which the appraised project is reviewed to be of utmost importance to inform stakeholders of any reframing that needs to be made in light of new prevailing trends, forces and priorities. This characteristic creates particular challenges for those stakeholders looking for certainty for negotiations and the formation of subsequent contracts. However, it is contended here that obtaining certainty may simply not be possible and that PPP practitioners, for example, may need instead to embrace this organic nature of megaprojects (in particular) rather than ignore it so that both project risks and opportunities can be better identified and greater resilience introduced to appraisal exercises.
Presented in this manner, the authors caution that PLMCA cannot, and must not, be considered a panacea for mega infrastructure project appraisal. As earlier emphasized, it does however offer a more holistic platform for complex appraisal decision-making for such projects, accommodating at the same time other appraisal expertise and evidence from multi-stakeholder perspectives. What immediately follows is a simple explanation of the generic phases of a PLMCA appraisal process followed (in Section 4.3) by an outline of how PLMCA may benefit PPP applications for MTPs.
Fig. 4
Policy constraints defining the decision-making space in a PLMCA exercise. Source: Dimitriou et al. (2013)
A PLMCA appraisal process carried out in line with the features outlined earlier essentially consists of three key phases, incorporating different activities or steps as follows (Table 1):
The problem-structuring phase: This involves the scoping (definition) of the decision context(s) of the project (political, legislative, spatial, temporal, cultural, etc.), identification of the decision-makers and other key stakeholders involved in the project and specification of the aim(s) of the use of the appraisal analysis (and decision space identified). It also entails the collection of information regarding the possible options appraised, as well as criteria and related targets and indices likely to be used. The model-building phase: This involves defining the key appraisal criteria and deciding on the relative importance attributed to them by stakeholders, typically, of a numerical scale with “weighting”. The model-use phase: This is where the performance of each alternative under scrutiny is determined, employing some form of scoring expressed either quantitatively (where possible) or qualitatively (possibly involving the use of proxy measures/variables). The model output here is to inform the decision-making process rather than direct or dictate it.
Three main groups of stakeholders can be distinguished in any PLMCA appraisal exercise, each having different roles throughout. They are as follows:
The client agency: This is typically the project’s principal sponsor responsible, among other things, for setting the project’s terms of reference (ToR) of the PLMCA appraisal exercise. It is generally charged to oversee the exercise and co-operate with the analyst team (see in the following) by informing/guiding the appraisal process as it progresses. The analyst team: This comprises a small team of experts, typically headed by a specialist advisor, who lead the PLMCA appraisal exercise on behalf of the client agency. It is usually charged with carrying out the primary analysis of the problem(s) identified that gave cause to the need and/or demand for the project in question. The team furthermore, in consultation with the client, identifies key project stakeholders, elaborates project objectives and related performance criteria, mediates discussions and negotiations among stakeholders, presents the results of each step of the PLMCA appraisal process and incorporates new elements learnt during the exercise into subsequent developments where appropriate. The project’s key stakeholders: As earlier indicated, these are the key individuals/groups of individuals and/or organizations/agencies that have a vested interest in the project and its success. They can also include those parties affected by the project and its outcomes, usually supported by commissioned experts to appraise areas of specific interest to one or more stakeholders. More typically, they comprise parties that can influence the project’s outcomes and who commission experts to support/advance aspects affecting their interests. In addition to the client, the project stakeholders are the principal sources for obtaining important information concerning matters such as:
the nature of the problem(s) under investigation; the possible alternative course of actions to address the problem; the development of project objectives and related appraisal criteria; the scenarios to be considered in the appraisal; the weights and scores and the risks and opportunities entailed by each project option under a specific scenario (or scenarios).
Generic phases and of the PLMCA appraisal process and stakeholders. Source: Adapted from Belton and Stewart (2002).
| Phase 1: The problem-structuring phase | Client agency and analyst team (closed approach) | Step 1.1: Problem definition (including issue analysis) |
| Phase 2: The model-building phase | Analyst team (closed approach) | Step 2.1: Formulation of objective/appraisal criteria (adopting an open or closed process) |
| Phase 3: The model-use phase | Analyst team (closed approach) | Step 3.1: Scoring (adopting an open or closed process) |
The premise was forwarded at the outset of this paper that there are significant potential benefits to be had in applying PLMCA to the appraisal of mega infrastructure projects delivered by PPPs and MTPs in particular. The text that follows seeks to elaborate on this premise, explaining further why the authors hold this view and how PLMCA may be applied to the appraisal of MTPs delivered by a DBFO PPP model. The belief these benefits are real rests on the advantages that PLMCA is seen to offer by forging more explicitly links and interdependencies between the public policies and regulations (at different levels and in different sectors) that ultimately (should) frame and shape such developments and the infrastructure investment decision-making undertaken by PPP stakeholders who place greater emphasis on project performance forecasts and indicators in an attempt to ensure that acceptable rates of return and efficiencies are attained.
A common criticism of PPPs is the opaque nature of the decision-making that leads up to the approval of many such projects (World Bank 2015); in some cases, this criticism even extends to how the projects are subsequently managed and operated. By offering a framework that is both more holistic and transparent, and which presents (side-by-side) shared multi-sector and multi-stakeholder perspectives of the costs and benefits (not all monetized) of different options and actions reflective of known public policy and regulative guidelines, it is contended that the accountability of such decision-making is much enhanced. Put more specifically, by consciously seeking to link the numerous policies and regulations (plus their related performance criteria and metrics) with infrastructure investment concerns that have an impact on project viability and performance outcomes in line with pre-agreed PPP specifications and cost yardsticks (again with related performance criteria and metrics), the PLMCA framework offers an invaluable common platform (typically in a matrix form) that facilitates the population of information/data about the project in question and its potential impacts. This shared knowledge enables more informed trade-offs to be made by multiple stakeholders in their negotiations and decision-making about choices. These choices range from more qualitative concerns related to a project’s key agents of change objectives and the complex sustainable development issues they may seek to address at one end of the spectrum to project performance and costing yardsticks of a more quantitative kind that relate to the partnership contract(s) agreed by all parties concerning outcomes and outputs at the other end.
In line with the preceding text, in the context of a PLMCA appraisal exercise for an MTP to be delivered by a PPP employing the DBFO model, three main groups of stakeholders can be identified as participants of a PLMCA appraisal exercise. They include the following:
The client agency: In the case of a strategic national MTP, this is likely to be a lead sponsor government department, such as the Ministry of Transport working closely with say the Ministry of Finance/Treasury (supported sometimes by a specialist government PPP department for extra capacity and capability). As the project’s principal sponsor, this organization will be responsible for setting the project’s ToR and thus the scope of the PLMCA appraisal exercise. It will also be charged with overseeing the overall PLMCA exercise and co-operate with the analyst team by informing/guiding the appraisal process as it progresses about such matters as the realistic assessment of whether emerging objectives can realistically be transferred to an “output obligation” to be taken on by the private sector in a PPP legal and commercial structure. This assessment will be a VLM test and will consider if the private sector can indeed bring innovation to each objective such as, for example, identified social outcomes around skills and employment The analyst team: As earlier indicated, this would comprise a small team of experts headed by a specialist advisor (sometimes partnered with an appointed staff member of the client agency) who leads the PLMCA appraisal exercise. Typically, the team and its head will be from a consultancy organization commissioned by the client and charged with carrying out the primary analysis of the “problematic(s)” presented as the underlying purpose/need for the project in question. By illustration, this could be confirmation as to whether certain policy objectives are likely to be better addressed by the public sector or can successfully be passed to the future private–sector PPP partner. This will be a VLM test and will include assessment of whether there would be clear measurable criteria that would facilitate inclusion and subsequent measurements of delivery (or not) in typical PPP contract documents. In consultation with the client, this team will also identify the key project stakeholders, elaborate/scrutinise project objectives and related performance criteria, mediate discussions and negotiations among project stakeholders, present the results of each step of the PLMCA appraisal process and incorporate new elements learnt during the exercise into subsequent decision-making developments. The analyst team will then be required to support the procurement phase of the PPP as they will be able to assist in the engagement process with industry and support the legal and commercial teams as they plan and complete the procurement process. The project’s key stakeholders: Again, as earlier indicated, key stakeholders are likely to be made up of parties that are judged by the client to have a vested interest in the project and its success. In the context of a MTP delivered by a DBFO PPP, for a major UK transport project and as well as the key sponsoring departments, these would typically include local authority areas that are affected by the scheme; other major infrastructure providers, both public and private; major landowners, both public and private, where land value capture may indeed provide a major source of funding and representation from any major consultation groups already identified. In more enlightened circumstances, they could also include other parties affected by the project and its outcomes, such as community groups of local residents in any regeneration areas. More typically, however, they comprise stakeholders who can directly influence the project’s outcomes as outlined earlier. Together with the client, the key stakeholders are the principal sources for obtaining important information concerning
the nature of the problem(s) under investigation that the project is to address, such as transport capacity, journey times, modal shifts and environmental drivers; the possible alternative course of actions identified, such as existing network upgrades; the development of project objectives and related appraisal criteria to be employed, such as passenger numbers, air quality and direct employment; the scenarios to be considered in the appraisal, such as route and station hub choices; the weights and scores to be applied to the various dimensions, such as environmental, social and institutional and the risks and opportunities entailed by each project option under a specific scenario (or scenarios) such as economic cycles and technology changes.
The following text theorizes in very basic terms how steps of a PLMCA approach might be applied to the appraisal of an MTP delivered by a DBFO PPP model:
The problem-structuring phase: The design and development of the proposed PLMCA framework here would commence with a primary analysis of the challenges, problems and issues that the proposed MTP is to address. This would be undertaken by the analyst team, on behalf of the client. By way of illustration and as alluded to earlier, this could entail an initial framing of the outputs required from the MTP and perhaps two or three major options that will allow through comparison a better understanding of the relative importance of certain policy objectives. Through the analysis of appropriate historical and current legislative and administrative documents, supported by interviews with the project sponsor and other key institutional actors involved in/affected by the project, the analyst team would then arrive at preliminary definitions/clarifications regarding:
the purpose of the appraisal exercise; by illustration, this could be how to achieve a 20% modal shift from cars to public transport and an improved overall traveller experience; the nature (and dynamics) of the problem under investigation; by illustration, this could be where context is important in identifying any major constraints to initially work within, in particular planned for time by which outcomes are to be delivered; the boundaries (political, legislative, spatial, temporal, cultural, etc.) within which the appraisal exercise should be carried out; by illustration, this could be whether there are realistic market scale challenges and/or legislative barriers that would allow a PPP style procurement, more specifically of the pubic private ownership of assets; the possible courses of action to be implemented to address the problems and issues encountered; by illustration, this could be the required programme for enabling legislation to be adopted that would allow key infrastructure to be managed and owned by the private sector; the key project stakeholders involved or affected by the proposed development (besides the client agency); by illustration, this could be as noted above various levels of local government and key commercial land owners and industry; the policy frameworks that frame the project appraisal exercise that all stakeholders should ultimately comply with or look to amend; by illustration, this could be national regional and local policies and would cover both spatial policy (planning) and industry-based development plans such as tourism and industrial and the contextual features of the prevailing economic, social, cultural, political and environmental forces and trends that the decision-making process is unavoidably exposed to; by illustration, this could be core economic growth assumptions and general inflation factors to be scenario tested at a later stage. The model-building phase: Having built a decision-making platform that facilitates consensus building among all partners of the PPP project to address project problems/issues, there is then a need to turn to the fact that there are limits in PPP (indeed in all) arrangements as to how far and how much individual stakeholder interests can be served/fulfilled. At one level, these limits are set by public policy and government plans and guidelines that frame the PPP project, as in the case for example where the provision of clinical care is rarely included in a PPP health project. At another level, limits can also arise from data availability constraints. Other constraints, including budgetary limits, plus time and contextual constraints, can also have an impact on stakeholder project preparations. Once a complete set of objectives has been established, the analyst team (in collaboration with experts often individually commissioned by stakeholders) will then be required to operationalize and transform such objectives into more measurable criteria where possible. For example, targeted employment numbers in agreed sectors and targeted demographic groups during and after the MTP delivery are quantified. The articulation of a set of specific objectives starting from a few generic goals (such as social inclusion) and the identification of a measurable criterion for each objective generally (including outcome-based measures) follow a hierarchical structure as alluded to earlier. In this value tree, general interests (such as an objective to achieve public transparency) are presented on the left and specific measurable indicators of performance stand on the right (such as number of public presentations and freedom of information requests responded to). The value tree is then presented to the stakeholder groups who are asked to validate the project objectives and related criteria listed and, if necessary, propose some additions to this list. The model-use phase: As outlined earlier, experts play an important role during the construction of the impact profile of the options under investigation, providing forecasts, data and judgments about the prospective performances of each alternative. For example, for each scenario/option, a scored matrix can be produced and the project scenarios/options are prioritized. Then, by varying weightings, the stakeholders can gain a better understanding of the areas of real importance to their final choices on the scenario/option. Stakeholders are then exposed to a series of in-house workshops and panel discussions during which the results of these analyses are presented. They would subsequently be asked to score, according to their judgements and evidence before them, the performance of each option against each of the appraisal criteria. In so doing, they can also indicate the quantifiable and non-quantifiable risks (and opportunities) associated with the performance of a given project option against each appraisal criterion. The resultant performance table containing the outcomes of the appraisals can then become a form of risk (and opportunity) register. Different from other MCA methodologies, the preferences of each stakeholder group in a PLMCA approach are not necessarily aggregated immediately but can be presented as a series of stakeholder-specific appraisal tables. This allows an opening up of the analysis to highlight both differences and similarities in the positions of the different actors before moving towards more consensus building of closing-down phases if required. Weights may also be assigned to criteria to reflect their relative importance, for example, in the context discussed here, the relative importance of social inclusion or environmental impact. Weighting, however, is the most controversial stage of any MCA methodology as the adoption of diverse weighting systems by different stakeholders is likely to produce totally different outcomes. There are as a result huge debates over how weights should be defined, as well as who should ascribe them.
A PLMCA framework for the appraisal of MTPs delivered by PPPs can also incorporate scenario planning to test the robustness of project options against different scenarios, where scenarios describe how the driving forces of a development context might behave. This would need to include plausible descriptions of exogenous conditions (economic, social, environmental, political and technological) in a possible and probable future, where the purpose of scenario planning is to identify the key factors and driving forces likely to exert the highest impact on the long-term success of a project. It should be appreciated that while the construction of scenarios is generally the domain of experts, stakeholders can also be involved and be asked to provide their own specific knowledge and perspectives to enrich the overall scenario structure. It should also be noted that in a PLMCA exercise of the kind discussed here, it is envisaged that under different scenarios, shifts in the weights of the criteria, as well as changes in the scores of each option, may occur (Hall 1980). This is so since it is likely that under different political, economic, social, environmental and technological conditions and prevailing values, the importance of the appraisal criteria may change (different weights) and the same option may perform differently (attracting different scores).
Following the above proposed steps, the results of the appraisal are represented by a collection of “snapshots” that can take the form of tables, graphs, photos, charts and diagrams illustrating the performances of the different options according to the perspective of different stakeholders under various scenarios (but also at different time–space scales, according to the different dimensions of sustainability, etc.). The process can also be reiterated if aspects of the problematic remain unclear, or indeed, if the problematic itself needs redefining. The whole exercise thus turns out to be highly dynamic so that judgments regarding the nature of the problem(s) the mega project is expected to address can alter, the key stakeholders involved or potentially affected by the problem may also change, the political relevance of different items may alter and project alternatives or impacts may similarly display sudden changes. To capture these dynamics, feedback loops between the various steps of the process, as are frequent consultations amongst the different parties, and a continuous monitoring of the decision-making context to adapt the decision-making processes to the new emerging conditions are essential. The implications for the appraisal of an MTP delivered by a DBFO PPP model of PPP are challenging given that the private funding model behind PPP seeks to manage risk within a tightly drawn funding envelope. This means that changes in scope and flexibility are expensive to provide for as funds need to be held available on an as needs basis. The point however is to asses more widely than a typical PPP the potential benefits from an MTP, and even if not able to transfer contractual responsibility to the private sector, the objectives are clear to all and final project success can still be measured against this wider framework. This leads to an overall risk reduction as all stakeholders are accountable to deliver what they can.
What makes PLMCA approach so timely is
the increasing recognition globally of MTPs as strategic agents of change with impacts that go well beyond the transport sector; the fast-rising popularity worldwide of PPPs as a means to deliver such projects on the basis of the access they provide governments to private-sector funding as public resources become more constrained and the traditional view no longer prevails that economic performance should be the dominant concern in MTP appraisal because sustainability objectives and related concerns are all moving very fast up the global development agenda.
PLMCA has the potential to shed light on whether PPPs
as applied to the delivery of MTPs meaningfully reduces public sector debt; give private financiers and investors undue influence over national and/or regional strategic development priorities; provide the level of transparency necessary to achieve meaningful public consultation and involvement and transfer risks from the public sector to the private sector in line with the promises/claims made.
The authors also contend that public officials would be much better informed by the use of PLMCA to ascertain whether the long-term contractual arrangements between PPP partners and public sector agencies limit the flexibility of future decision makers to respond to changing conditions. A PLMCA approach to project appraisal could further reveal whether a PPP offers the necessary levels of open communication and trust as platforms for project stakeholders to negotiate and collaborate.
What was disturbing in researching for this paper is how much rhetoric surrounds
the emphasis on iron triangle performance criteria and the resulting notion of failure that has therefore dominated much of the narrative of mega infrastructure development literature in the last 15 years, at the expense of appropriate consideration of such projects as agents of change; the “fit for purpose” claims of CBA and its derivatives, despite their limited scope and capabilities to address issues beyond what can be sensibly monetized and the promises of what PPPs can deliver and the extent to which they ultimately carry the investment risks that otherwise would have remained in the public sector.
What is clear is that a holistic appraisal methodology akin to the kind advocated for MTPs would do much to address many important issues, all too often suffocated by a “business as usual” mentality that primarily looks to delivering projects on time and to cost and specification without wider scrutiny.
What is also apparent is that PLMCA should not be seen a panacea for addressing all outstanding issues of MTP appraisal and delivery. Instead, it offers a holistic, systematically designed and usable platform that can be populated by information provided from multiple stakeholder perspectives (across sectors) that better informs stakeholder negotiations and decision-making.
With information technology advances and software innovations currently racing ahead, the capacity and capability of a PLMCA information platform yielding more benefits and opportunities at greater speeds, and at lower costs, add to the timely prospects of what is being proposed. This is also very important for post-construction monitoring (not discussed thus far).
Evidence from work conducted in the field of PLMCA appraisal studies by the OMEGA Centre to date suggests that while the approach can be identified with generic principles, to be effective, it needs to be bespoke, and nowhere is this more important than with PPP projects.