Accès libre

Societal and strategic perspectives of ex post project evaluation

Ewa Kozień
Ewa Kozień
 et 
Marcin Stanuch
Marcin Stanuch
   | 11 mars 2024
International Journal of Management and Economics's Cover Image
À propos de cet article
Article précédent
Article suivant
Citez
Article Category: Conceptual Paper
Publié en ligne: 11 mars 2024
Pages: 108 - 122
Reçu: 21 mai 2023
Accepté: 19 déc. 2023
DOI: https://doi.org/10.2478/ijme-2024-0002
Mots clés
© 2024 Ewa Kozień et al., published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Introduction

Increased interest in the issues of project evaluation is observed on the part of public and private organizations, but also organized programs and European Union funds. It should be noted that there are differences between the project management in public and private organizations [Tangsgaard et al., 2022].

The literature on the subject develops concepts, models, and methods for evaluating projects from an economic and management perspective, as well as in the field of creating project management methodologies. The issue of evaluating project implementation is a key element of project management, considered in the context of its success or failure. The assessment is the basis for making a decision on the implementation or abandonment of the project. It is worth emphasizing that the evaluation of a project can be made at every stage of its implementation [Helden et al., 2012], using specific criteria, which reduces the risk of not achieving the project’s goal, failing to meet the deadline for its completion, exceeding the planned budget, or failing to achieve the planned effects [Kozień, 2019]. Befani [2010] shows how important it is to identify appropriate criteria in evaluation. In management practice, the most commonly used project appraisal directive is perceived in terms of time and costs. Thus, the effectiveness of project implementation means its implementation is “fast” and “cheap,” which has business justification in the context of the competitiveness of organizations operating on the market. Simushi and Wium [2020] point to the importance of project evaluation in terms of time and cost due to the frequent occurrence of the problem of exceeding both parameters. Many researchers reduce the assessment of projects to two (time and cost) [Rani, 2013] or three (time, cost, and quality) parameters of the “iron triangle,” pointing to difficulties in assessing project quality [Belout and Gauvreau, 2003; Wyngaard et al., 2012; Caccamese and Bragantini, 2013; Rani, 2013; Smith and Magnusson, 2015; Silva et al., 2016; Pollack et al., 2018]. There are also conclusions where the importance of quality is considered to be the most important element of competitiveness [Stojcetovic et al., 2014]. In the literature on the subject, there are views of researchers who believe that the use of the iron triangle in project evaluation does not determine the success of its implementation [Shenhar, 2004; Toor and Ogunlana, 2010; Caccamese and Bragantini, 2012]. There are also statements that the success of the project is multidimensional and does not only concern the parameters of the iron triangle [Pinto et al., 2021].

Already in 1998, Grundy [1998] pointed out that strategy implementation and project management do not have to be independent, and tools from strategic management can be imported into project management. In some studies on project implementation, attention is paid to the implementation of strategic goals [Haniff and Galloway, 2022].

A project is an activity that, within the time and cost constraints imposed, aims to achieve specific goals [Murray-Webster and Dalcher, 2019]. However, as Volden and Welde [2022] point out, in the case of public projects, the intention is very often to bring benefits to a wide range of people. Therefore, the effects of project implementation (implementation benefits) should also be considered from the societal perspective [Samset and Volden, 2016]. As the Project Management Institute (PMI) puts it, the societal effect is increasingly taken into account by specialists implementing projects and is an important step toward achieving the Sustainable Development Goals (SDG) [PMI, 2020]. This means that the ex post evaluation of a project implementation of a specific type should also include the societal benefit criterion.

In recent years, the concept of project success has been increasingly discussed in the project management literature [Pinto et al., 2021]. In particular, when talking about the success of the project, attention is paid to the purpose of the projects and the benefits that their implementation brings to various groups [Ika, 2009; Breese et al., 2015; Zwikael and Meredith, 2021]. The literature usually proposes methods (tests) for assessing the success of a project, separate for private sector projects (mainly based on the iron triangle principle) and separately for the public sector (including societal effects).

Several approaches to ex post evaluation of project implementation can be found in the project management literature. An interesting model is the OECD Development Assistance Committee [OECD, 2002], used, e.g., by the World Bank to evaluate various types of projects, which includes five criteria: efficiency, effectiveness, impact, relevance, and sustainability. In 2019, the OECD model was extended by the sixth criterion—coherence [OECD, 2019]. Kozień [2019] proposed a quantitative method for the ex post evaluation of project implementation, resulting in a single quantitative measure of project implementation effectiveness. Each criterion was assessed on a scale of 1–5 according to the Likert scale [Likert, 1932]. The method included six criteria (project goal, quality, time, cost, societal effects, and strategic effects). The proposed method is basically dedicated to soft projects or hard projects generating societal or strategic effects.

The discussion conducted by many researchers regarding the selection of criteria and methods of project evaluation allowed for the formulation of the research problem; whether the assessment of the project taking into account two (time and cost) or all parameters of the “design triangle” is not too much of a simplification in the context of the uniqueness of project solutions that significantly affect the civilization and technological development of societies on a global scale. The following research questions were formulated:

Is limiting the evaluation of projects only to time and cost parameters sufficient?

Is it appropriate to adopt only the parameters of the “design triangle” (time, cost, and quality) for project evaluation?

Will the inclusion of additional criteria for the evaluation of projects enable its objectification in the context of a broader view of the benefits for the development of civilization?

What criteria should be included in the project evaluation?

The research objective is to identify the key criteria for ex post evaluation of the implementation of projects that go beyond the “project triangle.” The subject of the study is projects diversified in terms of field, size and level of innovation. Only completed projects were evaluated. The study was conducted in two stages. In the first stage, completed projects were assessed. The proprietary multicriteria method of ex post evaluation of project implementation was used to evaluate the projects. In the second stage, based on the results of the project evaluation, an analysis of the key criteria for ex post evaluation of project implementation was carried out using Hellwig’s correlation method [Hellwig, 1968].

The novelty of the proposed project implementation effectiveness assessment method is the inclusion in the assessment of societal and strategic effects that may be important in the assessment of a certain group of projects, in particular soft projects. The proposed method refers to the comprehensive consideration of strategic and societal criteria and defines one measurable parameter, the value of which is the basis for ex post evaluation of project implementation. The presented method can be helpful in public management, when social expectations are broader than just the execution of the work itself, which may be the construction of a building, e.g., a hospital, school, organizing a concert, exhibition, or marking an architectural and educational trail. From the point of view of the implementation of public projects, it is very important for many of them to achieve societal effects, and in a broader perspective, strategic effects. The proposed and discussed method, universal and flexible in its formulation, allows to achieve such a goal.

The applied research method using quantitative analyses is in line with the research methodologies used by the author and related to the mathematical modeling of phenomena in economics and management. An example is the use of interval analysis to measure uncertainty in the identification of the stage phase of companies [Kozien and Kozien, 2017], the use of fuzzy logic for the ex ante risk assessment in the project [Kozien and Kozien, 2018], or using the risk-list method with statistical analysis for risk assessment in the project [Kozień, 2018]. In the discussed in the article problem, the mathematical approach concerns the quantitative description of individual factors affecting the assessment, and in particular, proposing a new quantitative assessment of the impact of strategic and social factors, and then performing a detailed correlation analysis.

The research hypothesis is as follows: “for the ex post evaluation of project implementation, it is not enough to rely on time and cost criteria, but it is necessary to supplement these criteria with strategic and societal effects.”
Materials and methods
Research methodology

In connection with the implementation of the research goal, a research methodology was formulated including the discussed below steps.

In Step 1, a critical analysis of the literature on the subject made it possible to identify a research gap and formulate a research problem; whether the assessment of the project taking into account two (time and cost) or all parameters of the “design triangle” is not too much of a simplification in the context of the uniqueness of project solutions that significantly affect the civilization and technological development of societies on a global scale. Four research questions were also defined, and the research objective and the research hypothesis were formulated. In Step 2, a case study was used to analyze each project. Qualitative data and quantitative data on completed projects were analyzed on the basis of the available documentation (the type of project, as well as time, cost, and quality were defined in the context of compliance or exceeding of individual parameters with the project’s baseline plan) and finally aggregated. In Step 3, based on a critical analysis of the literature on the subject and the obtained quantitative and qualitative data on each of the 109 projects, the researched projects were grouped by field (Table 1), and then the field classification of projects was used to distinguish 60 projects, the so-called hard and 49 soft ones related to investment in human development. The ex post evaluation of the projects was based on a quantitative method, which includes the following six criteria: purposefulness, time, cost, quality, strategic effects, and societal effects. Projects are assessed on a five-point scale [Likert, 1932]. In Step 4, the obtained results of the ex post assessment of implementation of projects are used to analyze correlation of the key ex post evaluation criteria of projects by the Hellwig correlation method. In Step 5, conclusions coming from the evaluation and correlation analysis are formulated.

Groups of investigated projects by their types

Project group (codes) Project type Number of projects
Cultural(C1–C10) Soft 10
Investment(I1–I28) Hard 28
Student(S1–S12) Soft 12
IT(IT1–IT14) Hard 14
Sports(SP1–SP4) Soft 4
Societal(SL1–SL8) Soft 8
Researchanddevelopment(RD1–RD13) Hard 13
Innovative(IE1–IE5) Hard 5
Educational(EL1–EL15) Soft 15
Total number of projects 109
Method of ex post evaluation of projects

Evaluation (Latin aestimatio) means value or estimation and has a valuing character [Słownik Języka Polskiego, 1978]. Each assessment shall be made according to quantitative and/or qualitative criteria. Depending on the number of criteria taken into account, one or multicriteria assessment methods can be distinguished. The problem of evaluation was referred to the projects.

The evaluation of projects as unique activities requires the definition of the timing and criteria for their evaluation in the context of the implementation of diverse projects. The timing of the project evaluation can be linked to the phases of its implementation, hence the ex ante evaluation of the project corresponding to the initial phase, the on-line evaluation relevant to the implementation phase, and the ex post evaluation carried out immediately after the end of the project. In line with the stated research objective, the issue of evaluation of projects was considered immediately after their completion (ex post evaluation). Ex post evaluation is an overall assessment of a project according to the adopted criteria. The overall assessment of the project is influenced by the partial assessments made at the stage of project preparation and implementation. However, there is a significant difference between the partial evaluation (ex ante, on line), which may be adjusted during the project implementation (e.g., in terms of the degree of achieving the goal, time management, and cost), and the overall evaluation of the project (ex post), which is a final assessment without the possibility of making changes to the project. The overall assessment of the project concerns: the effectiveness of achieving the project objectives in accordance with the specific needs, rational management and management of resources (time, cost, quality), the impact on the environment of the effects achieved in the project, both planned and unexpected, and maintaining the achieved effects after the end of the project.

In management practice, a commonly used directive on ex post project evaluation can be seen due to the two criteria: time and cost; it means that the project is completed quickly and cheaply, which has a business justification in the context of the competitiveness of organizations operating on the market. However—should the assessment not take into account additional criteria due to the diversity of projects? In the conducted study, for the ex post evaluation of the implementation of various projects, the method which belongs to the multicriteria methods and includes the six criteria was applied. The catalog of ex post evaluation criteria for assessment of project implementation, due to the domain differentiation of projects, takes into account the purpose and basic parameters of the so-called “iron triangle,” that is: project goal, time, cost and quality, and additionally strategic and societal effects. The strategic effects of the project concern the assessment of its scope. It is possible to choose between the international, national, regional, and local scope of the project. An original checklist was created to assess societal effects in projects [Kozień, 2019]. The assessment of each of the above-mentioned criteria is arbitrarily and subjectively done, assuming the value of the coefficient of the i-th partial rating factor si for the i-th criterion (i = 1, …, 6). Then, the total rating factor s is determined according to formula (1). The criteria are connected with: purpose (s1), time (s2), cost (s3), quality (s4), strategic effects (s5), and societal effects (s6) in the extended method. The assumed weight values for individual criteria are justified as follows: the sum of the weights for the iron triangle is 0.5; time and cost have the same weight; quality is more important than time and cost understood independently; time and cost taken as a whole weigh more heavily than the goal; purpose and quality are of the same (and most individually) importance, due to the importance of the parameters; goal, strategic effects, and societal effects have a total weight of 0.5; and strategic effects and societal effects have the same weight (and the same as time and cost). s=0.2s1+0.15s2+0.15s3+0.2s4+0.15s5+0.15s6 $${\rm{s}} = 0.2{{\rm{s}}_1} + 0.15{{\rm{s}}_2} + 0.15{{\rm{s}}_3} + 0.2{{\rm{s}}_4} + 0.15{{\rm{s}}_5} + 0.15{{\rm{s}}_6}$$

The values of the partial rating factors are assumed as follows:

Criterion 1: achievement of the project goal. The s1 parameter is a quantitative description of the degree of compliance of the objectives with the needs defined in the project and takes three values: s1 = 1—the goal has been achieved; s1 = 0.5—the goal was partially achieved; s1 = 0—the goal was not achieved.

Criterion 2: project implementation time. The s2 parameter is the ratio of the actual project implementation time to the planned time; in such a description, the value of the partial assessment coefficient and in a discrete description takes five values: s2 = 1.2—implementation below the planned time; s2 = 1—implementation in the planned time; s2 = 0.6—slight time delay (<20%); s2 = 0.2—large time overflow (between 20% and 50%); and s2 = 0—unacceptable timeout (over 50%).

Criterion 3: project implementation cost. The s3 parameter is the ratio of the actual cost of the project to the planned cost; in such a description, the value of the partial assessment coefficient; and in a discrete description takes five values: s3 = 1.2—implementation below the planned cost; s3 = 1—implementation at the planned cost; s3 = 0.6—slight cost overrun (<20%); s3 = 0.2—large cost overrun (between 20% and 50%); and s3 = 0—unacceptable cost overrun (over 50%).

Criterion 4: quality. The s4 parameter is a quantitative description of the implemented quality of the project, bearing in mind the planned quality, which meets the requirements specified by the quality standards, and regarding the ability to meet the identified and expected needs and takes four values: s4 = 1—planned quality achieved; s4 = 0.7—average planned quality; s4 = 0.3—low planned quality; and s4 = 0—no planned quality. The quality assessment is individualized depending on the type of project. Standards, patterns, and expert opinions may be helpful in assessing the quality. For a given type of project, checklists should be developed to identify the degree of deviation from the planned quality.

Criterion 5: achieving strategic effects. The s5 parameter describes quantitatively the comparison of the actual strategic effects achieved through the project with the assumed and unplanned strategic effects in the project and takes six values: s5 = 1—strategic effects of international multilateral scope were achieved; s5 = 0.8—reached strategic effects with a range international bilateral; s5 = 0.6— national strategic effects were achieved; s5 = 0.4—strategic effects of a regional range were achieved; s5 = 0.2—local strategic effects were achieved; and s5 = 0—no strategic benefits are achieved.

Criterion 6: obtaining societal effects. The s6 parameter describes comparison of the actual societal effects achieved through the project with the assumed and not-assumed societal effects in the project. The assessment was made using the checklist, which includes four synthetic criteria: development-oriented, integration-related, security-related, and basic societal. Each of these criteria has the same weight of 0.25. Exceptionally, the values of the assessment of societal effects s6 in the project assessment are summed up from among those listed below partial assessments coefficients s61, s62, s63, and s64, meeting the condition (2). Achievement of development-oriented, integration-oriented, security-related societal effects and the achievement of basic societal effects give finally the s6 societal effect assessment value equal to one. The partial assessments coefficients take the following values: s61 = 0.25—development-oriented societal effects were achieved; s61 = 0—development-oriented societal effects were not achieved; s62 = 0.25—integration-oriented societal effects were achieved; s62 = 0—integration-oriented societal effects were not achieved; s63 = 0.25— societal effects related to safety were achieved (concerns the environment of human functioning); s63 = 0—no safety related societal effects were achieved; s64 = 0.25—basic societal effects have been achieved (securing human existence/life needs); and s64 = 0–basic societal effects have not been achieved.

 s6=s61+s62+s63+s64 $${{\rm{s}}_6} = {{\rm{s}}_{61}} + {{\rm{s}}_{62}} + {{\rm{s}}_{63}} + {{\rm{s}}_{64}}$$

In reference to the Likert scale [Likert, 1932; Bhote and Bhote, 2000; Geoff, 2010], five levels of project rating are introduced, calling them convention and binding with the value of the total rating factor s (1). Individual border values are arbitrarily adopted, depending on the project implementation area and subjective rating. The final project evaluations are: A—exceptionally high rating of implementation of the project (s > 0.88), B–high rating of implementation of the project (0.68 < s ≤ 0.88), C—average rating of implementation of the project (0.48 < s ≤ 0.68), D—low rating of implementation of the project (0.28 < s ≤ 0.48), and E—inaccurate implementation of the project (s ≤ 0.28).

The detailed results of the ex post rating of projects implementation are given in Table A1 in the Appendix.
Correlation analysis

The Hellwig’s method was used to determine the optimal econometric model. In the case of the model, the distinguished criteria will be called the explanatory variables, and the final rating—the dependent variable. Hellwig’s method is based on the study of the correlation between the dependent variable (y) and the explanatory variables (X1, X2, …, Xn, where n determines the number of variables) by building a matrix of correlation coefficients R (3) [Hellwig, 1968].R0=[r1r2⋮rn], R=[1r12…r1nr121…r2n⋮⋮⋱⋮r1nr2n…1]

For the model, one should choose those variables that are strongly correlated with the dependent variable and poorly correlated with each other [Gałecka and Smolny, 2018]. In order to determine the nature of the interdependence of individual variables with each other, the linear correlation coefficients were calculated. The estimator of the linear correlation coefficient r^ is defined as (4), where: x¯, arithmetic mean of x values; y¯, arithmetic mean of y values; and x, y, value of selected criteria [Aczel and Sounderpandian, 2009].r^=∑(x−x¯)(y−y¯)∑(x−x¯)2∑(y−y¯)2

The correlation coefficient ranges from -1 to +1. Another condition for recognizing variables as explanatory variables is their sufficient variability. If there are variables in the set with low variability, they should be excluded from the model. To this end, the level of volatility should be determined (5), where: σk is the standard deviation of a given variable; xk− is the arithmetic mean of all units of a given variable [Abdi, 2010]. vk=σkxk− $${v_k} = {{{\sigma _{\rm{k}}}} \over {\overline {{{\rm{x}}_{\rm{k}}}} }}$$

At this stage, a certain critical value for the level of variation should be established depending on the type of study. It was assumed in the study that the level of variability of features (Vk) cannot be <0.1, variables below this value will be eliminated from the model. In fact, it is difficult to meet all the assumptions of a classical linear regression model and the variables are often correlated to some extent. After the initial elimination and selection of appropriate explanatory variables, the optimal model is selected, which is related to carrying out the L = 2n–1 combination, where n refers to the number of explanatory variables. For all variables specified in the combination, individual information capacity indices can be calculated in the form (6), where: hkj—individual indicator of information capacity, rj—value from the correlation vector R0, rlj—value from the correlation matrix R, l—combination number, j—variable number in the combination, and mk—number of variables in k-th combination [Hellwig, 1968].hkj=rj21+∑l=1mk|rlj|

The next step is to calculate the integral capacity of all combinations (7) [Hellwig, 1968].Hk=∑j=1mkhkj(k=1,2,…,2m−1)

From the range of integral capacities (Hk), it is necessary to choose the combination that has the best result (the integral capacity is the highest).

The aim of the study is to assess the correlation between the number of variables influencing the final assessment of the implemented project and to find the best combination of variables describing it. The study was based on the evaluation of 109 projects defined by five criteria: time, cost, quality, strategic effects, and societal effects. The purpose criterion was not taken into account for the analysis and indication of the key criteria of the ex post evaluation of projects using the Hellwig correlation method. This is due to the fact that for each of the presented projects, the goal has been achieved, and for all the projects, this criterion is equal to 1. From the point of view of the study in question, this value does not meet the assumptions regarding a certain asymmetry of the explanatory variables in the model. The research was divided into three cases: case I—soft projects (49 projects); case II—hard projects (60 projects); and case III—soft and hard projects (total 109 projects).
Results

The project ex post evaluation criteria presented in the previous section were subject to preliminary analysis in order to check whether they meet the formal criteria and can be defined as explanatory variables. Table 2 presents the results of calculations of the level of variability of the data used, broken down into relevant research samples.

Coefficients of variation of individual project criteria (i = 2,…,6) for cases I-III

Variable name Variable symbol si Case I si(I) Case II si(II) Case III si(III) Average (si(I) + si(II) + si(III))/3
Time s2 0.09 0.32 0.24 0.22
Cost s3 0.13 0.37 0.29 0.26
Quality s4 0.08 0.13 0.11 0.11
Strategic effects s5 0.67 0.72 0.71 0.70
Societal effects s6 0.33 0.38 0.37 0.36

The greatest variability (according to Table 2) is characteristic of strategic effects and societal effects, where their average value is, respectively, 0.70 and 0.36. It proves their different influence on the final evaluation of the project, depending on the characteristics of the project. Another element is the cost, which, as in the case of the previous evaluation criteria, will be included in the research. The situation is slightly different in terms of time and quality, where the initial requirements were not met and case I was not included in the study. This means that in the case of soft projects, such criteria do not constitute the main priority in the implementation of the project.

The next step was to determine the correlation matrix between the chosen explanatory variables and the dependent variable. The results of the calculations of this stage are presented in Tables 3–5, where the vector of the linear correlation coefficients of the explanatory variables and the dependent variable and the matrix of the correlation coefficients between the potential explanatory variables were included in one—strict structure (1). The names of individual variables were presented by means of their symbols (in accordance with Table 2).

Correlation matrix for soft projects (case I)

S s3 s5 s6
S 1.0000 0.5560 0.5370 0.4590
s3 0.5560 1.0000 0.0980 0.1200
s4 0.5370 0.0980 1.0000 0.0510
s5 0.4590 0.1200 0.0510 1.0000

Correlation matrix for hard projects (case II)

S s2 s3 s4 s5 s6
S 1.0000 0.8110 0.8320 0.5040 0.2840 0.2570
s2 0.8110 1.0000 0.7800 0.2600 0.1360 0.1500
s3 0.8320 0.7800 1.0000 0.3070 0.1390 0.1020
s4 0.5040 0.2600 0.3070 1.0000 0.0230 0.1320
s5 0.2840 0.1360 0.1390 0.0230 1.0000 0.6460
s6 0.2570 0.1500 0.1020 0.1320 0.6460 1.0000

Correlation matrix for all projects (case III)

S s2 s3 s4 s5 s6
S 1.0000 0.7870 0.8170 0.4960 0.2600 0.3710
s2 0.7870 1.0000 0.7470 0.2230 0.1260 0.0020
s3 0.8170 0.7470 1.0000 0.3310 0.1300 0.0150
s4 0.4960 0.2230 0.3310 1.0000 0.0220 0.1290
s5 0.2600 0.1260 0.1300 0.0220 1.0000 0.3340
s6 0.3710 0.0020 0.0150 0.1290 0.3340 1.0000

According to the assumptions of the model, the explanatory variables should be weakly correlated with each other and strongly with the dependent variable. The correlation thus obtained (designated conventionally as “r”) can be interpreted as follows [Kafle, 2019]: 0 < r < 0.4—low correlation; 0.4 ≤ r < 0.7— middle correlation; 0.7 ≤ r < 1—high correlation; and r = 1—completely positive correlation.

Thus, referring to the above-mentioned ranges, it was assumed that the correlation values of the explanatory variables with those explanatory variable that fall within the low correlation range (0 < r < 0.4) would be rejected from the model. In the case of correlations of explanatory variables with each other, those whose value is in the range of high and total correlation (0.7 ≤ r ≤ 1) will not be taken into account. It was assumed that possible values of standard deviation for the selection of variables for the model are ± 0.1 (for these ranges) due to the importance of the assessment criteria. Hence, the following explanatory variables were selected for detailed analyses done as previously for the three group of projects: case IV (soft projects): cost (s3), strategic effects (s5), societal effects (s6); case V (hard projects): cost (s3), quality (s4) and case VI (soft and hard projects): time (s2), cost (s3), quality (s4), and societal effects (s6).

In case II and case III, a significant correlation can be noticed between the time and the cost of the project implementation. Projects in which time is one of the main determinants of the finalization strongly influences the costs that must be incurred to complete such a project on time. Shortening the project implementation time usually requires greater involvement of human and material resources, thus incurring greater financial outlays to achieve the intended goals. For this reason, a strong correlation between these two variables is noticeable; therefore, in the case of the sample on hard projects, the factor that was less correlated with the dependent variable, i.e., time, was rejected (the difference with cost was 0.022). In case III, a decrease in the correlation between the aforementioned criteria can be noticed; therefore, due to the importance of these two criteria for project evaluation, the authors of the study decided to include time and cost in the econometric model within all analyzed projects. This will allow for the verification and presentation of a full view of the evaluation of all the considered projects, thus remaining within the assumed margin of deviation, when selecting the explanatory variables for the model. After full selection of data and selection of the most optimal parameters for the model, individual and integral indicators of information capacity were calculated. Finally, the following numbers of the project fulfilled the discussed requirements and analyzed by Hellwig correlation method: 7 soft projects (for the case IV), 3 hard projects (for the case V), and 15 projects in total (for the case VI). The results of the Hellwig correlation analyses are given in Tables 6–8.

Values of individual and integral information capacities for selected soft projects from case IV

k s3 s5 s6 Hk
1 0.3096 0.0000 0.0000 0.3096
2 0.0000 0.2880 0.0000 0.2880
3 0.0000 0.0000 0.2105 0.2105
4 0.2819 0.2622 0.0000 0.5441
5 0.2765 0.0000 0.1879 0.4644
6 0.0000 0.2740 0.2003 0.4743
7 0.2541 0.2506 0.1798 0.6845

Values of individual and integral information capacities for selected hard projects from case V

k s3 s4 Hk
1 0.6918 0.0000 0.6918
2 0.0000 0.2537 0.2537
3 0.5293 0.1941 0.7235

Values of individual and integral information capacities for all selected projects from case VI

K s2 s3 s4 s6 Hk
1 0.6194 0.0000 0.0000 0.0000 0.6194
2 0.0000 0.6676 0.0000 0.0000 0.6676
3 0.0000 0.0000 0.2456 0.0000 0.2456
4 0.0000 0.0000 0.0000 0.1375 0.1375
5 0.3546 0.3822 0.0000 0.0000 0.7369
6 0.5066 0.0000 0.2009 0.0000 0.7075
7 0.6180 0.0000 0.0000 0.1372 0.7552
8 0.0000 0.5017 0.1845 0.0000 0.6862
9 0.0000 0.6581 0.0000 0.1355 0.7936
10 0.0000 0.0000 0.2175 0.1218 0.3394
11 0.3145 0.3214 0.1581 0.0000 0.7940
12 0.3542 0.3791 0.0000 0.1352 0.8685
13 0.5057 0.0000 0.1817 0.1216 0.8089
14 0.0000 0.4962 0.1682 0.1203 0.7847
15 0.3142 0.3191 0.1460 0.1200 0.8993

From a series of obtained data, the final values were those with the highest integral capacity of the examined case (H). The analysis showed that the success of soft projects (case IV) is mainly a derivative of costs and strategic effects, where the value of each of them influencing the final evaluation of the project exceeded 0.25. The third significant criterion is the societal effects, the econometric model of which allowed to estimate the value of approx. 0.18. Such results are justified by the characteristics of soft projects, the implementation of which usually concerns societal projects (e.g., societal or professional integration), training courses (increasing the qualifications of employees), as well as conducted research enabling the implementation of innovative solutions based on knowledge.
Discussion

In the case of hard projects (case V), the evaluation of the implemented project is influenced by the cost, the value of which was about 0.53, while in the case of quality, this result is much lower and amounts to 0.19. Similar conclusions were obtained by another team of researchers who determined that in the case of construction projects, the cost is the most important [Banihashemi et al., 2021], the results were obtained using the SWARA-TOPSIS fuzzy method. Faten Albtoush et al. [2020] in their study state that a successful project is limited by three criteria: cost, time, and quality, but their main item is cost. In the case of the study by Senouci et al. [2016] carried out on 122 construction projects, it was shown that the cost overrun of the construction project was statistically significant at a significance level of 0.05 based on a linear regression model.

The last group analyzed were all projects (case VI) for which the significant criteria are time and cost, amounting to approximately 0.32 of the total project evaluation value. Elements such as quality and societal effects have less influence here, where the coefficient is 0.14 and 0.12, respectively. This value is more than two times lower than in the case of the above-mentioned measures. The sequence of actions aimed at achieving the set project objectives assumes that for most projects the criteria of the “iron triangle” will be taken into account in the first place, i.e., time, cost, and quality. Each of these three criteria affects each other (you cannot change one without affecting the others). Quality can be defined as the result of multiple use of these three criteria. Considering case III, without taking into account the criterion of time, the final result would change from H15 = 0.8993 to H14 = 0.7847, which proves a relatively lower accuracy of the re-projection of the econometric model. It should not be argued that the use of the model in the case where the integral value (H) is lower is associated with misinterpretation; on the contrary, in some projects (where time is not the main determinant), it may be more effective. For such projects, cost would remain a significant criterion, the impact of which on the final grade is almost 0.5. The quality is slightly better here, the value of which has increased by approximately 0.02 percentage points. In the case of societal effects, the value did not change significantly.
Conclusions

The analysis allowed to determine the significance of the indicated five criteria, i.e.: time, cost, quality, strategic effects, and societal effects in the ex post assessment of the implementation of the project. An in-depth analysis in this study was made for three separate research samples, which were distinguished due to the domain diversity of projects. The results of the analysis allowed to formulate the following conclusions:

In the case of soft projects, strategic and societal effects have an impact on the ex post evaluation in addition to costs. In citizen projects, for example, they have a greater impact than quality and time criteria. The indicated ex post evaluation criteria in soft projects correspond to the specificity of societal projects related to broadly understood development. Therefore, when implementing such projects, it is important to remember about the appropriate specification of the goal and striving to achieve the expected societal effects and increasing their area of influence.

In the case of hard projects, the key criterion for their ex post evaluation is cost. The weight of the cost for the final evaluation of the project is approximately 53%. Hard projects are mainly of an investment nature; therefore, the possibility of their implementation particularly depends on the financial capital, which was confirmed by the analysis carried out using the Hellwig correlation method.

The key criterion for the three research samples, i.e., for all projects subject to ex post evaluation, is cost. This means that the financial aspect in the project is the basic element that allows to compile the financial resources needed to cover the project implementation costs and has an impact on the construction of the project budget. On the basis of the conducted study, the time criterion was specified as crucial in the assessment of all projects and not included in the analysis of soft and hard projects, which means that although time management in projects is important, in the case of distinguished project types, the emphasis in their assessment should be directed to qualitative criteria related to strategic and societal effects.

The analysis of the key ex post evaluation criteria for all projects indicated the cost, time, and quality, i.e., the parameters of the so-called “iron triangle.” However, from the point of view of appraisal units and due to the thematic diversity of projects and their specificity, the authors believe that it is justified to include criteria such as strategic and societal effects in the ex post evaluation.

Defining the key criteria that have a significant impact on the ex post evaluation of the project allows you to build a well-thought-out strategy aimed at striving for consistent completion and to ensure the maximum level of stakeholder satisfaction.

The discussed method of quantitative project implementation assessment, including the iron triangle and societal and strategic effects criteria, can be dedicated to any type of project with the appropriate definition of criteria and proper selection of weights. In this sense, it is universal and flexible.

The article presents the issue related to the selection of appropriate ex post evaluation criteria for hard projects, soft projects, and all projects under analysis. It is worth nothing that the selection of key ex post evaluation criteria for projects determines the approach to their management. Hence, it is important to answer the question as to how we want to achieve the intended results in the project. Is the directive fast (time) and cheap (cost) is enough for assessment? Or should project evaluation take into account criteria related to societal effects and their range (strategic effects)? Should not the evaluation of projects as unique activities influencing changes and societal development as well as technical progress be extended with other important criteria? The research objective has been achieved, the research hypothesis has been positively verified, all the research questions have been answered, and the solution to the research problem is important for the professionalization of project evaluation methods in the context of their fuller evaluation as unique activities.

According to the authors, for the sake of sustainable development, the strategic and societal criteria should be raised and noticed in the ex post evaluation of projects by stakeholders. Despite the diversity in the field, as well as between public and private projects, this key issue of project evaluation should be developed in future research.
eISSN:
2543-5361
Langue:
Anglais
Périodicité:
4 fois par an
Sujets de la revue:
Business and Economics, Political Economics, Economic Theory, Systems and Structures, Business Management, Management Accounting, Financial Controlling, Cost Calculation, Investment
RSS Feed de la revue