Uncovering the hidden gem: The role of the undervalued quality in projects

In the most influential definitions, time and cost are usually considered constraints as two crucial factors for the project and the project manager (Wright 1997; Shenhar et al. 2001; Ashkanani and Franzoi 2022). However, other important factors also determine project success. For example, the Iron Triangle, also known as the Project Management Triangle, is a concept that describes the relationship between three key components of a project: scope, time and cost. In addition to scope, quality also plays a crucial role. At the theoretical level, the quality is apparently present in the definitions (Juran and Godfrey 1998; Kerzner 2009; International Organization for Standardization (ISO) 2015a; Project Management Institute 2017), but its economic relevance and the precise determination of its importance are not addressed in the literature. However, quality is a critical element of the project because it defines the level of excellence expected for its deliverables (Basu 2014; Pinto 2016). Some researchers propose different options for the Iron Triangle components, suggesting that quality could be substituted with scope. This viewpoint is widely accepted and has been advocated by researchers such as Badewi (2016) and Van Wyngaard et al. (2013).

This exploratory study aims to explore the influence of traditional success factors (time, cost, and quality [TCQ]) on project outcomes, specifically focusing on their association with economic benefits. Traditionally, various factors have been examined for their impact on project success. Identifying the extent of TCQ factors’ role in projects with economic benefits can offer valuable insights for project management stakeholders. Moreover, investigating the significance of individual TCQ factors in determining economic project outcomes can enhance current project management practices and open avenues for further research in this field. A stopgap and novel index Project Unified Index (PUI) is introduced for a comprehensive economic performance assessment based on TCQ.

The Iron Triangle is a project management framework that balances cost, time and quality, with the understanding that changes in one area will affect the other two. Although there is an ongoing debate about whether the Iron Triangle is the sole measure of project success, with some advocating for additional criteria, it is widely accepted that cost, time and quality remain crucial and fundamental parameters in project management. Despite efforts to establish success criteria in project management, there is a lack of consensus due to divergent perspectives and recommendations, with quality often overlooked as a primary criterion. At lower levels, measuring quality is relatively straightforward, as well-established metrics exist across various industries (e.g. rework rate, defect rate, overall equipment effectiveness [OEE], first pass yield [FPY] in manufacturing) (Ahmed 2013; Jaqin et al. 2020; Ádám and Sebestyén 2023). However, as a more customer-centric approach is adopted, qualitative and subjective factors become significant, making it challenging to define good quality. Quality, being subjective and hard to measure, often becomes a secondary yet impactful criterion (Moorthy et al. 2011; Pollack et al. 2018). Subjective measures consider perceptions and feelings, whereas objective measures focus on quantitative assessments (Ponkanti and Madoun 2015). Measuring quality at higher levels presents many challenges and is much more difficult than determining cost or time in projects. Further research is still needed to identify relevant quality metrics and measurement techniques to integrate quality as a primary success criterion in project management practices for a long time.

The data used in this study were collected through a questionnaire survey. The questionnaire contained mainly questions about TCQ factors. These can be answered on a five-point or a three-point Likert scale. Time and cost are easily measurable and can be well ranked on a proportional scale. In both cases, we set the threshold at 30%. For instance, regarding time, if the project is not completed exactly on time, a 30% delay is considered a threshold between a substantial and a minor delay (‘Late’ is up to +30%, ‘Much late’ is later than +30%). Similarly, if the project is completed earlier by <30%, it is categorised as ‘Earlier’, but if the completion is >30% ahead, it is labelled as ‘Much earlier’. We also set the budget threshold at 30% similarly (such as ‘Much over budget’, ‘Over budget’). Respondents provide their answers unambiguously using a five-point Likert scale.

For quality, we adhered to the typical industry standard of quality determination in the construction industry, which examined how the project meets its specifications and requirements. Due to the lack of a quantifiable proportional measure for quality, it was assessed using a simpler three-point scale. This approach acknowledges the challenge of perceiving quality, making a more detailed scale, such as a five-point scale, less practical. Participants can assess whether the project is completed in line with or better/worse than the expected quality.

The use of Likert scales in the social and business management field is considered standard. However, several criticisms have been published about their application (Holt 2014; Árva et al. 2019). Since the intervals between the values measured on an ordinal scale are not considered equal, comparing such variables using arithmetic mean and standard deviation is controversial (Norman 2010; Vargha 2002). In addition to the problem of the applicability of statistical methods, Árva et al. (2019) point out in their study that the Likert scale forces the respondent to choose between a few values based on their subjective perception when filling in the questionnaire. An additional problem may be the loss of varying performance associated with an accurate assessment. In practice, this usually results in a higher frequency of one or two categories than others, making it difficult to detect the small but possibly significant differences between variables or variable groups that the data show. In such cases, the indicator that can be used instead of the arithmetic mean, the median, is often the same for many identical responses and does not show any meaningful difference between the variables under consideration.

The use of nominal and ordinal scales cannot be avoided in business sciences, but the analysis, the choice of indicators used and the conclusions drawn must take these properties mentioned above into account. Consequently, the test of means often shows no difference for ordinal scales, which can sometimes lead to the incorrect conclusion that the effect of the variables is the same.

The literature offers several solutions to overcome the problems above, many of which require considerable mathematical or statistical knowledge. However, in our opinion, their application in business practice may be hindered by the lack of data and information required to apply the method, more complex mathematical procedures and the difficulty of interpreting the results. Therefore, in the present study, we use simple techniques and indices that do not rely on parametric conditions, preserve the ordinal nature of the data, making them easy to interpret and compare (Cerchiello and Giudici 2012). Indicators based on the shape of the distributions and variability (heterogeneity) were therefore used to examine the relative importance of the Iron Triangle factors. To assess the relative importance, the Quantile Based Index (QBI), based on quantiles and frequency excesses of the cumulative distribution function, was used by Cerchiello and Giudici (2012). Besides this index, the Stochastic Dominance Index (SDI) based on the cumulative distribution function and the Leti Index was applied.

QBI is formulated as follows (Cerchiello and Giudici 2012): 1 QBI=∑k=1Kqk+[ 1−∑k=1K−1(Fqk−kK)50(K−1) ] \[QBI=\sum\limits_{k=1}^{K}{{{q}_{k}}+\left[ 1-\frac{\sum\nolimits_{k=1}^{K-1}{\left( {{F}_{qk}}-\frac{k}{K} \right)}}{50\left( K-1 \right)} \right]}\] where K is the number of points of the measurement scale, F(k) is the cumulative distribution function, ∑k=1Kqk\[\sum\limits_{k=1}^{K}{{{q}_{k}}}\] is the sum of the K points scale that contains the predefined quantiles, and ∑k=1K−1(Fqk−kK)\[\sum\limits_{k=1}^{K-1}{\left( {{F}_{qk}}-\frac{k}{K} \right)}\] is the sum of the total frequency excesses at each predefined quantile, whose normalisation is obtained by dividing it by its maximum. We can normalise the index to a [0;1] range with the following transformation: 2 QBI′=QBI−ab−a \[QB{I}'=\frac{QBI-a}{b-a}\] where ‘a’ is the minimum and ‘b’ is the maximum value of the QBI, which depends on the number of categories of the measurement scale.

Based on the cumulative distribution function, which is a monotonously increasing function between 0 and 1, a summary index (SDI) can be calculated as follows: 3 SDI=∑k=1KFk \[SDI=\sum\limits_{k=1}^{K}{{{F}_{k}}}\] where F_k is the cumulative distribution function and K is the number of classes. The minimum value of SDI is 1 when all data points are in the highest class, and the maximum is K when all data points are in the lowest class. We can normalise SDI by dividing it by its maximum value, K. In this case, the minimum value of the normalised SDI is 1/K, so we cannot compare measurement scales with different category numbers. We transformed the SDI to the [0;1] range. Thus, we can compare the SDI’ with different numbers of class measurement scales and with other indexes: 4 SDI′=SDI−1K−1 \[SD{I}'=\frac{SDI-1}{K-1}\]

Regardless of the number of points on the measurement scale, SDI’ = 0 if all data fall in the largest class and 1 if in the smallest. SDI’ = 0.5 if the distribution of data is entirely symmetric across categories. For example, for a three-item scale, SDI’ = 0.5 if all the data are concentrated in the middle class, but the index value is 0.5 if no data are in the middle class and half the data are in the smallest and half in the largest class. In addition, a qualitative ranking has also been provided based on the median and the Leti index (Lorenzini and Cerchiello 2013; Mussini 2018). Leti index is defined as: 5 L=2∑k=1K−1Fk(1−Fk) \[L=2\sum\limits_{k=1}^{K-1}{{{F}_{k}}\left( 1-{{F}_{k}} \right)}\]

The L = 0 if frequencies are concentrated in one category, regardless of which class contains all the data. Leti index L = (K-1)/2 if heterogeneity is highest when frequencies are equally distributed between the lowest and the highest categories. Leti index can be normalised by dividing L by 2(K-1) (Mussini 2018). As with the other indices, the normalised Leti index is denoted by L’. While the SDI index is a second-order stochastic dominance measure, ideal for comparison purposes, the Leti index aims at measuring heterogeneity between statistical units (Lorenzini and Cerchiello 2013).

The research addresses selecting suitable metrics as well. It acknowledges Likert scale limitations and employs alternative indices like QBI, SDI and the Leti Index. Additionally, a new index to characterise overall project performance is proposed. A significance level of 0.05 is utilised for the analysis, corresponding to a 5% chance of observing a result as extreme as, or more extreme than, the one obtained, assuming that the null hypothesis is true.

Our research focuses primarily on assessing the success of projects, and TCQ elements were tested in several ways. Data were collected through an online questionnaire survey between November 2019 and August 2022. A total of 372 respondents completed the questionnaire, of which 354 were evaluable for the research. Of the total, 18 questionnaires were not suitable for processing and were therefore not included in the analysis. The research is targeted at an indefinable population, and the sampling was done using a snowball sampling method, which is not statistically representative. Industry professionals contacted by the PMI Budapest Chapter contributed voluntarily to participate. Additionally, participants who were enrolled in business master’s programmes at the Budapest University of Technology and Economics while actively engaged in project environments provided data for the study. Consequently, the sampling method influenced the primary characteristics of the respondents’ backgrounds. The exploratory analysis aimed at getting to know the population and identifying the factors that caused its success or failure. The descriptive statistical processing of the responses shows that a wide range of sectors was reached, from small and medium enterprises to large companies, from small projects with 1–9 people to large projects with >250 people, and also by industry classification, from services to manufacturing, from health to finance, covering many other sectors. Data analysis and visualisation were mainly carried out using Minitab and Statistica statistical software and, to a lesser extent, other statistical and spreadsheet software.

As a first step in the analysis, a descriptive statistical analysis of the projects was carried out based on the 354 evaluable responses. A wide range of statistical methods were used to systematically organise, simplify, review, and present the data. Data on the general characteristics of companies and projects are presented in Tables 1 and 2 (in percentage). Almost half of the projects (47%) had a budget of <$1 million, 20% had a budget of $1–10 million and 10% had a budget of >$10 million. The average project budget was around $5 million, and 60% of the projects ended in profit. About 58% of the organisations implementing projects belong to the business, 22% to the public (government) and 14% to the NGO (Non-Governmental Organization) sectors. The vast majority of the respondents had been directly involved in projects, with the most significant proportion (39%) as team members with an average of 1.7 years of project management experience, and 40% as project managers, project management team members, or team managers with an average of 3.7 years of project management experience. Using Minitab software notation, the percentage of data missing in a given category when displaying data is asterisked (N*).

The survey asked respondents to rate the success of projects based on traditional project evaluation criteria (time, budget and compliance). The results are shown in Table 2.

As expected, the three criteria were mainly rated as medium by respondents, except time, where the category ‘Late’ was the most frequent. The median is also in this category. When comparing the project plan, the data show that a ‘typical’ project is slightly behind schedule, is on budget and meets quality expectations. However, it is also clear that, apart from the quality criterion, where the proportion of projects of better (19%) or worse (22%) quality than specification is almost equal, only a tiny proportion of projects perform better than planned in terms of budget and time. This would be a good indicator of realistic project design if the projects were running behind schedule or over budget by a similar percentage.

However, the data do not show this. Only 13 projects (3.67%) were completed ahead of schedule, 34% were completed on time and 62% were delayed. The distribution is not as asymmetrical for budget as for time, but the situation is similar. Leaving aside the quality factor for now, in summary, only a small proportion of projects were completed earlier or spent less than planned. Far more common are overruns on time, on budget, or both.

Based only on the descriptive statistical results, the quality shows exactly the symmetry that was missing above for time and cost. From the point of view of judging the success of projects, quality appears to be a ‘neutral’ evaluation factor in the ‘Iron Triangle’, when time and cost exceed the planned project performance. This research focuses on the impact of traditional project evaluation TCQ factors on economic performance. Although descriptive statistical indicators show that projects were typically completed to the expected quality but late or over budget, 52% of the projects examined were profitable, and only 34% were unprofitable. However, the proportion of non-responders about their economic performance (13.5%) is relatively high. If the latter is ignored, where economic performance is known, 60% of the projects are profitable, and 40% are not.

We are examining the impact on economic performance and the relationship between the TCQ elements. The RQ focuses on the impact on economic performance. To investigate this, a wide range of statistical methods, from cross-tabulation analysis, through rank correlation analysis and multi-correspondence analysis, to the use of classification and regression trees, have been applied, as already described in the methodology section. Following the descriptive statistical analysis, the question arises whether and how the classical project evaluation criteria together indicate the economic effectiveness of a project and whether one of them is more important than the others.

To test this, as a first step, we created a PUI from the elements of the Iron Triangle to characterise the overall performance of the projects, which was then scaled by the sum of the three factors after scaling the textual assessment of the TCQ factors with numbers. For each factor, the ‘worse’ performance from the customer’s point of view was given the lower value. At the time, ‘Much late’ and ‘Much over budget’ scored 1, while at the other extreme, ‘Much earlier’ and ‘Much under budget’ scored 5. On-time and on-budget performance is the middle score of 3. The scale scores 1–3 for quality, with one being ‘Imperfectly’ and three being ‘Better’. A well-planned and implemented project has a PUI of 3 + 3 + 2 = 8. At one extreme, PUI = 3, when the project product is delivered with significant delays, well over budget and of poorer quality than expected. The other extreme is PUI = 13, representing the best-performing project in all TCQ factors. Table 3 shows the PUI values of the projects together (330 sample size due to missing answers) and grouped by dichotomous categories of question Q6 about project economic profit, Q6 = ‘Yes’ or ‘No’. Figure 1 shows the relative frequency of groupings by ‘Yes’ and ‘No’ categories of economic profit.

Fig. 1:

Graphical distribution of PUI-values by economic benefit (Q6). PUI, Project Unified Index.

According to the results of one-variable descriptive statistical analyses, the mode of the projects based on the PUI indicator is 7, and the median is also 7, which is also true for the groups based on economic profit. Analysing the TCQ indicators shows separately that projects are typically more likely to be late and over budget than on budget. Based on the PUI score, these effects appear consistent. They do not compensate for time delays by better quality or budget savings.

At first glance, the graphical representation does not show any significant difference between the PUI values of the projects based on economic performance. In Figure 1, the solid blue line indicates the relative frequencies of PUI scores for profitable projects, whereas the red dashed line indicates the relative frequencies of PUI scores for unprofitable projects. However, the figure already shows the difference in the shape of the distribution. For both groups, the ‘Yes’ distribution (continuous line) tends to be more frequent for higher values, whereas the ‘No’ distribution tends to have a less relative frequency for higher values. Excluding the mode and median value of 7 in both groups, the interval 3–6 contains 26.1% of the ‘Yes’ category and 43.4% of the ‘No’ category. The interval 8–11 has 35.9% of the ‘Yes’ category and 20.5% of the ‘No’ category. The differences are almost identical, 17.3% for the below median and 15.4% for the above median categories, but with the opposite sign.

A Mann–Whitney U test was performed to determine whether the differences between the two groups of economic outcomes shown in Figure 1 are statistically significant. The computed results of the test are given in Table 4.

The test shows a somewhat surprising result compared to previous expectations. Looking at the variables individually, the cost variable directly related to profit is not significantly different (p-value >40%) for profitable and non-profitable projects. Quality and time are statistically marginal. Both variables have p-values >2%, <5% but >1%, and the p-values adjusted for tied values are around 1%. However, for the PUI variable showing the combined effect of the three factors, the test rejects the null hypothesis, i.e. the coincidence of the means (medians). The p-value of 0.0008 (0.0011 without correction) is very small at a 5% significance level. However, there is a significant difference between the PUI values of projects that generate economic profit and those that do not. The conventional TCQ scores are significantly different in the two groups. Based on the data in Table 3, the average ranking numbers calculated from the sum of ranks shown in Figure 1 and Table 4 (average ranking number (NO) = 124.7 <average ranking number (YES) = 157.4), the PUI values of loss-making projects are lower than those of profitable projects, i.e. they perform worse than profitable projects in one or more of the TCQs.

5.1

Examining the impact and importance of TCQ variables

The main objective of our study is to examine which, if any, of the Iron Triangle elements have the most influence on economic performance in relative terms. Grouping the data in terms of economic outcomes (‘Yes’ or ‘No’ categories), we defined the indices and compared the changes in the indices in the two groups. The results are shown in Table 5. The table shows the relative frequencies (in%) by TCQ factors, with separate column headings ‘Yes’ and ‘No’ for profitable and unprofitable projects, respectively.

Tab. 5:

Change of indices by Q6

Numeric label	Time		Cost		Quality
	Yes	No	Yes	No	Yes	No
1	10.87	18.18	5.08	8.40	16.58	29.66
2	45.65	51.24	31.64	31.09	62.98	55.93
3	39.13	27.27	53.11	52.94	20.44	14.41
4	3.81	3.31	9.60	6.73	–	–
5	0.54	0.00	0.57	0.84	–	–
Median	2	2	3	3	2	2
QBI’	0.524	0.424	0.573	0.571	0.735	0.715
SDI’	0.656	0.711	0.578	0.599	0.481	0.577
L’	0.390	0.393	0.378	0.394	0.602	0.664

QBI, Quantile Based Index; SDI, Stochastic Dominance Index; L, Leti Index.

As seen from the means, there is no difference between the two groups. The medians of the two groups are equal for all factors regardless of the group breakdown. The two groups have relatively small differences, but as the PUI indicator has shown, there is an overall significant difference between the TCQ factors for profitable and unprofitable projects. Consistent with previous results, the cost variable indices have hardly changed, with budget adherence (or non-adherence) not being the leading cause of the difference in economic performance. The distribution of projects by budget is basically the same for both groups. The QBI index remains the largest for the quality factor, with the median value for economic performance being the dominant factor in both groups, where the concentration of data remains the largest. Nevertheless, the SDI’ and L’ indices also indicate a change in asymmetry. The difference in the value of SDI’ is 0.1, and the value of the indices varies around 0.5. For the economically non-profitable group, the index is almost 0.6, indicating a shift towards the first, i.e. ‘imperfectly’ category, whereas in the profitable group, on the contrary, the index value has become <0.5, indicating a smaller but more asymmetry towards the ‘better’ category. The L’ index also indicates a similar asymmetry shift. The index increased from L’ = 0.602 for the ‘Yes’ group to L’ = 0.664 for the ‘No’ group, indicating a higher proportion of category one than the other group. For Time, the QBI’ index changed the most. The index is 0.1 lower in the ‘No’ group, indicating a shift in relative frequencies towards higher scale values in the ‘Yes’ group. Since the first three of the five categories saw the most substantial change in relative frequencies, the SDI’ indices changed only slightly, and the L’ index remained essentially unchanged. Based on these findings, we believe that the Iron Triangle factors of quality and time performance play a decisive role in economic performance. The changes indicated by the indices are illustrated by the difference in the relative frequencies of the two groups (‘Yes’–‘No’ differences) in Figure 2.

Fig. 2:

Differences in the relative frequencies of the two groups (‘Yes’–‘No’).

The most considerable difference in quality is in category 1 in the ‘No’ group. In this category, the proportion of ‘Imperfectly’ projects is 13% higher, whereas in the other two categories (‘Exactly’, ‘Better’), the difference is already positive in favour of economically profitable projects. The situation is similar over time. Most economically successful projects are completed on time (category 3, ‘On time’). Calculated from the absolute value of the differences between the corresponding categories of the two groups, the average difference in relative frequencies for the three factors is 5.2% for Time, 1.4% for Cost and 8.7% for Quality.

5.2

Importance of TCQ factors with C&RT

To confirm the validity of the above conclusions, we used a Classification and Regression Tree (C&RT) procedure, the main objective of which is to group the observations, in our case in terms of economic profit as the dependent variable, in such a way that the groups are as homogeneous as possible. The operation of the classification trees is based on statistical procedures, which are not the purpose of this paper.

Classification procedures are often used to build forecasting models. However, this study aims to identify the determinants of the TCQ factors. The main question is in which order and according to which categories the C&RT method forms subgroups of the data. For this reason, no large size tree was constructed. The procedure was performed using the Statistica for Windows program, and the resulting tree structure is shown in Figure 3. As shown in the figure, the first grouping factor is quality. The ‘Better’ and ‘Exactly’ categories are more representative of economic gains, whereas the ‘Imperfectly’ category is more representative of economic losses.

Fig. 3:

Tree graph for Q6 with C&RT.

Projects are assessed primarily on an attribute-based, such as a TCQ basis. These are the most critical attributes for project managers. A comprehensive examination was conducted during this empirical study, encompassing the following questions. What are the particular traditional success factors linked to economic benefits in projects? Is there a predominant factor with a more significant impact?

The projects seem to be the same in terms of mean values, but after a more profound analysis of profit, clear differences can be found between projects with and without profit. One of the novelties of the analysis is that we can answer the above questions with inputs that can be ranked on a sliding scale using appropriate techniques. The finding diverges from influential definitions that typically regard time and cost as constraints and essential factors for project success (Wright 1997; Shenhar et al. 2001; Ashkanani and Franzoi 2022). Although cost remains directly related to economic performance, our study indicates that quality has a more significant impact on generating economic benefits. We believe that this is possible because the cost is well quantifiable, measurable, and easy to monitor, allowing for increased attention during project implementation. In contrast, quality can only be measured at a basic level, primarily focusing on compliance only. However, this discrepancy underscores the evolving understanding of project success criteria and the increasing recognition of quality as a primary determinant of project success. Moreover, the literature often includes quality as a primary success criterion, albeit placing a higher priority on time and budget delivery (Atkinson 1999; Chan et al. 2002). The findings challenge this hierarchy by demonstrating the superior influence of quality on economic benefits in projects. Further research is needed to measure project quality at a higher level and express it in terms of some indicators. This phenomenon may be the reason why quality emerges as the most critical determinant of economic benefit. However, it can be argued that if time and cost are under strict control at all times, the profitability of a project may depend on how organisations manage quality. So, quality can be the determining factor.

This exploratory research demonstrates that the importance of quality is much greater than previously thought. Therefore, the key direction for future research will be to quantify and, where necessary, redefine quality in all industries in a customer-centric way.