Unsuccessful bids: Coefficient of variation of bids as indicator of project risk
e | 22 giu 2020
Article Category: Research Paper
Pubblicato online: 22 giu 2020
Pagine: 2193 - 2199
Ricevuto: 02 set 2019
Accettato: 15 apr 2020
DOI: https://doi.org/10.2478/otmcj-2020-0015
Parole chiave
© 2020 Young Joo Kim et al., published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
For a large and complex project, any responsive and responsible bidder would put together a bid that is most reasonable in her judgment. She would consume significant resources to carry out tasks such as calculations, correspondence, conversations, document study, meetings, deskwork, and travel (Laryea and Hughes 2011). However, under the typical lowest-price sealed-bid project award auction environment, once bids are opened, unsuccessful bids are typically discarded. In addition, the winning bidder is the one who is expected to underestimate the final cost more than his competitors (Chang et al. 2014). Therefore, it is likely that unsuccessful bids convey important—even more important than the successful bid—information of which the owner organization can take advantages regarding the proposed project. Information could indicate uncertainties noticed by the bidders but glossed over by the owner. For example, the scope of work or field conditions may not be clearly stated in bid documents; availability of labor, equipment or material in neighboring areas may be uncertain; or potential issues may occur in terms of right-of-way, site access, or third-party involvement (railroads, utilities. etc.) (Wright and Williams 2001). In this article, the authors documented their effort to answer the following question: Are projects that received more dispersed estimates at the bidding stage more likely to experience greater project cost changes upon completion?
This research question merits an investigation as it is critical for project owners, contractors, or any project stake-holders to identify a risky project at an early stage; if the proposed project is indicated to be riskier than others, that is, if greater cost changes are expected, the project owner may cancel the project entirely, re-invite bidders once issues are addressed, or increase contingency in the project budget. If a risky project is identified earlier by using bids, there is still time for project stake holders to take corrective actions. Therefore, the outcome of this study can be useful for owner organizations and contracting organizations to identify risky projects to take on using bidding data.
Prior research has looked into project bidding data. For example, Skitmore et al. investigated bid-spread, the difference between the lowest and second lowest bids, or money left on the table, or foregone profit, in a “lowest wins” auction (Skitmore et al. 2001). Kuprenas studied the relationship between construction cost performance and bid characteristics. The construction cost performance was measured through change orders (as a percentage of bid value), and the bid characteristics were measured through the percentage of the bid budget, the number of bidders, bid date, date since last bid, bid cost per square foot, and bid amount. The results of a series of regression analyses of 24 public sector projects from a multibillion dollar school construction program in Los Angeles, California were presented, and it was found that the percentage of bid budget, the number of bidders, bid date, date since last bid, and bid amount were correlated with change order amounts (Kuprenas 2005). Minchin et al. used contractor bid unit prices to estimate the impact of night construction on cost and productivity for transportation projects. Assuming that a contractor's bid amount reflected expected relative project productivity, it was reported that, with a few exceptions, granting contractors maximum flexibility in deciding what construction activities to be done at night would result in substantial cost savings to transportation agencies (Minchin et al. 2013).
Williams et al. extensively examined bidding data to predict final construction cost. Using data on completed projects obtained from the New Jersey Department of Transportation for 298 highway construction projects, it was reported that median bid and normalized median absolute deviation were the best predictors of completed construction cost (Wright and Williams 2001). The variability of the submitted bids did not seem to be an indicator of the completed construction cost (Williams 2002). There appeared to be some relationship between the bidding ratios (second lowest bid ratio, mean bid ratio, and maximum bid ratio) and the completed cost of the Texas Department of Transportation highway construction projects; the bidding ratios appeared to increase as uncertainty about the completed project cost increases. Higher values of the bidding ratios were observed both for projects that were constructed at a much greater cost than the original low bid amount and for projects that were constructed for amounts significantly less than the original low bid amount (Williams 2005). In particular, similar to the aim of this study, it was found that the value of the ratios [second lowest bid ratio, mean bid ratio, maximum bid ratio, and the coefficient of variation (CV)] tended to be larger for projects where the completed cost deviated significantly from the original low bid (Williams et al. 2005; Williams 2007).
The association between the number of bidders and competition has also been examined. Carr investigated bid price competition measured through pre-bid project estimates, actual bid prices, and number of bidders. By quantitatively analyzing public project bids under a condition of free, open, and unfettered competition, the impact of reduced competition on project bid prices was studied. After examining a single building type, designed by a single firm, with pre-bid estimates prepared, over a limited time period, it was found that reducing the number of bidders would result in increased project bid prices (Carr 2005). At the same time, Li et al. reported that owners could offset the effect of a reduced number of bidders in part by scheduling their projects to periods of construction slowdown or by bundling their projects together into a single larger project, after analyzing building projects in Utah (Li et al. 2008).
Other researchers have studied bidding data with various interests. Shrestha et al. analyzed 435 bids on 113 public street projects and showed a strong correlation between the lowest bid price and the final construction cost but no correlation between the lowest bid price and the construction cost growth; also, the indication was that the higher the number of bidders, the lower the price of the bid is (Shrestha and Pradhananga 2010). Baek et al. used historical bid data of highway projects in the State of Georgia between 2005 and 2015 and reported that total contract price, number of pay item, duration, the annual total value of projects, architecture billings index, annual total number of projects, asphalt cement price index, and number of bids were significant factors that contributed to the degree of competition/number of bidders (Baek and Ashuri 2018), and that the following variables had statistically significant relations with submitted unit price bids: the quantity of the bid item, number of nearby asphalt plants, total contract price, Georgia asphalt cement price index, producer price index for construction machinery manufacturing, GDP, crude oil prices (WTI), ratio of bid item, pavement length, population, hauling distance between quarry and asphalt plants, number of bidders, total monthly asphalt size of resurfacing and widening projects awarded in the same month at the level of the county, and number of hires (Baek and Ashuri 2019).
Several researchers, especially Williams and colleagues, have investigated bidding data to predict project final costs. However, a strong relationship between an award bid and its final cost may be a natural evidence of a project team's earnest efforts to deliver a project on budget. The purpose of this article was not to estimate project final costs from bidding data; this study intended to validate the utility of bids to identify projects that were more likely to experience greater cost changes during execution.
Collective intelligence is a framework for the investigation of collectives. Collectives are “systems where each agent aims to optimize its performance, but there is a well-defined set of system-level performance criteria.” There is no centralized control; therefore, underlying systems, agents, or neurons modify their behavior to maximize rewards. It is a behaviorist approach; the framework is broad enough to encompass the real-world collectives that are often too complex to represent in a tractable model (Wolpert et al. 1999; Tumer and Wolpert 2004; Wolpert 2004). Collective intelligence framework was suitable for this study as:
In bidding for winning a project, there is no centralized control, that is, a bidder is free to submit any bid at his discretion; bidders can be seen as agents/underlying systems/neurons trying to maximize reward, that is, winning a contract; there are system-level performance criteria, that is, the lowest price; and collective efforts by bidders foster good global performance, that is, economic delivery of a project.
According to the Encyclopedia of Research Design, the CV, a primarily descriptive statistic, eliminates the unit of measurement from the standard deviation of a series of numbers by dividing it by the mean of the series of numbers (Salkind 2010). CV is useful in comparing the relative variability of positive random variable distributions and has been widely used in many scientific areas; CV is appropriate for comparing the risk of alternative investments, and a lower CV ratio represents a lower risk (Curto and Pinto 2009). CV has been widely used to measure relative risk in the areas of engineering, medicine, agricultural economics, archeology, and financial management (Weber et al. 2004). In finance, for example, CV was related to evaluating project risks in an uncertain situation (Brief and Owen 1969); CV was used as an indicator of risk in bank lending environments to show that a bank with a relatively high CV for the environmental portfolio was likely to fail at a higher rate (Gunther and Robinson 1999); and, CV was used, among other descriptive statistics, to investigate risk, return, and portfolio diversification in major painting markets during the period 1976–2001 (Worthington and Higgs 2004). In sociology, at the same time, CV was applied to compare different nations’ geographic mobility and homicide rates by regions in the USA (Martin and Gray 1971), and it has also been applied as an index of diversity (Bedeian and Mossholder 2000).
More relevant to this study, CV was used to measure the spread of the submitted bids (Williams et al. 2005). CV is an indicator of bidders’ uncertainty about the value of a project. It can be postulated that a project with a high CV indicates that there is considerable uncertainty among bidders about the cost of the project, whereas a project with a low CV shows little disagreement among bidders (Wright and Williams 2001). As the authors aimed to explore the possibility of using the dispersion of bids as a project risk indicator, adopting CV was deemed justified.
Private organizations’ project cost data are usually difficult to access from outside; therefore, the authors searched for project cost data available in the public domain. From the Ohio Department of Transportation, bid estimates (Ohio Department of Transportation, no date b) and project final cost data (Ohio Department of Transportation, no date a) were obtained; then, two datasets were matched through project identification numbers. It was decided to investigate projects that costed the agency more than U$5 million, based on the assumption that larger projects would have more uncertainties at their bidding stage. A total of 910 bids on 222 case projects that were completed between 2008 and 2018 were identified for further investigation.
To quantify project cost changes, the following algebraic formula was adopted (Eq. 1). The rationale behind using an absolute value is that the potential increase of the cost of a project would be considered a risk to an owner organization (Wright and Williams 2001), whereas the potential decrease of the cost of a project below the award bid would be considered a risk to a contractor for reduced revenue. Thus, regardless of positivity or negativity, a change in the cost of a project was deemed as a risk.
Then, each project's CV of bids was calculated. Twelve sole-bid projects were dropped from the dataset due to the lack of CV. The remaining 210 case projects were further processed.
Case projects were sorted from the largest to the smallest according to the value of their CV and divided into two groups: Projects with above-average CV and projects with below-average CV to compare the means of project cost changes of each group. The hypothesis was that projects with above-average CV would show a higher mean value of project cost changes, with statistical significance. Another round of analysis was conducted based on the number of bidders as a distinguishing factor for splitting the data with varying CV percentiles. Collected data were analyzed using Microsoft Excel™.
Characteristics of 210 case projects were summarized in Table 1.
Characteristics of case projects
| Number of case projects | ea. | 210 | |
| Mean, award bid | U$ | 20,916,801.39 | SD 31,430,304.83 |
| Mean, project final cost | U$ | 21,324,687.90 | SD 31,831,462.92 |
| Mean, number of bidders | ea. | 4.28 | SD 2.13 |
| Mean, CV of bids | % | 7.42 | SD 4.76 |
| Mean, project cost change | % | 5.36 | SD 5.99 |
After case projects were analyzed and CV values were calculated, an independent samples
The results of the analysis are summarized in Table 2. An independent samples
Summary of analysis results
| Number of case projects | ea. | 210 |
| Mean, number of bidders (SD) | ea. | 4.28 (2.13) |
| Mean, CV (SD) | % | 7.42 (4.76) |
| Mean, project cost change (SD) | % | 5.36 (5.99) |
| Mean, project cost change, above-average CV (SD) [A] | % | 6.49 (6.73) |
| Mean, project cost change, below-average CV (SD) [B] | % | 4.24 (4.92) |
| – | 0.01 |
Further, the authors performed another round of analysis, using the number of bidders as a distinguishing factor for splitting the data with varying CV percentiles (Table 3). In general, as the sample size decreases, the
Comparison of means of project cost changes against the number of bidders and CV percentile
| 50% | Mean, top 50% CV,% | 7.61 | 4.45 | 6.75 | 4.24 | 6.39 | 7.98 |
| Mean, bottom 50% CV,% | 3.66 | 6.20 | 3.91 | 3.93 | 3.03 | 5.03 | |
| 0.01 | 0.36 | 0.16 | 0.87 | 0.15 | 0.24 | ||
| 40% | Mean, top 40% CV,% | 7.64 | 4.62 | 6.52 | 4.26 | 6.09 | 8.94 |
| Mean, bottom 40% CV,% | 3.28 | 5.47 | 3.24 | 4.24 | 3.13 | 4.50 | |
| 0.02 | 0.70 | 0.12 | 0.99 | 0.33 | 0.14 | ||
| 30% | Mean, top 30% CV,% | 6.98 | 4.15 | 6.97 | 4.75 | 7.27 | 9.24 |
| Mean, bottom 30% CV,% | 3.36 | 5.63 | 3.20 | 5.27 | 2.94 | 2.50 | |
| 0.04 | 0.60 | 0.15 | 0.87 | 0.27 | 0.05 | ||
| 20% | Mean, top 20% CV,% | 6.11 | 4.76 | 8.62 | 3.18 | 3.99 | 6.19 |
| Mean, bottom 20% CV,% | 3.61 | 5.61 | 2.97 | 6.34 | 4.00 | 2.19 | |
| 0.31 | 0.84 | 0.15 | 0.35 | 0.99 | 0.01 | ||
| 10% | Mean, top 10% CV,% | 7.59 | 8.04 | 12.67 | 2.13 | 2.80 | 7.30 |
| Mean, bottom 10% CV,% | 4.22 | 8.54 | 3.97 | 8.58 | 2.95 | 1.17 | |
| 0.44 | 0.95 | 0.26 | 0.55 | 0.96 | 0.02 | ||
What has been observed in this investigation supports the seminal work done by Williams et al. Williams et al. analyzed highway projects conducted by the Texas Department of Transportation from 1995 to 2000 and stated that there were significant differences between (a) the means of the bid ratios including CV for groups where the completed project costs differed significantly from the original bid amount and (b) for groups where the projects were completed near the original bid amount (Williams 2005; Williams et al. 2005). The authors also made the same observation by analyzing large projects completed by the Ohio Department of Transportation from 2008 to 2018. Therefore, the phenomenon—projects with higher CV of bids experience greater project cost changes than projects with lower CV—was observed in two datasets that were from different locations and periods. Yet, it is worth noting that as Williams et al. were investigating bidding data to predict project final costs, their finding of the higher CV values occurred both for projects that had large cost increases and for projects that were completed for significantly less than the original bid amount was not greatly advantageous to the purpose of their investigation.
If information that is available as early as at bid opening can show that the project's final cost is likely to change significantly, the owner organization could use this information to better budget their financial resources and carefully monitor the project progress (Williams 2002). Therefore, the outcome of this study encourages project owner organizations to identify risky projects early and to prepare realistic contingency budgets by considering not only winning bids but also unsuccessful bids.
This study has several limitations. First of all, it can be questioned whether the dataset used in this study represents the whole construction industry. The issue is the availability of data in other construction sectors. Bidding data that private organizations have accumulated may not be easily accessible to academic researchers. Second, projects that cost less than U$5 million among the Ohio data were not included in the analysis. To address this deficiency, the researcher plans to incorporate smaller projects in a follow-up study.
In summary, the authors tried to answer a narrowly defined research question—are projects that received more dispersed estimates at the bidding stage more likely to experience greater project cost changes upon completion? A total of 210 large projects conducted by the Ohio Department of Transportation between 2008 and 2018 were analyzed. It was found that large projects that received more dispersed cost estimates at the bidding stage experienced greater project cost changes upon completion. In particular, large projects with above-average CV of bids showed greater deviations from original award amounts than large projects with below-average CV. The observation also confirmed prior research that analyzed bidding data from a different location and from a different period (Williams et al. 2005; Williams 2007). The outcome validates the utility of bids in identifying projects that are more likely to experience greater cost changes during execution.
If owner organizations or bidders learn that certain projects are more likely to experience greater cost changes during execution, would they still pursue the projects? This may be a question reserved for decision-makers. Nevertheless, it is an advantage to have time on project stakeholders’ hands so that they can review the viability of the project in advance, if the project is indicated as a risky one at the bidding stage. It is also possible to even reject all bids and to re-define the scope of the project for a new round of bidding. The same approach can be adopted by prime contractors as they receive bids from subcontractors for work packages. Therefore, it is recommendable to have an organizational policy to look whether CV of bids exceeds a certain threshold based on the organization's in-house dataset. The development of complex and mathematically rigorous models is not necessary.
To further validate the finding reported herein, implementing a larger scale investigation that includes projects by other state departments of transportation, projects by private owners, and projects delivered through different delivery methods could be considered by the research community.