Development of a Machine Learning-Based Model for Predicting the Incidence of Peripheral Intravenous Catheter-Associated Phlebitis

Peripheral intravenous catheters (PIVCs) are the most commonly used invasive medical devices in hospitalised patients, especially critically ill patients in intensive care units (ICUs) [1]. However, PIVCs are associated with different types of complications, with phlebitis being the most common [1–2]. PIVC-associated phlebitis is a serious healthcare complication because it not only causes patient discomfort but can also result in infections or other outcomes, thus resulting in prolonged hospital stays and increased expenditure [3,4,5,6,7,8,9,10,11,12,13,14]. The early identification and accurate assessment of PIVCs is essential to prevent the occurrence of PIVC-associated phlebitis. Machine learning techniques have been established as a reliable and robust tool to predict outcomes in emergency and critical care settings [15,16,17,18,19,20,21,22]. Implementing machine learning (ML)-based predictive modelling in ICUs with electronic medical records could improve healthcare quality by alerting medical staff of impending complications in advance, and the timely removal of symptomatic PIVCs is important to prevent the occurrence of phlebitis [5]. The Infusion Nurses Society and National Health Service hospitals in England (epic3) do not recommend the routine replacement of PIVCs but recommend replacement if clinical findings of suspected infection, such as phlebitis, or other complications are observed [23,24]. However, given that phlebitis has resulted in a need for skin grafting, progressed to necrosis, and increased the risk of death in severe cases [11], a predictive model for phlebitis would aid in the removal of PIVCs when clinically indicated rather than at predesignated time periods. Although several studies have investigated the risk factors for the occurrence of phlebitis, few models have been developed to predict PIVC-associated phlebitis [25]. To bridge this knowledge gap, this study utilised the AMOR-VENUS database, which is an epidemiological database of PIVCs in critically ill patients [1]. The current study aimed to develop several ML-based models by using previously identified risk factors to predict the occurrence of PIVC-associated phlebitis [1,14] and to validate and compare the predictive performance of these models.

The RSF model for the survival time analysis of phlebitis occurrence showed relatively high prediction performance compared with the COX model. However, no significant differences in prediction performance were observed among the models with phlebitis occurrence as the binary outcome. Factors with high predictive importance included inserted site, catheter material, age, and nicardipine. Predictive models for the occurrence of PIVC-related phlebitis in orthopaedic patients have been reported [25]. However, to the best of our knowledge, there has been no report of predictive models for the occurrence of PIVC-related phlebitis in critically ill patients. Risk factors for phlebitis may vary depending on the clinical setting [7,11,14]. Therefore, it is inevitable that a prediction model of PIVC-related phlebitis occurrence should be developed for each group of target patients, such as critically ill patients, orthopaedic patients, and paediatric patients. The previous study used sophisticated statistical methods (Bayesian regression model) to create a prediction model for PIVC-related phlebitis [25], but the target population was limited to orthopaedic patients. Unlike patients managed in general wards, critically ill patients admitted to the ICU are administered several drugs, and it is necessary to consider the effect of those drugs in the prediction model. However, the previous prediction model for orthopaedic patients did not include enough drugs as risk factors, thereby making it difficult to apply the model to predict the occurrence of phlebitis in critically ill patients. In addition to the LR model that treated the outcome of phlebitis as a binary variable, this study also examined a predictive model for time-to-event outcome that considers the time until the occurrence of phlebitis. The notion of time is very important in the outcome of phlebitis, and the outcome of peripheral venous catheter complications, such as phlebitis, should be examined in terms of time-to-event outcomes [7,8,14]. To date, there have been no reports of studies using time-to-event outcomes in the development of predictive models for intravascular catheter-related complications. Therefore, our model for predicting the occurrence of phlebitis in critically ill patients can be considered novel.

Owing to a lack of predictive models for phlebitis in critically ill patients, it was unclear what type of model building method should be used to create predictive models. Bayesian regression model was used to construct a prediction model for PIVC-related phlebitis in orthopaedic patients, as mentioned above [25]. Several prediction models have also been reported to predict thrombotic complications in peripherally inserted central venous catheter (PICC) [42,43,44,45,46] and central venous catheter (CVC) [45, 47,48,49]. Moreover, LR models have also been used in most of the abovementioned studies to create prediction models. Regression models assume linearity between risk factors and outcomes. However, given that the relationship between phlebitis and risk factors is not necessarily based on linearity, creating predictive models using LR models or COX models may not be sufficient as methods for the prediction of occurrence of PIVC-related phlebitis. Considering that ML enables the construction of predictive models that do not assume linearity between exposure variables and outcomes, it has been widely used in recent years, and the fields of emergency medicine and intensive care are no exception [15,16,17,18,19,20,21,22]. The ML models mainly used in the articles reported to date are LASSO, RF, and gradient boosting decision tree [15,16,17,18,19,20,21,22,23]. In the current study, we used these three models for binary outcomes along with the RSF for time-to-event outcomes to construct prediction models [15,16,17,18,19,20,21,22, 34]. Given that each model has its own characteristics, it is important to compare which method of model building is better. The predictive performance of the previously reported models for complications of intravascular catheters was 0.716 for phlebitis in PIVC, 0.65–0.82 for venous thromboembolism in PICC [42, 44–45, 48], and 0.69–0.80 for catheter-related blood stream infection in CVC [43, 46, 49]. On the other hand, in the current study, the predictive performances with the c-statistic of the model for PIVC-related phlebitis in critically ill patients were 0.68 (95% CI 0.62–0.73) for RF and 0.66 (95% CI 0.60–0.71) for RSF. It is difficult to compare the results of this study with those of previous studies because of the differences in the patient backgrounds. The prediction performance of a clinical prediction model is considered moderate if it exceeds 0.7 [50]. The prediction performance of the model developed in this study was poor because of several factors that contributed to the development of PIVC-related complications, thus making it difficult to accurately predict the occurrence of PIVC-related outcomes. Although the number of factors used to predict outcomes is large due to the various types of drug administered in critically ill patients, there are roughly 10 non-drug predictors, and these machine learning predictive models are likely to be easy to use clinically. However, factors that influence PIVC-related complications can often be easily modified, such as PIVC insertion site and catheter gauge. Therefore, even if the predictive performance is not very high, the predictive model itself may be clinically useful because interventions can be easily implemented. Additionally, although the number of factors used to predict outcomes is large due to the various types of drugs administered in critically ill patients, there are roughly 10 non-drug predictors, and these machine learning predictive models are likely to be easy to use clinically. The benefits of creating a predictive model for development of PIVC-related phlebitis using ML includes the ability to identify high-risk patients and optimization of preventative strategies for such patients. Additionally, the model can be used to monitor the effectiveness of interventions and for the evaluation of safety and efficacy of new treatments. This can ultimately lead to better patient outcomes and reduced healthcare costs. Therefore, the predictive model itself may be clinically useful even if the predictive performance of the created model is not very high.

The benefits of creating a predictive model for development of PIVC-related phlebitis using ML includes the ability to identify high-risk patients and optimisation of preventative strategies for such patients. Additionally, the model can be used to monitor the effectiveness of interventions and for the evaluation of safety and efficacy of new treatments. This can ultimately lead to better patient outcomes and reduced healthcare costs. Furthermore, this study was able to create a predictive model for phlebitis development by considering the survival time. By evaluating the predictive models for occurrence and survival time, it was possible to create a predictive model that would be easier to use in clinical practice. Although the number of factors used to predict outcomes is large due to the various types of drug administered in critically ill patients, there are roughly 10 non-drug predictors, and these machine learning predictive models are likely to be easy to use clinically.

This study had a few limitations. First, the predictive performance of the model for phlebitis was approximately 0.7 on the AUC scale, which was not indicative of a high predictive performance. However, various factors affect the occurrence of IV catheter-related complications, not just phlebitis, and it may be difficult to achieve the same high prediction performance as other prediction models [21–22]. Although we incorporated all the previously reported risk factors for phlebitis into our model [3,4,5,6,7,8,9,10,11,12,13,14], there may still be unknown risk factors. Second, the predictive models were not validated externally. Although validation cohort were created within the same cohort by using the cross-validation method, it may not have been sufficient. External validation in a different cohort is needed to assess the generalizability and validity of the predictive models developed in this study, particularly for the random forest model which may be prone to overfitting based on the high c-statistic in the development cohort. Third, among the several factors incorporated into the predictive model in this study, the drug factor may have been inadequately handled. Medications are known to have a significant effect on the occurrence of PIVC-related phlebitis, and critically ill patients receive a variety of medications. The influence of medications on PIVC-related phlebitis may not only depend on their administration or absence but also on factors such as dosage, duration of administration, and initiation and termination times. In the prediction model created in this study, we only considered the presence or absence of medication administration and did not account for the potential effect of medications as time-dependent factors. Considering that critically ill patients are likely to receive a greater variety of medications, incorporating the influence of medications as time-dependent factors into the prediction model could potentially lead to improved predictive performance. Finally, some of the factors incorporated into the predictive model in this study, particularly those related to catheters, had high missing rates. Although multiple imputation method was used, it was not sufficient, and the predictive model may not have accurately predicted the outcomes.

The RSF model for the survival time analysis of phlebitis occurrence showed relatively high prediction performance compared with the COX model. However, no significant differences in prediction performance were observed among the models with phlebitis occurrence as the binary outcome. Further investigations are required to develop models that accurately predict PIVC-related phlebitis in critically ill patients.