Risk of Ovarian Malignancy Algorithm and Pelvic Mass Score for the prediction of malignant ovarian tumors: a prospective comparative study

Ovarian cancer is one of the most common female cancers worldwide. Frequently, ovarian malignancy in diagnosed at an advanced stage, which usually leads to poor prognosis. Early diagnosis of the malignant potential of ovarian masses allows prompt intervention, and referral if needed, which has a great impact on the overall outcome and prognosis⁽¹⁾. Still, there is no available universal screening method for malignant ovarian masses. Many researchers have tested various parameters aiming at an earlier diagnosis of ovarian masses, including clinical features, biological markers, and different imaging modalities; yet, they have failed to reach a consensus on the optimum screening method for ovarian cancer⁽²⁾. Given the compelling need for an accurate test that can differentiate benign from malignant pelvic masses, various prediction models, scoring systems, and several markers have been analyzed. Despite these efforts, a universally accepted screening test is still unavailable⁽¹⁾.

Risk of Ovarian Malignancy Algorithm (ROMA) is a test that combines serum cancer antigen 125 (CA 125) and human epididymal protein 4 (HE4) together with the menopausal status, to obtain a numerical score⁽³⁾. ROMA was approved by the FDA for distinguishing malignant from benign pelvic masses in 2011⁽⁴⁾. Based on the morphological characteristics of ovarian masses in trans-vaginal ultrasonography, different scoring systems have been proposed⁽⁵⁾. Doppler ultrasonography can also have a major role in distinguishing ovarian masses, their vasculature being devoid of muscles; malignant tumors show a higher diastolic flow and decreased impedance on Doppler examination⁽⁶⁾. The scoring system called the Pelvic Mass Score (PMS), proposed by Rossi et al., takes into account all parameters, including Sassone score, vascular distribution, resistance index, and menopausal status, all of which have been proven by various studies to have a statistically significant association with the risk of malignancy⁽⁷⁾.

This study compares the value of the ROMA and PMS scoring systems in diagnosing malignant ovarian masses.

This prospective comparative study was conducted from March 2021 to April 2022. Ethical approval was obtained from the Research Ethics Committee (REC) for Human and Animal Research at the Faculty of Medicine, Helwan University (REC-FMHU, serial number: 20-2021). A total of 258 women diagnosed with ovarian mass and eligible for a surgical intervention according to applicable institutional guidelines were enrolled in the study. Pregnant women, as well as women refusing surgery, were excluded from the study; thus, a total of 225 women were ultimately recruited into the study (Fig. 1).

Fig. 1.

A thorough history was taken, and physical examination and ultrasound assessment were done in every recruited woman before enrollment. After clarifying the study aim, all women matching the inclusion criteria were asked for an informed written consent.

Un ultrasound unit (Samsung Medison Co, LTD, Korea Model H60 or Toshiba Aplio 400, Toshiba Medical Systems, Japan) was used to perform pelvi-abdominal ultrasound (3.5 MHz convex probe) for the assessment of masses, ascites and metastases, and in virgins. Color flow Doppler was performed to measure the resistance index of the mass (Fig. 2, Fig. 3, Fig. 4, Fig. 5, Fig. 6). Trans-vaginal ultrasound (7.5 MHz transducer) was performed in non-virgin females. Preoperative venous blood samples were collected from each patient for a full laboratory panel and estimation of CA 125 level by solid phase two-site chemiluminescent immunometric assay using Immulite^® 1000 OM-MA and HE4 serum level using human WAP four-disulfide core domain 2, WFDC2(HE4), ELISA kit by ElAab^®. Each scoring system was calculated before surgery as follows:

PMS was calculated according to Rossi et al., 2011, with a cut-off value of 29⁽⁷⁾. PMS=SASS×LOGCA125×VAS×MSRI {\rm{PMS}} = {{{\rm{SASS}} \times {\rm{LOG}}\left( {{\rm{CA}}125} \right) \times {\rm{VAS}} \times {\rm{MS}}} \over {{\rm{RI}}}}

SASS – numeric value of the Sassone score.

Fig. 2.

Fig. 3.

Fig. 4.

Fig. 5.

Fig. 6.

The Sassone scoring system was calculated as shown in Tab. 1 (cutoff value 9)⁽⁸⁾.

ROMA classifies women as being at a low or high risk for malignant disease using the following algorithms:

Pre-menopausal: Predictive Index (PI) = − 12.0 + [2.38 × LN (HE4)] + [0.0626 × LN (CA125)]

The following thresholds will be selected for:

Pre-menopausal women: ○

Pp ≥12.5% = high risk of finding ovarian malignancy.

The predictive probability algorithm (ROMA) was developed from two separate studies^(3,9).

All women were subjected to a surgical intervention (according to the applicable institutional guidelines), using either open or laparoscopic technique. Histopathological examination of the removed specimens was done, and in line with the recognized gold standard, it was compared to the pre-operative diagnosis of both scoring systems.

Sample size justification: The required sample size was calculated using PASS© version 11 (NCSS, LLC. Kaysville, Utah, USA). The primary outcome measure was the accuracy of the scoring systems studied in the discrimination between women with benign or malignant ovarian masses. A previous study reported that the area under the receiver-operating characteristic curves (ROC) for the prediction of malignant pelvis masses was 0.898 for the ROMA⁽¹⁰⁾. Another study reported that the area under the curve (AUC) for the PMS was 0.96⁽⁷⁾. Thus, it is estimated that a sample size of 225 women would achieve a power of 80% (type II error = 0.2) and type I error of 0.05 (confidence level = 95%) to detect a difference of 0.062 between the AUCs for the ROC curves associated with the PMS and the ROMA. The AUCs are assumed to equal 0.898 under the null hypothesis of no difference, and to be 0.96 and 0.898 for the PMS and the ROMA, respectively, under the alternative hypothesis.

IBM© SPSS© Statistics version 21 (IBM© Corp., Armonk, NY, USA), was used for data analysis. The normality of numerical data distribution was checked by the D’Agostino-Pearson test. The ROMA value Pp of ≥12.5 for pre-menopausal women or ≥14.4 for post-menopausal women, and a PMS >29 were used as a cut-off value for the differentiation between benign and malignant ovarian masses, and were examined by the construction of two-by-two contingency tables with the histopathological diagnosis regarded as the gold-standard test. The following diagnostic indices were calculated for each diagnostic tool: sensitivity, specificity, correct classification rate, misclassification rate, false positive rate, false negative rate, positive predictive value, and negative predictive value.

Receiver-operating characteristic (ROC) curve analysis was used to examine the value of the diagnostic indices and biomarkers for the differentiation between benign and malignant ovarian masses. The best cut-off value was identified as that associated with the highest Youden’s J index.

Comparison of the sensitivities of diagnostic tools was done by compiling two-by-two contingency tables for the classification of women with malignant ovarian masses using either tool, and the discordance between both tools was tested using McNemar’s test. To compare the specificities of diagnostic tools, similar contingency tables was constructed for the classification of patients with benign ovarian masses using either tool, and the discordance between both tools was tested using McNemar’s test. To adjust for multiple pairwise comparisons of sensitivities or specificities with McNemar’s test, the Bonferroni method was used to adjust the level of significance for the number of multiple pairwise comparisons. This indicated that a two-sided p-value <0.017 was statistically significant to keep the final type I error at 0.05. For other analyses, a conventional two-tailed p-value <0.05 was considered statistically significant.

Out of the 258 women primarily enrolled into the study, 33 were excluded according to the exclusion criteria, so ultimately a total of 225 patients with adnexal mass were included in the study. Their median age was 40 years (interquartile ranging from 23 to 65 years). A total of 151 women (67.1%) were pre-menopausal and 74 (32.9%) were post-menopausal. Benign lesions were identified in 133 cases, representing 59.1%, while malignant lesions were detected in 92 cases, representing 39.9% (Fig. 1). Various histopathological types of both malignant and benign lesions are shown in Tab. 2 and Tab. 3.

Age was found to be a significant factor associated with malignancy (p <0.0001), with the median age for benign cases being 35 years (23–52), and that for malignant − 50.5 years (34–65). As for the menopausal status, malignancies were more prevalent in post-menopausal women (p <0.001). Among benign cases, 115 women (85.2%) were pre-menopausal in comparison to 20 (14.8%) post-menopausal, while among malignant cases, 54 women (60.0%) were post-menopausal, while 36 (40.0%) were pre-menopausal.

An analysis of different tumor markers and indices in both benign and malignant cases, including HE4, CA 125 as well as the ROMA, PMS and Sassone scores, showed a significant difference between benign and malignant cases (Tab. 4). Receiver-operating characteristic (ROC) curve analysis for the value of various indices and biomarkers in the differentiation between benign and malignant ovarian masses showed that PMS had the greatest area under the curve (AUC) − 0.93, followed by ROMA − 0.91 (Tab. 5). Comparison of the areas under the ROC curves for the differentiation between benign and malignant ovarian masses using various diagnostic indices and biomarkers showed PMS and ROMA to have the highest predictive probability compared to other diagnostic indices; yet, no statistically significant difference was found between the two scoring systems (p = 0.353, 95% CI −0.025 to 0.070) (Fig. 7). Comparing the sensitivities of different scoring systems showed ROMA to be more sensitive than the Sassone score (p = 0.012). PMS is also more sensitive than the Sassone score (p = 0.007), while a comparison of ROMA and PMS showed no significant difference (p = 1.000) (Tab. 6).

Fig. 7.

The frequently encountered late diagnosis of ovarian cancers with its resultant poor prognosis highlight the importance of early diagnosis of ovarian masses to enable early management, which has a great impact on the patient’s overall prognosis⁽¹⁾. Prompt diagnosis of ovarian malignancies and early referral to a gynecologic oncologist can improve the patient survival rates⁽¹¹⁾. However, an accurate method for the prediction of ovarian malignancy is still unavailable⁽²⁾. The menopausal status, age, sonographic parameters, and tumor markers, separately, might be valuable for distinguishing malignant from benign adnexal masses. Nevertheless, a scoring system integrating all these variables would be probably more accurate in diagnosing malignant masses⁽⁶⁾.

The patient’s pre- or post-menopausal status is a fundamental clinical parameter for determining the risk of malignancy⁽⁷⁾. In ovarian neoplasms, the post-menopausal status is associated with a statistically significant risk of malignancy⁽⁷⁾. This study found the menopausal status to be associated with the malignant potential of adnexal masses. Malignancy was more prevalent in post-menopausal women (p <0.001), with 60.0% of malignant cases present in post-menopausal patients.

Nevertheless, no universal test discriminating benign from malignant adnexal masses is available. A scoring system predicting ovarian malignancy would allow better preoperative counseling and preparation as well as prompt referral of patients to a specialized center⁽²⁾. CA 125 is a widely used marker for epithelial ovarian cancer (EOC). CA 125 is considered to be the best marker for the clinical follow-up of women with ovarian cancer⁽⁶⁾. CA 125 is valuable in differentiating benign from malignant ovarian masses, diagnosis of post-menopausal ovarian masses, follow-up of ovarian cancer, and detection of recurrence following surgical treatment⁽⁶⁾. However, inadequate specificity remains a problem, as many benign diseases, both gynecological and non-gynecological, are associated with elevated serum CA 125 levels. Also, it shows low sensitivity in early ovarian cancer stages. Regarding HE4, another proposed tumor marker for ovarian cancer, its levels can also be elevated due to smoking and use of oral contraceptives, and in several benign diseases such as renal failure, effusion, liver disease, lung disease, and chronic heart failure⁽¹²⁾. Compared to single marker assays, ROMA has a better diagnostic performance for EOC, with higher sensitivity and greater accuracy compared to HE4 in post-menopausal women^(13,14).

The results of the current study support the hypothesis that ROMA has a higher diagnostic accuracy compared to CA 125 or HE4 alone. Still, several cases with false-positive ROMA scores were encountered, one study suggested that patients with a false-positive ROMA score had significantly lower median serum T3 levels than those with a true-negative ROMA score, and the true-positive ROMA group also showed significantly lower serum T3 levels than in false-negative ROMA group⁽¹²⁾. This correlation between serum T3 levels and the diagnosis of malignant ovarian masses needs to be studied more thoroughly to be validated.

Sonographic features of adnexal masses have been utilized by various scoring systems to help in predicting the malignant potential of these masses⁽⁵⁾. Doppler ultrasonography also has an essential role in differentiating ovarian masses. Malignant tumors have defective muscle layers in their vasculature, and thus have a greater diastolic flow and reduced impedance. These dissimilarities from the vasculature patterns of benign masses are very useful in the diagnostic process⁽⁶⁾. One case series, involving 14,317 women and adopting a cut-off point for RI of 0.40, showed only one false-positive and two false-negative results⁽¹⁵⁾. In another study with the same cut-off point, the sensitivity was only 25% and specificity 89%⁽¹⁶⁾. A number of studies in the literature have proposed other cut-off values for RI: one study showed 58.9% sensitivity and 86.7% specificity after adopting a cut-off level of 0.56. Consequently, a standardized Doppler flowmetry with a generally accepted RI cutoff point is difficult to achieve⁽⁶⁾.

The PMS is a composite of multiple parameters having statistically a significant association with the risk of ovarian malignancy⁽¹⁷⁾. Rossi et al. detected 100% sensitivity, 93.8% specificity, 70% positive predictive value (+PV), and 100% negative predictive value (−PV) when adopting 29 as the cut-off value⁽⁷⁾, and other authors analyzing the same cut-off point showed sensitivity, specificity, +PV and −PV of 70%, 100%, 100% and 76.9%, respectively⁽⁶⁾. There are limited data concerning the PMS scoring system available in the literature. After adopting a cut-off point of 14, 95% sensitivity, 80% specificity, 82.6% +PV, and 94.1% −PV were encountered⁽⁶⁾. The current study postulated a higher cut-off value, where sensitivity, specificity, +PV and −PV were found to be 83.3%, 90.37%, 85.2%, and 89.1%.

The study showed no statistical difference between the two scoring systems, ROMA and PMS, and found both systems to have an acceptable diagnostic accuracy. Other proposed systems for the diagnosis of malignant masses including ultrasound parameters, CA 125, and the menopausal status are studied to define the malignant potential of adnexal masses. Risk of Malignancy Index (RMI) is one of these systems. In fact, the sample recruited in this study is included as a part of another ongoing study comparing the three scoring systems, with a calculated sample size of 792 participants. The study has the strength point of being prospective, and included a relatively acceptable sample size. Still, including all types of ovarian masses including both EOC and other types of ovarian cancers may be considered as a limitation due to the different nature of these tumors, and might explain the relatively high proposed cut-off values.

Both PMS and ROMA seem to be promising scoring systems for discriminating benign from malignant ovarian masses, but more research is needed to determine the optimum diagnostic pathway, especially one yielding the least false-negative results.