<abstract xmlns="http://www.w3.org/1999/xhtml">

<sec><h3>Introduction</h3>
<p>Illness severity scoring tools, such as PRISM III/IV, PIM-3, and PELOD-2, are widely used in pediatric critical care research. However, their application is hindered by complex calculation processes, privacy concerns with third-party online calculators, and challenges in accurate implementation within statistical packages.</p>
</sec>
<sec><h3>Methods</h3>
<p>We have developed a comprehensive, open-source toolkit for implementing the PIM-3, Simplified PIM-3, and PELOD-2 scores. The toolkit includes the pim3 and pelod2 commands and is compatible with Stata versions 12 and above. It features robust data validation, error messaging, a graphical interface, and support for SI and Imperial units. The toolkit's accuracy was validated through unit testing and synthetic data, comparing results with existing implementations.</p>
</sec>
<sec><h3>Results</h3>
<p>In performance tests, the toolkit exhibited a median processing time of 21.82 seconds for PELOD-2, 14.06 seconds for PIM-3, and 9.74 seconds for Simplified PIM-3, when applied to datasets of 10,000,000 records. It consistently achieved 100% accuracy in both synthetic data tests and manual spot checks.</p>
</sec>
<sec><h3>Conclusion</h3>
<p>The toolkit decreases processing time and improves accuracy in calculating pediatric critical care severity scores such as PELOD-2, PIM-3, and Simplified PIM-3. Its application in large datasets and validation highlights its utility as a tool for streamlining pediatric critical care research.</p>
</sec>
</abstract>

Introduction
Illness severity scoring tools, such as PRISM III/IV, PIM-3, and PELOD-2, are widely used in pediatric critical care research. However, their application is hindered by complex calculation processes, privacy concerns with third-party online calculators, and challenges in accurate implementation within statistical packages.

Methods
We have developed a comprehensive, open-source toolkit for implementing the PIM-3, Simplified PIM-3, and PELOD-2 scores. The toolkit includes the pim3 and pelod2 commands and is compatible with Stata versions 12 and above. It features robust data validation, error messaging, a graphical interface, and support for SI and Imperial units. The toolkit's accuracy was validated through unit testing and synthetic data, comparing results with existing implementations.

Results
In performance tests, the toolkit exhibited a median processing time of 21.82 seconds for PELOD-2, 14.06 seconds for PIM-3, and 9.74 seconds for Simplified PIM-3, when applied to datasets of 10,000,000 records. It consistently achieved 100% accuracy in both synthetic data tests and manual spot checks.

Conclusion
The toolkit decreases processing time and improves accuracy in calculating pediatric critical care severity scores such as PELOD-2, PIM-3, and Simplified PIM-3. Its application in large datasets and validation highlights its utility as a tool for streamlining pediatric critical care research.


<div xmlns="http://www.w3.org/1999/xhtml">
<sec id="j_jccm-2023-0033_s_001"><div>Introduction</div>
<p>Outcome prognostication tools are indispensable in pediatric critical care research serving to control for the illness severity of pediatric intensive care unit patients. Several scores have been proposed, with the most notable being the Pediatric Risk of Mortality (PRISM III/IV), the Paediatric Index of Mortality 3 (PIM-3), and the Pediatric Logistic Organ Dysfunction-2 (PELOD-2) [<a ref-type="bibr" href="#j_jccm-2023-0033_ref_001">1</a>,<a ref-type="bibr" href="#j_jccm-2023-0033_ref_002">2</a>,<a ref-type="bibr" href="#j_jccm-2023-0033_ref_003">3</a>].</p>
<p>The application of each of these validated scoring systems involves complex, multi-step calculations. Online calculators are available for some of these scores, but they vary in their capacity to support multiple patients and the accuracy of their calculations. Additionally, submitting patient data to third-party websites may raise concerns about privacy and data protection. Implementing outcome prognostication scores in statistical packages is resource-intensive and prone to errors. As a result, researchers undertake custom implementation, leading to duplicated efforts and potential concerns about reproducibility if the implementation is incorrect.</p>
<p>Building on our previous work, where we detailed the development and implementation of a Stata command for calculating PRISM III and IV scores [<a ref-type="bibr" href="#j_jccm-2023-0033_ref_004">4</a>], this paper introduces a comprehensive toolkit for outcome prognostication in critically ill pediatric patients. This toolkit, including open-source implementations of the PIM-3, Simplified PIM-3, and PELOD-2 scores, aims to simplify these processes, thereby potentially reducing the challenges associated with their application in clinical research.</p>
</sec>
<sec id="j_jccm-2023-0033_s_002"><div>Methods</div>
<p>We employed the software framework described in our previous work to develop the pim3 and pelod2 commands. The commands accurately implement the methods described in the original publications [<a ref-type="bibr" href="#j_jccm-2023-0033_ref_001">1</a>,<a ref-type="bibr" href="#j_jccm-2023-0033_ref_002">2</a>,<a ref-type="bibr" href="#j_jccm-2023-0033_ref_003">3</a>, <a ref-type="bibr" href="#j_jccm-2023-0033_ref_005">5</a>]. Where necessary, each command is equipped with robust data validation, clear error messaging, a user-friendly graphical interface, and support for alternative units. This further reduces potential errors caused by inappropriate conversions. Clear and comprehensive documentation is provided for the usage of the commands. The commands are compatible with Stata versions 12 and above.</p>
<p>We have consolidated these commands into a toolkit for pediatric critical care research (<italic>pccmtoolkit</italic>). In addition to the <italic>pim3</italic> and <italic>pelod2</italic> commands, the toolkit includes the <italic>prismscore</italic> command as well as other utilities that streamline data management and processing. The entire toolkit, as well as the individual commands, are accessible on the Statistical Software Components (SSC) archive (<ext-link xmlns:xlink="http://www.w3.org/1999/xlink" ext-link-type="uri" xlink:href="https://ideas.repec.org/s/boc/bocode.html">https://ideas.repec.org/s/boc/bocode.html</ext-link>). Development versions of the toolkit are available on GitHub (<ext-link xmlns:xlink="http://www.w3.org/1999/xlink" ext-link-type="uri" xlink:href="https://github.com/razvanazamfirei/pccmtoolkit">https://github.com/razvanazamfirei/pccmtoolkit</ext-link>).</p>
<p>Each command was validated through unit testing, as well as synthetic data. The results were compared to other implementations of the score and manual spot checks were performed to ensure that calculations reflect the original methods. As described previously, we generated 10,000,000 patient records with random realistic data and predefined PELOD-2 and PIM-3 scores. Data generation was done through a separate process that did not rely on the developed command. Each command was benchmarked by performing calculations over 5 distinct sets of 10,000,000 patient records, using Stata/MP 18.0 2-cores. Results are presented as median (IQR). Institutional Review Board approval was not required.</p>
</sec>
<sec id="j_jccm-2023-0033_s_003"><div>Results</div>
<p>The run-time results for each individual command are described in <a ref-type="table" href="#j_jccm-2023-0033_tab_001">Table 1</a>. The median time required to calculate scores for 10,000,000 records was 21.82 (21.40 – 22.31) seconds for PELOD-2, 14.06 (13.65 – 14.14) seconds for PIM-3, and 9.74 (9.71 – 9.98) seconds for the Simplified PIM-3 score. Each command achieved 100% accuracy in synthetic data tests, as well as in manual spot checks.</p>
<table-wrap id="j_jccm-2023-0033_tab_001" position="float"><label>Table 1.</label>
<caption><p>Median run times for calculating pediatric severity scores.</p></caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="middle"><bold>Score Type</bold></th>
<th align="center" valign="middle"><bold>Run 1</bold> seconds</th>
<th align="center" valign="middle"><bold>Run 2</bold> seconds</th>
<th align="center" valign="middle"><bold>Run 3</bold> seconds</th>
<th align="center" valign="middle"><bold>Run 4</bold> seconds</th>
<th align="center" valign="middle"><bold>Run 5</bold> seconds</th>
<th align="center" valign="middle"><bold>Median (IQR) run time</bold> seconds</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="middle">PELOD-2</td>
<td align="center" valign="middle">21.92</td>
<td align="center" valign="middle">21.47</td>
<td align="center" valign="middle">21.82</td>
<td align="center" valign="middle">21.32</td>
<td align="center" valign="middle">22.70</td>
<td align="center" valign="middle">21.82 (21.40–22.31)</td>
</tr>
<tr>
<td align="left" valign="middle">PIM-3</td>
<td align="center" valign="middle">14.06</td>
<td align="center" valign="middle">13.59</td>
<td align="center" valign="middle">14.20</td>
<td align="center" valign="middle">13.71</td>
<td align="center" valign="middle">14.08</td>
<td align="center" valign="middle">14.06 (13.65–14.14)</td>
</tr>
<tr>
<td align="left" valign="middle">Simplified PIM-3</td>
<td align="center" valign="middle">10.14</td>
<td align="center" valign="middle">9.69</td>
<td align="center" valign="middle">9.73</td>
<td align="center" valign="middle">9.74</td>
<td align="center" valign="middle">9.81</td>
<td align="center" valign="middle">9.74 (9.71–9.98)</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Additionally, the graphical user interface developed for these commands, which is depicted in <a ref-type="fig" href="#j_jccm-2023-0033_fig_001">Figures 1</a> and <a ref-type="fig" href="#j_jccm-2023-0033_fig_002">2</a>, enables intuitive variable specification and option selection. This interface streamlines the process for users, making it more accessible and efficient, particularly for those who may not be as familiar with traditional command-line operations.</p>
<figure id="j_jccm-2023-0033_fig_001" position="float" fig-type="figure"><h2>Fig. 1.</h2>
<figCaption><p>Stata dialog depicting the graphical user interface for the PIM-3 and simplified PIM-3 commands.</p></figCaption>
<img xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="graphic/j_jccm-2023-0033_fig_001.jpg" src="https://sciendo-parsed.s3.eu-central-1.amazonaws.com/6677e24add1c3d1f8713dff5/j_jccm-2023-0033_fig_001.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Date=20240716T084419Z&amp;X-Amz-SignedHeaders=host&amp;X-Amz-Expires=18000&amp;X-Amz-Credential=AKIA6AP2G7AKP25APDM2%2F20240716%2Feu-central-1%2Fs3%2Faws4_request&amp;X-Amz-Signature=9deee02c32479bc663053776ed0ceff4595108644dbae949d21c3d734ced6c1f" class="mw-100"/>
</figure>
<figure id="j_jccm-2023-0033_fig_002" position="float" fig-type="figure"><h2>Fig. 2.</h2>
<figCaption><p>Stata dialog depicting the graphical user interface for the PELOD-2 command.</p></figCaption>
<img xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="graphic/j_jccm-2023-0033_fig_002.jpg" src="https://sciendo-parsed.s3.eu-central-1.amazonaws.com/6677e24add1c3d1f8713dff5/j_jccm-2023-0033_fig_002.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Date=20240716T084419Z&amp;X-Amz-SignedHeaders=host&amp;X-Amz-Expires=18000&amp;X-Amz-Credential=AKIA6AP2G7AKP25APDM2%2F20240716%2Feu-central-1%2Fs3%2Faws4_request&amp;X-Amz-Signature=cc76436ad2b8399746a37c5fe90c7c7f4f4ff66c1890c92d4a4c07d08c10d65f" class="mw-100"/>
</figure>
</sec>
<sec id="j_jccm-2023-0033_s_004"><div>Discussion</div>
<p>This study demonstrates the robustness and efficiency of the developed commands for calculating PELOD-2 and PIM-3 scores, indicating that the commands are highly reliable for research purposes. The median run times for the calculations suggest that these commands can process large datasets within a feasible time frame. The development of this toolkit serves as a valuable resource for pediatric critical care researchers, now offering the potential to calculate multiple types of severity scores over the same dataset. However, it is important to note that the validity of the scores depends on the underlying quality of the data. Scores may use different definitions for comorbidities and require different intervals for using clinical data. The documentation provided with the commands describes some requirements and limitations; however, it is not a substitute for accurate data extraction, or appropriate data cleaning and management.</p>
</sec>
<sec id="j_jccm-2023-0033_s_005"><div>Conclusion</div>
<p>Our work introduces a toolkit designed to enhance efficiency and accuracy in calculating the PELOD-2, PIM-3, and Simplified PIM-3 scores. Validation through synthetic data tests and manual spot checks confirmed 100% accuracy, ensuring reliability for research applications. This toolkit will serve as a resource for supporting pediatric critical care research, offering a streamlined approach to complex illness severity calculations.</p>
</sec>
</div>

Pediatric Critical Care Illness Severity Toolkit: Stata Commands for Calculation of Pediatric Index of Mortality and Pediatric Logistic Organ Dysfunction Scores

Department of Anesthesiology and Critical Care Medicine

Johns Hopkins University School of Medicine

George Emil Palade University of Medicine, Pharmacy, Science, and Technology of Targu Mures

Department of Pediatrics, Charlotte R. Bloomberg Children’s Center

Department of Physical Medicine and Rehabilitation

The Journal of Critical Care Medicine

This work is licensed under the Creative Commons Attribution 4.0 International License.

Introduction
Illness severity scoring tools, such as PRISM III/IV, PIM-3, and PELOD-2, are widely used in pediatric critical care research. However, their...

Pediatric Critical Care Illness Severity Toolkit: Stata Commands for Calculation of Pediatric Index of Mortality and Pediatric Logistic Organ Dysfunction Scores

Article Category: Research Article

Data publikacji: 30 sty 2024

Zakres stron: 16 - 18

Otrzymano: 23 paź 2023

Przyjęty: 10 gru 2023

DOI: https://doi.org/10.2478/jccm-2023-0033

Słowa kluczowe
pediatric risk of mortality, PELOD, PIM3, STATA

© 2024 Razvan Azamfirei et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Fig. 1.

Fig. 2.

Median run times for calculating pediatric severity scores.

Score Type	Run 1 seconds	Run 2 seconds	Run 3 seconds	Run 4 seconds	Run 5 seconds	Median (IQR) run time seconds
PELOD-2	21.92	21.47	21.82	21.32	22.70	21.82 (21.40–22.31)
PIM-3	14.06	13.59	14.20	13.71	14.08	14.06 (13.65–14.14)
Simplified PIM-3	10.14	9.69	9.73	9.74	9.81	9.74 (9.71–9.98)

Pediatric Critical Care Illness Severity Toolkit: Stata Commands for Calculation of Pediatric Index of Mortality and Pediatric Logistic Organ Dysfunction Scores

Article Category: Research Article

Data publikacji: 30 sty 2024

Zakres stron: 16 - 18

Otrzymano: 23 paź 2023

Przyjęty: 10 gru 2023

DOI: https://doi.org/10.2478/jccm-2023-0033

Słowa kluczowepediatric risk of mortality, PELOD, PIM3, STATA

© 2024 Razvan Azamfirei et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Fig. 1.

Fig. 2.

Median run times for calculating pediatric severity scores.

Słowa kluczowe
pediatric risk of mortality, PELOD, PIM3, STATA