<div xmlns="http://www.w3.org/1999/xhtml">
The realm of critical care medicine is always waiting for the game-changing innovation – that elusive breakthrough poised to dramatically transform practice and yield remarkable results. Various contenders have come and gone: surfactant therapy, synthetic colloids, drotrecogin alfa [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_001">1</a>–<a ref-type="bibr" href="#j_jccm-2023-0017_ref_003">3</a>]. While some have been consigned to the annals of history, a select few, such as low-tidal-volume ventilation, have endured [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_004">4</a>]. Others re-emerge time and again, hoping that they’ll have an evidence-based role somewhere. The level of excitement is not always correlated with the staying power of the innovation, and sometimes the hype can overshadow the reality.
For the past decade, we witnessed repeated attempts to integrate artificial intelligence techniques into clinical practice [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_005">5</a>, <a ref-type="bibr" href="#j_jccm-2023-0017_ref_006">6</a>]. Numerous discussions about the performance of image classification or predictive models have led to minimal real-world progress. Most falter, as their performance declines when they encounter data beyond their training and validation sets [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_007">7</a>–<a ref-type="bibr" href="#j_jccm-2023-0017_ref_009">9</a>]. Fatalist claims that these models would mean the end of one specialty or another have remained just that – claims [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_010">10</a>, <a ref-type="bibr" href="#j_jccm-2023-0017_ref_011">11</a>].
There is a new contender on the block: generative AI systems. These systems are known by various, often imprecise, names such as generative pre-trained transformers, large language models, and neural networks. However, they have become almost synonymous with ChatGPT in the public’s mind. Proponents of Chat-GPT predict that it will revolutionize patient care by expertly summarizing medical records, guiding clinical decisions, and serving as a virtual consultant [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_012">12</a>]. Its achievements, such as passing the USMLE Step 1 exam, co-authoring publications, and offering medical advice, lend credence to this idea [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_013">13</a>–<a ref-type="bibr" href="#j_jccm-2023-0017_ref_015">15</a>].
Large language models (LLMs) seem to possess extraordinary natural language processing abilities. They appear to understand, summarize, and generate text independently. Consequently, LLMs could swiftly synthesize and summarize copious amounts of medical literature, assimilate data from various patient chart sources, and apply the latest guidelines to support decision-making—a computer boasting perfect memory that seemingly “comprehends” clinical context and pathology. However, we must ask ourselves: is this understanding genuine or are we dwelling in a Potemkin village?
History cautions us that enthusiasm is rarely warranted in critical care. Searching for understanding behind the painted walls, we discover something else. Answers stemming not from genuine understanding, but from predictions [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_016">16</a>]. What we find is a very sophisticated model that generates content by predicting which words are likely to appear in similar contexts. While LLMs excel when processing familiar concepts, they falter when confronted with abstract or unusual scenarios [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_017">17</a>].
The distinction between a physician and a layperson lies not in textbooks but in experience and comprehension of disease processes. Although it is improbable that AI will practice independently, we are likely to see systems employing a human-in-the-loop strategy. By blending AI’s computational prowess with human clinicians’ nuanced understanding, AI can function as a decision aid rather than a decision maker. Some practitioners have already encountered this approach in the sepsis prediction model used by major EHRs [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_007">7</a>]. However, they may have also observed its subpar performance or its tendency to “cheat”, relying on clinicians initiating sepsis treatment to predict sepsis presence [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_018">18</a>]. It is not just an example of AI gone awry, but it exemplifies the ethical and practical challenges arising from AI algorithms’ opacity[<a ref-type="bibr" href="#j_jccm-2023-0017_ref_019">19</a>]. As models grow more complex and their decision-making processes become increasingly obscure, concerns about our ability to comprehend their recommendations arise. Sometimes, there is no human-friendly justification. Mistrust in the system undermines its role. But overreliance, or blindly following the system takes the human out of the loop.
We must note AI’s potential to exacerbate existing disparities in healthcare. Bias in AI algorithms can originate from multiple sources, including biased training data, flawed model assumptions, and the impact of historical practices [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_009">9</a>, <a ref-type="bibr" href="#j_jccm-2023-0017_ref_020">20</a>]. This may perpetuate systemic inequalities, particularly for vulnerable or marginalized populations, masking the bias beneath a veneer of scientific impartiality, or hiding it behind an opaque algorithm [<a ref-type="bibr" href="#j_jccm-2023-0017_ref_021">21</a>].
This is not to imply that these systems lack tremendous potential. When implemented appropriately, they could offer substantial benefits or aid in streamlining tedious tasks. If this enables clinicians to devote more time to direct patient care, it is a victory. After all, true care and empathy cannot be programmed.
I share the enthusiasm towards AI and my intent is not to diminish its potential role. Rather, I ask that we approach this technology with the same skepticism and apply the same degree of scrutiny as we do when evaluating other medical interventions. The future of critical care will not hinge on a single, revolutionary innovation, but on incremental steps, each edging us closer to our goal: providing the best possible care. I hope that AI is one of those steps, but we should remember: you cannot program empathy and genuine care.
</div>

Artificial Intelligence: The Next Blockbuster Drug in Critical Care?

Johns Hopkins University School of Medicine

The Journal of Critical Care Medicine

This work is licensed under the Creative Commons Attribution 4.0 International License.

Artificial Intelligence: The Next Blockbuster Drug in Critical Care?

Article Category: Editorial

Data publikacji: 08 maj 2023

Zakres stron: 61 - 63

Otrzymano: 28 kwi 2023

Przyjęty: 30 kwi 2023

DOI: https://doi.org/10.2478/jccm-2023-0017

© 2023 Razvan Azamfirei, published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.