Machine learning–based predictive modeling and generative AI show potential to support clinical decision-making in intensive care units (ICUs), but their use in routine clinical practice remains limited due to challenges related to reliability, transparency, and trustworthiness. A key reason is that current approaches often address only parts of the overall problem. Predictive models focus on estimating risk, while generative systems focus on presenting information, and the connection between both remains underdeveloped.
Empirical results from predictive modeling of critical complications in ICU patients illustrate the limitations of current approaches. Models trained on routinely collected clinical data achieve moderate performance and vary across prediction targets. Common challenges include class imbalance, missing data, and limited generalizability. These findings indicate that predictive outputs alone are often not reliable enough to support clinical decisions directly.
Recent developments in multimodal and temporally aware modeling aim to address some of these limitations by improving how patient data is represented. By combining structured clinical variables, high-frequency physiological signals, and unstructured text data, such models provide a more comprehensive view of patient states. The use of survival-based methods further allows risk to be modeled over time rather than as a static classification. This leads to more informative outputs that better reflect the dynamic progression of critical conditions. At the same time, the increased complexity of these models makes their results more difficult to interpret and apply in practice.
This creates a gap between increasingly detailed model outputs and their practical usability. Generative AI systems, particularly those based on retrieval-augmented generation (RAG), offer a way to address this gap by translating model outputs and patient data into structured, context-aware explanations. By grounding generated responses in established clinical knowledge, such systems can support interpretation while maintaining transparency, traceability, and auditability of AI-supported recommendations. Prototype implementations show that this approach can produce consistent and explainable outputs under practical constraints such as latency and cost, provided that system design includes controlled retrieval mechanisms and human oversight.
Taken together, these observations show that current approaches contribute to different stages of the same problem. Predictive models provide estimates of risk, multimodal methods improve how complex patient data is modeled, and generative systems support the interpretation of results. The main challenge lies not in any single component, but in how these elements relate to each other in practice.
The analysis therefore focuses on the interaction between prediction, representation, and interpretation as key dimensions of AI-supported decision-making in intensive care. It highlights that improvements in model performance alone are not sufficient and that usability, interpretability, and transparency need to be considered as first-class design objectives of AI-supported systems. In addition, the discussion addresses practical implications for the use of AI in clinical environments, including transparency, trust, and integration into existing workflows. The talk further discusses how explainable and auditable AI architectures can contribute to trustworthy decision support and strengthen digital sovereignty in data-intensive healthcare environments.
