Precise modeling of patient states in critical care — essential for timely intervention and outcome prediction — remains constrained by two fundamental challenges: (1) existing models struggle to integrate heterogeneous data streams (e.g., physiological signals, clinical notes, etc.), and (2) ground-truth labels for key states like sedation levels rely on sparse, subjective nurse assessments collected intermittently. To address multimodal integration, the team developed MedTsLLM, a framework that aligns numerical time-series with unstructured clinical text, by augmenting large language models (LLMs) to interpret a patient's physiological signals within the context of their medical records. To address label sparsity, the sedation scoring work applies machine learning to continuous physiological signals such as heart rate variability, generating high-fidelity proxies that replace intermittent manual assessments with continuous, objective baselines. By improving both how models synthesize multimodal data and the temporal resolution of patient state labels, these efforts advance the capacity to track and predict patient trajectories with greater precision in critical care settings.

Publications
MedTsLLM: Medical Time Series Analysis Using Multimodal LLMs
Nimeesha Chan, Felix Parker, Chi Zhang, William Bennett, Mung Yao Jia, James Fackler, Kimia Ghobadi
IEEE Journal of Biomedical and Health Informatics
MedTsLLM: Leveraging LLMs for multimodal medical time series analysis
Nimeesha Chan, Felix Parker, William Bennett, Tianyi Wu, Mung Yao Jia, James Fackler, Kimia Ghobadi
Machine Learning for Healthcare Conference (MLHC)