Precise modeling of patient states in critical care — essential for timely intervention and outcome prediction — remains constrained by two fundamental challenges: (1) existing models struggle to integrate heterogeneous data streams (e.g., physiological signals, clinical notes, etc.), and (2) ground-truth labels for key states like sedation levels rely on sparse, subjective nurse assessments collected intermittently. To address multimodal integration, the team developed MedTsLLM, a framework that aligns numerical time-series with unstructured clinical text, by augmenting large language models (LLMs) to interpret a patient's physiological signals within the context of their medical records. To address label sparsity, the sedation scoring work applies machine learning to continuous physiological signals such as heart rate variability, generating high-fidelity proxies that replace intermittent manual assessments with continuous, objective baselines. By improving both how models synthesize multimodal data and the temporal resolution of patient state labels, these efforts advance the capacity to track and predict patient trajectories with greater precision in critical care settings.

Publications
MedTsLLM: Medical Time Series Analysis Using Multimodal LLMs
Chan N, Parker F, Zhang C, Bennett W, Jia MY, Fackler J, Ghobadi K
IEEE Journal of Biomedical and Health Informatics.
Leveraging LLMs for multimodal medical time series analysis
Chan N, Parker F, Bennett W, Wu T, Jia MY, Fackler J, Ghobadi K
Machine Learning for Healthcare Conference.