TY  - JOUR
A1  - Vaid, Akhil
A1  - Somani, Sulaiman
A1  - Russak, Adam J.
A1  - De Freitas, Jessica K.
A1  - Chaudhry, Fayzan F.
A1  - Paranjpe, Ishan
A1  - Johnson, Kipp W.
A1  - Lee, Samuel J.
A1  - Miotto, Riccardo
A1  - Richter, Felix
A1  - Zhao, Shan
A1  - Beckmann, Noam D.
A1  - Naik, Nidhi
A1  - Kia, Arash
A1  - Timsina, Prem
A1  - Lala, Anuradha
A1  - Paranjpe, Manish
A1  - Golden, Eddye
A1  - Danieletto, Matteo
A1  - Singh, Manbir
A1  - Meyer, Dara
A1  - O'Reilly, Paul F.
A1  - Huckins, Laura
A1  - Kovatch, Patricia
A1  - Finkelstein, Joseph
A1  - Freeman, Robert M.
A1  - Argulian, Edgar
A1  - Kasarskis, Andrew
A1  - Percha, Bethany
A1  - Aberg, Judith A.
A1  - Bagiella, Emilia
A1  - Horowitz, Carol R.
A1  - Murphy, Barbara
A1  - Nestler, Eric J.
A1  - Schadt, Eric E.
A1  - Cho, Judy H.
A1  - Cordon-Cardo, Carlos
A1  - Fuster, Valentin
A1  - Charney, Dennis S.
A1  - Reich, David L.
A1  - Böttinger, Erwin
A1  - Levin, Matthew A.
A1  - Narula, Jagat
A1  - Fayad, Zahi A.
A1  - Just, Allan C.
A1  - Charney, Alexander W.
A1  - Nadkarni, Girish N.
A1  - Glicksberg, Benjamin S.
T1  - Machine learning to predict mortality and critical events in a cohort of patients with COVID-19 in New York City: model development and validation
JF  - Journal of medical internet research : international scientific journal for medical research, information and communication on the internet ; JMIR
N2  - Background:
COVID-19 has infected millions of people worldwide and is responsible for several hundred thousand fatalities. The COVID-19 pandemic has necessitated thoughtful resource allocation and early identification of high-risk patients. However, effective methods to meet these needs are lacking.

Objective:
The aims of this study were to analyze the electronic health records (EHRs) of patients who tested positive for COVID-19 and were admitted to hospitals in the Mount Sinai Health System in New York City; to develop machine learning models for making predictions about the hospital course of the patients over clinically meaningful time horizons based on patient characteristics at admission; and to assess the performance of these models at multiple hospitals and time points. 

Methods:
We used Extreme Gradient Boosting (XGBoost) and baseline comparator models to predict in-hospital mortality and critical events at time windows of 3, 5, 7, and 10 days from admission. Our study population included harmonized EHR data from five hospitals in New York City for 4098 COVID-19-positive patients admitted from March 15 to May 22, 2020. The models were first trained on patients from a single hospital (n=1514) before or on May 1, externally validated on patients from four other hospitals (n=2201) before or on May 1, and prospectively validated on all patients after May 1 (n=383). Finally, we established model interpretability to identify and rank variables that drive model predictions.

Results:
Upon cross-validation, the XGBoost classifier outperformed baseline models, with an area under the receiver operating characteristic curve (AUC-ROC) for mortality of 0.89 at 3 days, 0.85 at 5 and 7 days, and 0.84 at 10 days. XGBoost also performed well for critical event prediction, with an AUC-ROC of 0.80 at 3 days, 0.79 at 5 days, 0.80 at 7 days, and 0.81 at 10 days. In external validation, XGBoost achieved an AUC-ROC of 0.88 at 3 days, 0.86 at 5 days, 0.86 at 7 days, and 0.84 at 10 days for mortality prediction. Similarly, the unimputed XGBoost model achieved an AUC-ROC of 0.78 at 3 days, 0.79 at 5 days, 0.80 at 7 days, and 0.81 at 10 days. Trends in performance on prospective validation sets were similar. At 7 days, acute kidney injury on admission, elevated LDH, tachypnea, and hyperglycemia were the strongest drivers of critical event prediction, while higher age, anion gap, and C-reactive protein were the strongest drivers of mortality prediction. 

Conclusions:
We externally and prospectively trained and validated machine learning models for mortality and critical events for patients with COVID-19 at different time horizons. These models identified at-risk patients and uncovered underlying relationships that predicted outcomes.
KW  - machine learning
KW  - COVID-19
KW  - electronic health record
KW  - TRIPOD
KW  - clinical
KW  - informatics
KW  - prediction
KW  - mortality
KW  - EHR
KW  - cohort
KW  - hospital
KW  - performance
Y1  - 2020
U6  - https://doi.org/10.2196/24018
SN  - 1439-4456
SN  - 1438-8871
VL  - 22
IS  - 11
PB  - Healthcare World
CY  - Richmond, Va.
ER  -